TREC 2025 Proceedings
CUET-qwen14B-v5
Submission Details
- Organization
- CUET
- Track
- Detection, Retr., and Gen for Understanding News
- Task
- Question Generation Task
- Date
- 2025-08-08
Run Description
- Is this run manual or automatic?
- automatic
- Is this run based on the provided starter kit?
- no
- Briefly describe this run
- This run is designed to generate 10 investigative questions per news article to assess its trustworthiness for the TREC 2025 shared task. The code loads article topics from a JSONL file (trec-2025-dragun-topics.jsonl), and for each article, it uses a Qwen3-14B language model (through the Unsloth implementation) to generate questions that follow strict guidelines focusing on source credibility, evidence quality, origin tracing, and balance. The questions are generated using a LangChain LLMChain and a structured PromptTemplate. A retry loop ensures at least 10 valid and unique questions are produced per topic. The final output is saved in a TSV file for submission.
- What other datasets or services (e.g. Google/Bing web search, ChatGPT, Perplexity, etc.) were used in producing the run?
- No external datasets or retrieval services like Google, Bing, ChatGPT, or Perplexity were used in this run.
- Briefly describe LLMs used for this run (optional)
- The model used is unsloth/Qwen3-14B-unsloth-bnb-4bit, a 14-billion parameter transformer-based language model from the Qwen3 series, fine-tuned and optimized by Unsloth for memory efficiency. It is loaded in 4-bit quantized format to reduce resource usage, and supports RoPE scaling to handle longer input sequences efficiently. This model is known for high performance in text generation tasks and is accessed via the Hugging Face pipeline and LangChain integration for prompt-based inference.
- Please give this run a priority for inclusion in manual assessments.
- 1 (top)
Evaluation Files
Paper