TREC 2025 Proceedings
CUET-DeepSeek-R1-Qwen-32B
Submission Details
- Organization
- CUET
- Track
- Detection, Retr., and Gen for Understanding News
- Task
- Question Generation Task
- Date
- 2025-08-08
Run Description
- Is this run manual or automatic?
- automatic
- Is this run based on the provided starter kit?
- no
- Briefly describe this run
- This run processes news topics from the TREC 2025 dataset to generate exactly 10 ranked investigative questions for each topic, emphasizing trustworthiness, bias, motivation, and factual accuracy. It uses a few-shot prompt template with specific examples, sends the topic title and body to a quantized LLM through LangChain, extracts clean numbered questions via regex, retries up to three times if fewer than 10 are generated, fills missing ones with “N/A,” and outputs the results in a TSV submission file (CUET_run9.tsv) formatted with topic ID, team ID, run ID, rank, and question. Duplicate questions are also identified for review.
- What other datasets or services (e.g. Google/Bing web search, ChatGPT, Perplexity, etc.) were used in producing the run?
- No external APIs, datasets or services (like Google Search or ChatGPT) are used during the execution of this code.
- Briefly describe LLMs used for this run (optional)
- The run uses the unsloth/DeepSeek-R1-Distill-Qwen-32B-bnb-4bit model — a 32-billion-parameter distilled Qwen model optimized with unsloth for 4-bit quantized inference, reducing memory usage while maintaining high performance. It is integrated into a HuggingFace text-generation pipeline with a maximum sequence length of 2048 tokens and accessed through LangChain’s LLMChain to apply the custom prompt template and control generation parameters such as temperature and token limits.
- Please give this run a priority for inclusion in manual assessments.
- 1 (top)
Evaluation Files
Paper