TREC 2025 Proceedings
CUET-Mistral-Small-24B
Submission Details
- Organization
- CUET
- Track
- Detection, Retr., and Gen for Understanding News
- Task
- Question Generation Task
- Date
- 2025-08-08
Run Description
- Is this run manual or automatic?
- automatic
- Is this run based on the provided starter kit?
- no
- Briefly describe this run
- This run processes the official TREC 2025 topic file (trec-2025-dragun-topics.jsonl) to generate exactly 10 ranked investigative questions for each news article. A custom prompt template with few-shot examples is used to guide the model toward producing concise, non-redundant questions focused on evaluating trustworthiness, including aspects like bias, motivation, diversity of viewpoints, and factual accuracy. The code uses regex-based parsing to extract and deduplicate questions, retrying up to three times if fewer than 10 valid outputs are produced. Results are stored in a TSV file for submission, and duplicate detection is performed as a quality check.
- What other datasets or services (e.g. Google/Bing web search, ChatGPT, Perplexity, etc.) were used in producing the run?
- No additional datasets or external services such as Google Search, Bing, ChatGPT, or Perplexity were used in producing the run.
- Briefly describe LLMs used for this run (optional)
- The run uses unsloth/Mistral-Small-24B-Instruct-2501-unsloth-bnb-4bit, an instruction-tuned large language model based on the Mistral architecture. It was loaded via the unsloth library in 4-bit quantization mode to optimize GPU memory usage without significantly affecting output quality. The model was integrated with HuggingFace’s pipeline API and LangChain’s LLMChain for structured prompting and controlled text generation.
- Please give this run a priority for inclusion in manual assessments.
- 4
Evaluation Files
Paper