TREC 2025 Proceedings
CUET-unsloth-Mistral-Small
Submission Details
- Organization
- CUET
- Track
- Detection, Retr., and Gen for Understanding News
- Task
- Question Generation Task
- Date
- 2025-08-08
Run Description
- Is this run manual or automatic?
- automatic
- Is this run based on the provided starter kit?
- no
- Briefly describe this run
- This run processes the TREC 2025 topic file (trec-2025-dragun-topics.jsonl) containing news article titles and bodies. A custom PromptTemplate is used to instruct the LLM to generate 10 concise and critical investigative questions for each article, focusing on source bias, intent, diversity of viewpoints, and factual accuracy. The model’s output is parsed using a regex pattern to extract exactly 10 unique questions per topic, which are then saved in TSV format for submission.
- What other datasets or services (e.g. Google/Bing web search, ChatGPT, Perplexity, etc.) were used in producing the run?
- No external APIs, datasets or services (like Google Search or ChatGPT) are used during the execution of this code.
- Briefly describe LLMs used for this run (optional)
- The run used unsloth/Mistral-Small-Instruct-2409-bnb-4bit, an instruction-tuned variant of the Mistral architecture. It was loaded in 4-bit quantization using the unsloth library’s FastLanguageModel.from_pretrained method to optimize memory usage. The model was integrated with LangChain’s HuggingFacePipeline for text generation.
- Please give this run a priority for inclusion in manual assessments.
- 2
Evaluation Files
Paper