TREC 2025 Proceedings

CUET-unsloth-Mistral-Small

Submission Details

Organization
CUET
Track
Detection, Retr., and Gen for Understanding News
Task
Question Generation Task
Date
2025-08-08

Run Description

Is this run manual or automatic?
automatic
Is this run based on the provided starter kit?
no
Briefly describe this run
This run processes the TREC 2025 topic file (trec-2025-dragun-topics.jsonl) containing news article titles and bodies. A custom PromptTemplate is used to instruct the LLM to generate 10 concise and critical investigative questions for each article, focusing on source bias, intent, diversity of viewpoints, and factual accuracy. The model’s output is parsed using a regex pattern to extract exactly 10 unique questions per topic, which are then saved in TSV format for submission.
What other datasets or services (e.g. Google/Bing web search, ChatGPT, Perplexity, etc.) were used in producing the run?
No external APIs, datasets or services (like Google Search or ChatGPT) are used during the execution of this code.
Briefly describe LLMs used for this run (optional)
The run used unsloth/Mistral-Small-Instruct-2409-bnb-4bit, an instruction-tuned variant of the Mistral architecture. It was loaded in 4-bit quantization using the unsloth library’s FastLanguageModel.from_pretrained method to optimize memory usage. The model was integrated with LangChain’s HuggingFacePipeline for text generation.
Please give this run a priority for inclusion in manual assessments.
2

Evaluation Files

Paper