TREC 2025 Proceedings

CUET-qwen4B-v3

Submission Details

Organization
CUET
Track
Detection, Retr., and Gen for Understanding News
Task
Question Generation Task
Date
2025-08-08

Run Description

Is this run manual or automatic?
automatic
Is this run based on the provided starter kit?
no
Briefly describe this run
This run generates 10 ranked investigative questions for each topic in the TREC 2025 dataset using the unsloth/Qwen3-4B model. The prompt is enhanced with few-shot examples and explicitly instructs the model to rank questions based on importance, emphasizing critical thinking on bias, motivation, factual accuracy, and viewpoint diversity, including right-wing and centrist perspectives. The LangChain LLMChain is built around a HuggingFace pipeline with sampling enabled for generation. Each topic (title + truncated body) is passed to the model, and output is parsed using a regex to extract uniquely numbered questions up to 300 characters. The process includes a retry mechanism (up to 3 attempts) to ensure at least 10 valid questions, with padding as needed. The cleaned and deduplicated questions are saved in CUET_run4.tsv for TREC submission.
What other datasets or services (e.g. Google/Bing web search, ChatGPT, Perplexity, etc.) were used in producing the run?
No external APIs, datasets or services (like Google Search or ChatGPT) are used during the execution of this code.
Briefly describe LLMs used for this run (optional)
The model used is unsloth/Qwen3-4B, a 4-billion parameter language model optimized for low-resource inference through 4-bit quantization (bnb-4bit). It supports RoPE scaling for long sequences up to 2048 tokens. Integrated via HuggingFace and LangChain, the model generates responses based on few-shot prompts tailored to the investigative task. The generation settings (e.g., temperature, max tokens) are configured to allow creativity while keeping the responses concise and within constraints.
Please give this run a priority for inclusion in manual assessments.
5 (bottom)

Evaluation Files

Paper