TREC 2025 Proceedings

Submission Details

Organization

hltcoe-multiagt

Track

RAG TREC Instrument for Multilingual Evaluation

Task

Report Generation Task

Date

2025-08-21

Run Description

Document collection

['English subset', 'Arabic subset', 'Chinese subset', 'Russian subset']

Machine translation of documents

['Yes we used the organizer-provided machine translations']

Write a short description of your retrieval process

Reciprocal ranked fusion over multilingual retrieval using plaid-x, multilingual LSR, and Qwen3 single dense vectors with ANN search.

Write a short description of your generation process

This run leverages the gpt-researcher framework. It uses notetaking to identify the most informative parts of the document and then generates the answer based on the notes. The system generate 3 queries and uses the initial title to retrieve source documents. The top 4 documents from each retrieval is used as source material for the generation. The LLM is prompted to generate a report of the expected length. All generation is based on translated documents.

Which LLM(s) where used by your system?

Retrieval: XLMR and Qwen3-8B-Embedding; Generation: Llama-3.3-70B-Instruct

Open repository link

No

Assessing priority

2

Evaluation Files

• gptr_nt_q4d4_mt.autoargue (autoargue)
• gptr_nt_q4d4_mt.almost-human-judgments.tsv (almost-human-judgments.tsv)
• gptr_nt_q4d4_mt.almost-human-scores.tsv (almost-human-scores.tsv)
• gptr_nt_q4d4_mt.autoargue-scores.tsv (autoargue-scores.tsv)
• gptr_nt_q4d4_mt.autoargue-judgments.jsonl (autoargue-judgments.jsonl)