TREC 2025 Proceedings

Submission Details

Organization

hltcoe-multiagt

Track

RAG TREC Instrument for Multilingual Evaluation

Task

Report Generation Task

Date

2025-08-21

Run Description

Document collection

['English subset', 'Arabic subset', 'Chinese subset', 'Russian subset']

Machine translation of documents

['None']

Write a short description of your retrieval process

Reciprocal ranked fusion over multilingual retrieval using plaid-x, multilingual LSR, and Qwen3 single dense vectors with ANN search.

Write a short description of your generation process

This run leverages the gpt-researcher framework. It uses the entire document as context to generate the report. The system generates 2 queries and uses the initial title to retrieve source documents. The top 3 documents from each retrieval is used as source material for the generation. The LLM is prompted to generate a report of the expected length. All generation is based on documents written in their native language.

Which LLM(s) where used by your system?

Retrieval: XLMR and Qwen3-8B-Embedding; Generation: Llama-3.3-70B-Instruct

Open repository link

No

Assessing priority

10 (lowest)

Evaluation Files

• gptr_ka_q3d3_natv.autoargue (autoargue)
• gptr_ka_q3d3_natv.almost-human-judgments.tsv (almost-human-judgments.tsv)
• gptr_ka_q3d3_natv.almost-human-scores.tsv (almost-human-scores.tsv)
• gptr_ka_q3d3_natv.autoargue-scores.tsv (autoargue-scores.tsv)
• gptr_ka_q3d3_natv.autoargue-judgments.jsonl (autoargue-judgments.jsonl)