TREC 2025 Proceedings

Submission Details

Organization

hltcoe-multiagt

Track

RAG TREC Instrument for Multilingual Evaluation

Task

Report Generation Task

Date

2025-08-21

Run Description

Document collection

['English subset', 'Arabic subset', 'Chinese subset', 'Russian subset']

Machine translation of documents

['Yes we used the organizer-provided machine translations']

Write a short description of your retrieval process

Reciprocal ranked fusion over multilingual retrieval using plaid-x, multilingual LSR, and Qwen3 single dense vectors with ANN search.

Write a short description of your generation process

This run leverages the gpt-researcher framework.It uses a filtering approach to rank snippets in a document as most useful to the query. The top 4 snippets are selected and answers are generated based on the snippets. The system generates 2 queries and uses the initial title to retrieve source documents. The top 3 documents from each retrieval is used as source material for the generation. The LLM is prompted to generate a report of the expected length. All generation is based on translated documents.

Which LLM(s) where used by your system?

Retrieval: XLMR and Qwen3-8B-Embedding; Generation: Qwen3-8B-Embedding and Llama-3.3-70B-Instruct

Open repository link

No

Assessing priority

7

Evaluation Files

• gptr_e2_q3d3_mt.autoargue (autoargue)
• gptr_e2_q3d3_mt.almost-human-judgments.tsv (almost-human-judgments.tsv)
• gptr_e2_q3d3_mt.almost-human-scores.tsv (almost-human-scores.tsv)
• gptr_e2_q3d3_mt.autoargue-scores.tsv (autoargue-scores.tsv)
• gptr_e2_q3d3_mt.autoargue-judgments.jsonl (autoargue-judgments.jsonl)