TREC 2025 Proceedings
lg_nt_4q12r3l_mt_c
Submission Details
- Organization
- hltcoe-multiagt
- Track
- RAG TREC Instrument for Multilingual Evaluation
- Task
- Report Generation Task
- Date
- 2025-08-21
Run Description
- Document collection
- ['English subset', 'Arabic subset', 'Chinese subset', 'Russian subset']
- Machine translation of documents
- ['Yes we used the organizer-provided machine translations']
- Write a short description of your retrieval process
- Reciprocal ranked fusion over multilingual retrieval using plaid-x, multilingual LSR, and Qwen3 single dense vectors with ANN search.
- Write a short description of your generation process
- This run leverages the langraph framework. In a round, the approach produces a set of 4 queries. Notetaking is done on the top 12 documents for each query. The notes from documents retrieved with a single query are used to generate a partial answer. Partial answers from all queries are examined for completeness. If the answer is deemed to be incomplete, up to 4 new queries are produced to fill knowledge gaps. At the completion of at most 3 rounds, and answer is drafted and then shortened to fit the length limit. Each citation is checked to see that it supports the sentence. Unfaithful citations are removed. If a substitute can be found, another document is used instead. Otherwise, the sentence is removed. All generation is based on translated documents.
- Which LLM(s) where used by your system?
- Retrieval: XLMR and Qwen3-8B; Generation: Llama-3.3-70B-Instruct
- Open repository link
- No
- Assessing priority
- 1 (highest)
Evaluation Files