TREC 2025 Proceedings
lg_nt_4q12r3l_natv_c
Submission Details
- Organization
- hltcoe-multiagt
- Track
- RAG TREC Instrument for Multilingual Evaluation
- Task
- Report Generation Task
- Date
- 2025-08-21
Run Description
- Document collection
- ['English subset', 'Arabic subset', 'Chinese subset', 'Russian subset']
- Machine translation of documents
- ['None']
- Write a short description of your retrieval process
- Reciprocal ranked fusion over multilingual retrieval using plaid-x, multilingual LSR, and Qwen3 single dense vectors with ANN search.
- Write a short description of your generation process
- This run leverages the langraph framework. In a round, the approach produces a set of 4 queries. Notetaking is done on the top 12 documents native documents for each query. The notes from documents retrieved (which are expected to be in English) with a single query are used to generate a partial answer. Partial answers from all queries are examined for completeness. If the answer is deemed to be incomplete, up to 4 new queries are produced to fill knowledge gaps. At the completion of at most 3 rounds, and answer is drafted and then shortened to fit the length limit. Each citation is checked against the native document to see that it supports the sentence. Unfaithful citations are removed. If a substitute can be found, another document is used instead. Otherwise, the sentence is removed. All generation is based on native documents.
- Which LLM(s) where used by your system?
- Retrieval: XLMR and Qwen3-8B; Generation: Llama-3.3-70B-Instruct
- Open repository link
- No
- Assessing priority
- 4
Evaluation Files