TREC 2025 Proceedings

Submission Details

Organization

coordinators

Track

RAG TREC Instrument for Multilingual Evaluation

Task

Report Generation Task

Date

2025-08-21

Run Description

Document collection

['English subset', 'Arabic subset', 'Chinese subset', 'Russian subset']

Machine translation of documents

['None']

Write a short description of your retrieval process

This run uses the track-provided PLAID-X search service.

Write a short description of your generation process

The prompts were engineered to perform well on the NeuCLIR 2024 pilot task. This approach generates 10 queries and includes the top 3 documents from each search. Facts are extracted from documents and grouped to reduce duplicate information. The answer is intended to be independent facts as in a bulleted-list. Splade V3 is used for search. gpt-oss-20b is used to identify facts and group them.

Which LLM(s) where used by your system?

OpenAI OSS-20B

Open repository link

N/A

Assessing priority

4

Evaluation Files

• extractive-rag.autoargue (autoargue)
• extractive-rag.almost-human-judgments.tsv (almost-human-judgments.tsv)
• extractive-rag.almost-human-scores.tsv (almost-human-scores.tsv)
• extractive-rag.autoargue-scores.tsv (autoargue-scores.tsv)
• extractive-rag.autoargue-judgments.jsonl (autoargue-judgments.jsonl)