TREC 2025 Proceedings

Submission Details

Organization

hltcoe-rerank

Track

RAG TREC Instrument for Multilingual Evaluation

Task

Report Generation Task

Date

2025-08-21

Run Description

Document collection

['English subset', 'Arabic subset', 'Chinese subset', 'Russian subset']

Machine translation of documents

['Yes we used the organizer-provided machine translations']

Write a short description of your retrieval process

Searcher II pointwise reranking RRF of PLAID-X, Qwen-Embedding-8B, and multilingual LSR.

Write a short description of your generation process

LangGraph generator (reflection, note taking, query generation, etc). LangGraph uses Llama 3.3 70B for most steps. GPT-5 is used for final answer generation (drafting) and answer shortening (revising report). LangGraph generates 4 initial queries, retrieves 12 results per query, and runs up to 5 research loops.

Which LLM(s) where used by your system?

GPT-5 and Llama-3.3-70B

Open repository link

NA

Assessing priority

1 (highest)

Evaluation Files

• hltime-gpt5.searcher.autoargue (autoargue)
• hltime-gpt5.searcher.almost-human-judgments.tsv (almost-human-judgments.tsv)
• hltime-gpt5.searcher.almost-human-scores.tsv (almost-human-scores.tsv)
• hltime-gpt5.searcher.autoargue-scores.tsv (autoargue-scores.tsv)
• hltime-gpt5.searcher.autoargue-judgments.jsonl (autoargue-judgments.jsonl)