TREC 2025 Proceedings
UCSC-base-ensemble
Submission Details
- Organization
- ucsc
- Track
- Interactive Knowledge Acquisition Track
- Task
- Passage Ranking and Response Generation
- Date
- 2025-07-28
Run Description
- What type of manually annotated information does the system use?
- automatic: system does not use any manually annotated data and relies only on the user utterance and system responses (canonical responses of previous turns)
- How is conversation understanding (NLP/rewriting) performed in this run (check all that apply)?
- ['method uses large language models like LLaMA and GPT-x.']
- What data is used for conversational query understanding in this run (check all that apply)?
- ['method uses iKAT 23 data']
- How is ranking performed in this run (check all that apply)?
- ['method uses learned sparse retrieval (e.g.¸ SPLADE¸ etc.)', 'method performs re-ranking with a pre-trained neural language model (BERT¸ Roberta¸ T5¸ etc.) (please describe specifics in the description field below)']
- What data is used to develop the ranking method in this run (check all that apply)?
- ['method uses iKAT 23 data']
- Please specify all the methods used to handle feedback or clarification responses from the user (check all that apply).
- ['method does not treat them specially']
- Please describe the method used to generate the final conversational responses from one or more retrieved passages (check all that apply).
- ['method uses multiple sources (multiple passages)', 'method uses large language models to generate the summary.']
- Please describe how you integrate the PTKBs in your run (check all that apply)
- [' method uses a PTKB relevance model to detect the relevant ones', " method integrates PTKBs in the response generation method (e.g. include in the LLM's prompt)"]
- Which LLM did you use to generate the final response?
- ['method uses closed-source commercial LLMs (e.g. GPT-*)']
- Please describe the external resources used by this run, if applicable.
- none
- Please provide a short description of this run.
- This run uses the base prompt and combines 5 rerankers (debertav2, debertav3, albert, electra, robertav2) using min-max normalization to create the final ranked list. Response generation utilizes the top 20 ranked passages.
- Please provide a priority for assessing this run. (If resources do not allow all runs to be assessed, NIST will work in priority order, resolving ties arbitrarily).
- 1 (top)
Evaluation Files