organizers-baseline-gpt41mini-bm25-minilm-llama70b-gpt41mini_nopersonal

Submission Details

Organization: coordinators
Track: Interactive Knowledge Acquisition Track
Task: Passage Ranking and Response Generation
Date: 2025-07-28

Run Description

What type of manually annotated information does the system use?: automatic: system does not use any manually annotated data and relies only on the user utterance and system responses (canonical responses of previous turns)
How is conversation understanding (NLP/rewriting) performed in this run (check all that apply)?: ['method uses large language models like LLaMA and GPT-x.']
What data is used for conversational query understanding in this run (check all that apply)?: ['method uses iKAT provided manually rewritten utterances (note: this makes it a manual run)']
How is ranking performed in this run (check all that apply)?: ['method uses a knowledge graph (e.g.¸ Wikidata¸ Freebase¸ etc.)']
What data is used to develop the ranking method in this run (check all that apply)?: ['method uses iKAT 23 data']
Please specify all the methods used to handle feedback or clarification responses from the user (check all that apply).: ['method detects and uses result from canonical clarification questions']
Please describe the method used to generate the final conversational responses from one or more retrieved passages (check all that apply).: ['method uses supervised generative summarization (e.g. PEGASUS or similar)']
Please describe how you integrate the PTKBs in your run (check all that apply): [" method integrates PTKBs in the response generation method (e.g. include in the LLM's prompt)"]
Which LLM did you use to generate the final response?: [' method does not use LLMs and uses other techniques for response generation (please specify details in description below)']
Please describe the external resources used by this run, if applicable.: N/A
Please provide a short description of this run.: gpt41mini-bm25-minilm-llama70b-gpt41mini_nopersonal.jsonl
Please provide a priority for assessing this run. (If resources do not allow all runs to be assessed, NIST will work in priority order, resolving ties arbitrarily).: 2

Evaluation Files

organizers-baseline-gpt41mini-bm25-minilm-llama70b-gpt41mini_nopersonal.docs_trec_eval (docs_trec_eval)
organizers-baseline-gpt41mini-bm25-minilm-llama70b-gpt41mini_nopersonal.ptkb_trec_eval (ptkb_trec_eval)

Paper