UR_IW_run_1_task2 — Report Generation Task

Is this run manual or automatic?: automatic
Is this run based on the provided starter kit?: yes
Briefly describe this run: We used CoT query expansion (Jagerman et al., 2023) to transform questions from task 1 into queries. We searched on an Elastic Search index with MS Marco v2.1 segmented with a multi-match query using the standard english analyzer. We used the mono T5 reranker to rerank the top 1000 retrieval results. We judged the relevance up to the top 100 reranked documents, we checked for the following conditions: is the source of the retrieved document trustworthy (dataset Lin et al., 2023; PC trustworthiness score > 0.7), is the document relevant (using an LLM). We used the remaining documents to generate the report: We prompted an LLM to generate answers for the questions that we were able to retrieve segments for, if we had more than 10 questions that we were able to answer we used k-means to select the 10 most diverse questions based on their embedding. A shortener was employed to reduce the size of the report to a maximum of 250 words.
What other datasets or services (e.g. Google/Bing web search, ChatGPT, Perplexity, etc.)were used in producing the run?: gpt-5-nano source trustworthiness dataset (Lin et al., 2023)
Briefly describe LLMs used for this run (optional)
Please give this run a priority for inclusion in manual assessments.: 1 (top)