infolab_UD_run3 — LLM Ranking Task

Is the run manual or automatic?: automatic
Did you use the response metadata?: no
Did you use any additional data or external knowledge?: no
Did you use the development set?: yes
Did you train on the development set?: yes
Provide a description of this run, including details about your answers above.: Run 3: LLM Profile by clusters model. Consists of two main components: 1. Answers enconding: All nswers were embedded. 2. Profile per LLM: For each LLM, a profile was built consisting of k clusters. At inference: New query -> encoded -> compare to all centroids -> Final LLM score *Run Type: Automatic and Internal – no manual intervention was performed after seeing the test queries. And, only the discovery data was used to build profiles
Priority for pooling: 3