Detecting Correct Answers to Open Questions and Its Impact on Language Models’ Confidence Scores
Date:
Presented at the LatinX in AI (LXAI) Research Workshop colocated with NAACL 2024 in Mexico City, Mexico.
This presentation discussed calibration challenges in LLMs when answering open-ended questions, focusing on the effect of automated evaluation metrics on measured confidence scores and calibration error.