Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation

Lorenz Kuhn, Yarin Gal and Sebastian Farquhar introduce Semantic Uncertainty. Their method samples answers, groups bidirectionally entailing sequences and estimates entropy over meanings (§§3–4). Sampling and imperfect equivalence classification can limit the estimate.^[1]

Why read it here?

It distinguishes uncertainty in an answer’s meaning from variation in wording. The fabricated-citation case raises an additional question: whether an independent record authenticates a claim. Sampling consistency alone cannot supply that record.

Historical context

Reviewed original arXiv v1, submitted February 19, 2023. Selected introduction, methods and appendix discussion inspected, including rendered p.3. Later revisions, code, datasets and complete experiments were not audited. The question-answering setup does not validate citation checking in legal practice.^[1]