Session Chair: Dirk Bergemann
Title: When calibration goes awry: hallucination in language models
Abstract:
“Hallucinations” are a major problem for language models. We shed light on this phenomenon by showing that calibration, which is naturally encouraged during the pre-training of language models, leads to hallucinations. Moreover, the rate of hallucinations depends on the domain via the classic Good-Turing estimator. Interestingly, this estimate is small for facts like paper titles, which have been a notorious source of hallucinations. The analysis also suggests methods for mitigating hallucinations. This is joint work with Santosh Vempala and was done while the speaker was at Microsoft Research New England.
Mikhail Belkin, UCSD
Title: The puzzle of dimensionality and feature learning from LLMs to kernel machines
Abstract:
Remarkable progress in AI has far surpassed expectations of just a few years ago.
At their core, modern models, such as transformers, implement traditional statistical models — high order Markov chains. Nevertheless, it is not generally possible to estimate Markov models of that order given any possible amount of data. Therefore these methods must implicitly exploit low-dimensional structures present in data. Furthermore, these structures must be reflected in high-dimensional internal parameter spaces of the models. Thus, to build fundamental understanding of modern AI, it is necessary to identify and analyze these latent low-dimensional structures. In this talk, I will discuss how deep neural networks of various architectures learn low-dimensional features and how the lessons of deep learning can be incorporated in non-backpropagation-based algorithms that we call Recursive Feature Machines. I will provide a number of experimental results on different types of data, as well as connections to classical sparse learning methods, such as Iteratively Reweighted Least Squares.