Towards a Theoretical Understanding of the 'Reversal Curse' via Training
Dynamics
Towards a Theoretical Understanding of the 'Reversal Curse' via Training
Dynamics
Auto-regressive large language models (LLMs) show impressive capacities to solve many complex reasoning tasks while struggling with some simple logical reasoning tasks such as inverse search: when trained on ''A is B'', LLM fails to directly conclude ''B is A'' during inference, which is known as the ''reversal curse'' (Berglund …