Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning
Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning
Despite the great success of large language models (LLMs) in various tasks, they suffer from generating hallucinations. We introduce Truth Forest, a method that enhances truthfulness in LLMs by uncovering hidden truth representations using multi-dimensional orthogonal probes. Specifically, it creates multiple orthogonal bases for modeling truth by incorporating orthogonal constraints …