Alignment Makes Models More Decisive Without Making Them More Truthful

Key Points

• The study investigates how post-training adjustments affect the decisiveness and accuracy of large language models.
• Analyzed 3 model architectures
• Utilized 4 reinforcement learning methods
• Assessed changes in the commitment layer and representation geometry
• Decisiveness improved without a corresponding increase in accuracy
• The commitment layer remains unchanged under reinforcement learning
• Representations compress monotonically at the fixed prediction point

Abstract