Computational Arrow of Time in Large Language Models: Forward-Backward Perplexity Asymmetry as Independent Confirmation of the Fantasia Bound

Key Points

• This research aims to confirm the Fantasia Bound through measuring perplexity asymmetry in language models.
• Measured forward-backward perplexity asymmetry across eight languages.
• Analyzed three different language model architectures.
• Assessed the relationship of asymmetry with model size.
• Found 0.6-3.2% perplexity asymmetry in evaluated models.
• Observed scaling of asymmetry correlating with model size as predicted.
• Confirmed predictions of the conjugacy bound through measurements.