Beyond Bandwidth: SMT Scaling for LLM Inference on IBM POWER

Key Points

• The research aims to analyze how Simultaneous Multi-Threading affects Large Language Model inference performance across different IBM POWER generations.
• Characterization of SMT scaling across POWER9, POWER10, and POWER11 processors.
• Testing LLMs ranging from 1.1 billion to 7 billion parameters.
• Evaluation under strict single-core isolation.
• Identified performance improvements with SMT scaling on POWER processors.
• Notable differences in efficiency across the three POWER generations.
• Inferred optimal configurations for LLM inference.