Randomized trial shows up to 1.56× speedup in LLM workflows, suggesting end-to-end optimization is vital for efficiency.
Key Points
This research aims to optimize the serving of agentic workflows involving Large Language Models (LLMs) by addressing inefficiencies in current systems.
Introduced Helium, a workflow-aware serving framework for LLM invocations.
Integrated proactive caching and cache-aware scheduling to enhance prompt reuse.
Modelled agentic workloads as query plans to leverage classic query optimization.
Achieved up to 1.56× speedup over existing agent serving systems.
Demonstrated improved efficiency across various workloads through end-to-end optimization.