ABSTRACT Introduction As machine learning (ML)‐enabled systems become increasingly prevalent across industries, the engineering challenges of deploying and maintaining them in production have emerged as critical. The existing engineering knowledge base often derives from conceptual frameworks or case studies conducted by large technology companies, leaving a gap in the empirical nuances of deployment practices across a wide range of engineering contexts. Objective This study aims to investigate real‐world ML deployment workflows and inference architectures to identify common patterns, contextual variations, and underlying trade‐offs that influence how practitioners operationalize ML models in production. Methods We conducted a multi‐case study of eight ML systems across sectors, including advertising, finance, healthcare, manufacturing, and software platforms. We collected data through semi‐structured interviews with practitioners and the various inference architectures. We applied thematic analysis to extract recurring patterns across the deployment lifecycle, from model versioning to inference serving. Results Our findings reveal five core deployment themes: model versioning and storage, quality assurance, monitoring, model packaging, and inference serving. We identify a maturity gradient from manual, ad hoc practices to automated CI/CD pipelines and further highlight architectural trade‐offs between tightly coupled vs loosely coupled model deployment patterns. Other key findings include the rarity of model drift monitoring and the use of hybrid inference serving patterns to balance latency and scalability requirements. Conclusions Although ML domains and deployment architectures vary, we observed recurrent practices and strategic trade‐offs that provide an empirical foundation for developing more standardized, context‐aware ML engineering processes. Our findings provide actionable guidance for practitioners and identify under‐explored areas for future research.
Muiruri et al. (Mon,) studied this question.