Development of an Optimal Multi-Agent Reinforcement Learning Control Method for an Integrated PVT–Heat Pump System

• PPO achieved shortest payback (15.4 yr) without comfort loss. • Interior flow policies cut pumping: PVT −21%, FCU heat −32–35%, cool −18%. • PPO sustained ∼16% electrical and 22.5% thermal efficiency under storage coupling. Hybrid building systems that couple photovoltaic–thermal (PVT) collectors, an air-to-water heat pump (AWHP), and stratified storage can cut operating cost but are hard to control due to storage delays, seasonal nonstationarity, and subsystem coupling. We cast flow-rate control as a model-free multi-agent reinforcement learning (MARL) problem with centralized training and decentralized execution. Three agents (PVT, AWHP, fan-coil loop) act every 60 s in a year-long co-simulation of a reference office building. The reward minimizes tariff-weighted energy cost with comfort and constraint penalties; uniform safety bounds and slew-rate limits are applied. We evaluate Proximal Policy Optimization (PPO; continuous actions) and a discrete Dueling DQN against a supervised DNN and a rule-based controller. PPO learns smooth, storage-aware modulation that favors interior flow setpoints, preserves stratification, and reduces safety interventions. Over the full year, PPO delivers the best economics, achieving the shortest payback period (15.4 years) and the lowest 20-year life-cycle cost, outperforming Dueling DQN (16.0 years), the supervised DNN (17.3 years), and a conventional non-PVT/non-storage reference (17.7 years). Overall, PPO reduces tariff-weighted operating cost while maintaining comfort and constraint compliance, demonstrating quantitatively superior coordination of PVT charging, AWHP operation, and FCU draw in storage-coupled buildings.