SORS: INFN-CNAF big data processing for high energy physics jobs monitoring

The modern data centres provide the efficient Information Technologies (IT) infrastructure needed to deliver resources, services, monitoring systems and collected data in a timely fashion. At the same time, data centres have been continuously evolving, foreign large increase of resources and adapting to cover multifaceted niches.The CNAF group at INFN (National Institute for Nuclear Physics) has implemented a Big Data Platform (BDP) infrastructure, designed for the collection and the indexing of log reports form CNAF facilities. The infrastructure is an ongoing project at CNAF and it is available for the italian groups working in high energy physics experiments. Within this framework, the first data pipeline was established for the ATLAS experiment at CERN, using input from the ATLAS Distributed Computing System PanDa. This pipeline focuses on the ATLAS computational job data processed by the Italian INFN Tier-1 computing farm. The system has been operational and effective for several years, marking our initiative as the first to integrate job information directly with the infrastructure. Following the finalization of data transmission, our objective is to conduct an analysis and surveillance of the PanDA Jobs data. This will involve examining the performance metrics of the machines and identifying the log errors that lead to job failures.