What question did this study set out to answer?

The aim is to compare public datasets for Non-Intrusive Load Monitoring (NILM) to help researchers choose the best options for their work.

February 19, 2026Open Access

Advances in NILM dataset evaluation: a comparative review within the NILMTK framework

Key Points

The aim is to compare public datasets for Non-Intrusive Load Monitoring (NILM) to help researchers choose the best options for their work.
Reviewed multiple public NILM datasets chronologically
Benchmarking conducted using the NILMTK framework
Evaluated algorithms based on various metrics like F1-score, MAE, RMSE
Highlighted differences in performance among appliances and algorithms
Achieved an average F1-score of 0.6135 across datasets
Traditional algorithms showed the highest F1-score at 0.6600
Deep learning models achieved lower MAE (111.26 W) and RMSE (180.98 W) at a higher computational cost
Refrigerators and LED lamps provided higher accuracy, while washing machines had significant performance challenges.

Abstract

Abstract This study presents a comparative and chronological review of energy datasets fundamental to research in Non-Intrusive Load Monitoring (NILM). Aiming to assist researchers in selecting appropriate datasets aligned with their objectives, the article systematically benchmarks prominent public datasets, including recent ones such as DEPS, DSUALM, DSUALM10H, and OMPM/UALM2, using the open-source NILM toolkit (NILMTK) to facilitate reproducibility and comparison results. The chronological evaluation highlights the evolution of dataset features, from early limitations in standardization to the recent inclusion of high-frequency sampling and harmonic information enabled by open hardware platforms. Widely used NILM algorithms are evaluated across datasets employing various NILMTK metrics, including F1-score, MAE, RMSE, and EAE, revealing an average F1-score of 0.6135. Traditional approaches, such as the Factorial Hidden Markov Model, achieve the highest average F1-score (0.6600), demonstrating robust efficiency. Conversely, deep learning models, such as Recurrent Neural Networks, show superior accuracy with lower MAE (111.26 W) and RMSE (180.98 W) on specific datasets, but at a significantly higher computational cost. Results indicate significant variability in algorithm performance depending on dataset specifics and device types. Appliances with stable consumption patterns, such as refrigerators and LED lamps, demonstrate higher accuracy (e.g., F1-score of up to 0.72 for refrigerators in Dataport and 0.78 for LED lamps in UALM2). In contrast, complex or multistate devices, such as washing machines and electric heaters, present greater challenges (e.g., an F1-score of 0.0 for washing machines in iAWE and a high MNEAP for electric heaters in REFIT). Overall, NILM dataset selection depends not on a universally superior resource, but on the alignment of dataset attributes (sampling frequency and duration, number and variety of monitored homes, types of measurements, including harmonics and contextual data) and compatibility with NILMTK, tailored to the research objectives and problem complexity. While traditional algorithms offer computational efficiency, deep learning models provide greater accuracy at a higher cost. The use of open hardware platforms such as OpenZmeter and OMPM promotes reproducibility and accelerates NILM innovation. This review emphasizes the importance of striking a balance between study goals, technical resources, and measurement tools to establish a robust framework for algorithm development and validation in energy disaggregation.

Ask AI

Helpful

Bookmark

View Full Paper