High-Fidelity Lossy Compression of Scientific Time Series Data

As the volume of data generated by scientific instruments continues to grow, efficient time-series compression has become a key challenge in applied computing. Existing lossy scientificcompressors (e.g., the SZ family and ZFP) primarily focus on limiting pointwise error (e.g.,RMSE, error bounds), which does not necessarily guarantee preservation of scientificallyrelevant signal characteristics such as long-term trend slope, dominant frequency, or thebehavior of transients and the error-tail distribution.This paper presents SZYRYNGO v5, a standalone and cryptographically verifiable1D lossy compression codec designed to maximize scientific fidelity while achieving highcompression. The codec employs block-wise removal of a slow-varying component (trend/base-line models: none, linear2, poly2, asinh4), transient detection (event coding), adaptivequantization, reversible integer-domain delta coding, and entropy compression (e.g., LZMA).On public datasets SWPC/CO2/GHCNwe demonstrate compression ratios of 2.46–11.44× (depending on dataset and configuration), with high correlation and preservation ofthe dominant frequency (DomFreqErr = 0% in our tests). In addition to standard metrics(NMAE, correlation, trend slope error), we report error-tail metrics (P99 and P99.9 |e|)and exceedance fractions with respect to thresholds Tα = α · span