Software implementation of a deterministic approach to analyzing large files

this article offers an original deterministic approach to analyzing text files, which allows solving the problem of a sharp slowdown in the total data processing speed with an increase in file size over 500 MB. Two variants of this approach are considered: direct analysis of the file mapped to the virtual address space of the process using pointer arithmetic (as with the usual memory contents) and an approach based on alternating copying of blocks of the file mapped to memory into an additional RAM buffer and further analysis of the data already in the buffer. Each of the considered variants of the approach was programmatically implemented and the time characteristics of its execution on the same data set were obtained. The analysis of the results clearly indicates the advantage of using a file projection into memory and additional buffering when analyzing text files larger than 500 MB.