Expand: An ad-hoc file system for parallel and distributed environments

The use of data-intensive applications for massive data analysis, such as artificial intelligence and big data, is becoming increasingly common. However, running these applications efficiently in current HPC environments is challenging due to their design and the backend file systems they use. To address these issues, Expand, an open-source ad-hoc file system, was designed and developed. This is suitable for data-intensive applications because it is ad-hoc and highly optimized. Its modular design enables its use in both HPC and distributed environments. It also provides fault tolerance mechanisms for fault-sensitive applications. Finally, to avoid wasting computational resources, Expand has malleability mechanisms.