Nonlinear classification problems such as XOR are widely used to evaluate machine learning models beyond linear separability. In this study, a comprehensive benchmark is proposed to analyze eight classifiers (Logistic Regression, Linear SVM, RBF SVM, Decision Tree, Random Forest, KNN, MLPₛmall, MLPdeep) across four XOR variants (clean, noisy, rotated, high-dimensional). A total of 640 controlled experiments are conducted using multiple sample sizes and random seeds. Models are evaluated using a multi-objective framework including accuracy, training and inference time, memory usage, energy consumption, and model size. Results show that MLPdeep achieves the highest overall accuracy, while MLPₛmall provides competitive performance with significantly lower computational cost. Decision Tree offers a strong balance between efficiency and accuracy, whereas Random Forest achieves competitive accuracy at higher resource usage. High-dimensional XOR is the most challenging scenario, significantly reducing overall performance across models. Pareto frontier analysis further highlights optimal trade-offs between predictive performance and resource efficiency. The study demonstrates that no single model is universally optimal and emphasizes the importance of resource-aware model selection in nonlinear classification tasks.
Okutan et al. (2026) studied this question.