A Comparative Analysis of Motif-Based and Quantum Feature Learning Techniques in Machine Learning Models for Network Anomaly Detection

This study conducts a rigorous comparative evaluation of classical and hybrid quantum-classical machine learning (ML) models for network anomaly detection, with a particular emphasis on the impact of motif discovery and quantum feature extraction. The research explores how structural network motifs recurring subgraph patterns identified through directed graph analysis combined with quantum-enhanced feature representations, can improve the discriminative capacity of ML classifiers under noisy, real-world traffic conditions. A comprehensive set of experiments was conducted using six ML models: Support Vector Classifier (SVC), Random Forest, Gradient Boosting, XGBoost, Neural Network, and an ensemble meta-learner. Each model was evaluated across three distinct feature regimes: classical combined features, motif-only features, and quantum-derived features. Performance metrics including accuracy, precision, recall, F1-score, and ROC-AUC were analyzed to assess classifier robustness and generalization. The results demonstrate that motif-only features yielded superior classification performance compared to both classical and quantum-only features. Notably, the SVC model trained on motif features achieved the highest recall (0.515), F1-score (0.472), and ROC-AUC (0.609), indicating a strong ability to detect anomalous traffic patterns. Gradient Boosting on motif features also performed competitively with an accuracy of 71.1% and precision of 0.733. In contrast, models using combined features underperformed, suggesting that indiscriminate feature aggregation can dilute meaningful signal. Quantum-only features offered marginal improvements in recall over classical models but were outperformed by motif-driven approaches across most metrics. This research underscores the strategic value of structural motif discovery in feature engineering for network anomaly detection and reveals that quantum-enhanced features, while promising, require further refinement to consistently outperform classical methods. The study concludes that targeted motif-based representations provide a scalable and effective avenue for enhancing ML-based intrusion detection, and that their integration with quantum computing paradigms holds potential for future breakthroughs in cybersecurity analytics.