Beyond Discriminant Patterns: On the Robustness of Decision Rule Ensembles
Du, Xin Ramamoorthy, Subramanian Duivesteijn, Wouter Tian, Jin Pechenizkiy, Mykola
Du, Xin
Ramamoorthy, Subramanian
Duivesteijn, Wouter
Tian, Jin
Pechenizkiy, Mykola
Supervisor
Department
Machine Learning
Embargo End Date
Type
Conference proceeding
Date
License
Language
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Local decision rules are highly regarded for their interpretability, offering insights into granular patterns that are critical for explainable machine learning. While existing methods emphasize the identification of discriminative patterns to achieve high predictive accuracy, they often fail to account for robustness against distributional shifts that occur during deployment. This paper addresses this gap by proposing a novel approach to learning and ensembling local decision rules that are inherently robust across diverse training and deployment environments. Our method leverages causal inference principles, viewing distributional shifts as interventions on the underlying system. We incorporate two regularization techniques: graph-based regularization, which decomposes invariant features using causal graphs, and variance-based regularization, which promotes stability by introducing artificial features to guide decision boundaries. These techniques enable the generation of decision rules that excel in predictive power while maintaining stability under changing environmental conditions. Extensive experiments on synthetic and benchmark datasets validate the effectiveness of the proposed method. The results demonstrate significant improvements in robustness, outperforming traditional boosting ensembles when subjected to diverse and challenging environments. Quantitative and qualitative analyses further highlight how the integration of causal knowledge and adaptive regularization encourages the utilization of invariant features, leading to better generalization. This work emphasizes the importance of causal reasoning in the design of machine learning models, paving the way for future research into robust, interpretable, and reliable decision-making frameworks for real-world applications.
Citation
X. Du, S. Ramamoorthy, W. Duivesteijn, J. Tian, M. Pechenizkiy, "Beyond Discriminant Patterns: On the Robustness of Decision Rule Ensembles," 2026, pp. 1174-1183.
Source
2025 IEEE International Conference on Data Mining (ICDM)
Conference
IEEE International Conference on Data Mining (ICDM)
Keywords
40 Engineering, 46 Information and Computing Sciences, 4603 Computer Vision and Multimedia Computation, 4611 Machine Learning
Subjects
Source
IEEE International Conference on Data Mining (ICDM)
Publisher
IEEE