Graph-based Methods for Efficient, Interpretable and Reliable Machine Learning

Abstract

Machine learning algorithms have revolutionized fields such as computer vision, natural language processing, and speech recognition by offering the capability to analyze and extract information from vast datasets, a task far beyond human capacity. The deployment of these algorithms in high-stakes applications, including medical diagnosis, computational finance and criminal justice, underscores their growing importance. However, the decision-making processes of the so-called black-box models used in such areas raise considerable concerns. Therefore, enhancing the interpretability of these models is crucial, as it helps address issues like biases and inconsistencies in predictions, thereby making the models more comprehensible and trustworthy to end-users. Moreover, interpretability facilitates a deeper understanding of model behavior, such as the distribution of contributions across inputs. This deeper understanding can be applied to significantly improve efficiency. This is especially relevant as machine learning models find applications on edge devices, where computational resources are often limited. For such applications, significant improvements in energy efficiency and resource requirements can be obtained by optimizing and adapting model implementations based on an understanding of the models' internal behavior. However, such optimization introduces new challenges that arise due to factors such as complex, dynamically-determined dependency management among computations. This thesis presents five main contributions. The first contribution is the development of a novel type of interpretable machine learning model for applications in criminology and criminal justice (CCJ). The model involves graphical representations in the form of single decision trees, where the trees are constructed in an optimized fashion using a novel evolutionary algorithm. This approach not only enhances intrinsic interpretability but also enables users to understand the decision-making process more transparently, addressing the critical need for clarity in machine learning models' predictions. At the same time, the application of evolutionary algorithm methods enables such interpretability to be provided without significant degradation in model accuracy. In the second contribution, we develop new multi-objective evolutionary algorithm methods to find a balance between fairness and predictive accuracy in CCJ applications. We build upon the single-decision-tree framework developed in the first contribution of the thesis, and systematically integrate considerations of fairness and multi-objective optimization. In the third contribution, we develop new methods for crime forecasting applications. In particular, we develop new interpretable, attention-based methods using convolutional long short-term memory (ConvLSTM) models. These methods combine the power of ConvLSTM models in capturing spatio-temporal patterns with the interpretability of attention mechanisms. This combination of capabilities allows for the identification of key geographic areas in the input data that contribute to predictions from the model. The fourth contribution introduces a dynamic dataflow-graph-based framework to enhance the computational efficiency and run-time adaptability of inference processes, considering the constraints of available resources. Our proposed model maintains a high degree of analyzability while providing greater freedom than static dataflow models in being able to manipulate the computations associated with inference process at run-time. The fifth contribution of the thesis builds on insights developed in the fourth, and introduces a new parameterized design approach for image-based perception that enables efficient and dynamic reconfiguration of convolutions using channel attention. Compared to switching among sets of multiple complete neural network models, the proposed reconfiguration approach is much more streamlined in terms of resource requirements, while providing a high level of adaptability to handle unpredictable and dynamically-varying operational scenarios.