Theses and Dissertations from UMD
Permanent URI for this communityhttp://hdl.handle.net/1903/2
New submissions to the thesis/dissertation collections are added automatically as they are received from the Graduate School. Currently, the Graduate School deposits all theses and dissertations from a given semester after the official graduation date. This means that there may be up to a 4 month delay in the appearance of a give thesis/dissertation in DRUM
More information is available at Theses and Dissertations at University of Maryland Libraries.
Browse
2 results
Search Results
Item LOW-POWER AND SECURE IMPLEMENTATION OF NEURAL NETWORKS BY APPROXIMATION(2022) Xu, Qian; Qu, Gang; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Neural networks (NN), one type of machine learning (ML) algorithms, have emerged as a powerful paradigm for sophisticated applications such as pattern recognition and natural language processing. In this dissertation, we study how to apply the principle of approximate computing to solve two challenging problems in neural networks, namely energy efficiency and security. More specifically, we investigate approximation across three stacks in the implementation of NN models: computation units, data storage, and the NN model itself. Computation units, such as adders and multipliers, have been one of the main targets for power-efficient implementation of neural networks. Many low-power approximate adders and multipliers have been proposed. NNs also require complex operations like logarithms, despite the heavy usage and high energy consumption on such operations, they are not optimized for energy efficiency. Our first contribution is a truncation-based approximation method that can balance the computation precision and energy consumption of the popular floating-point logarithmic operation. We derive a formula for the most energy-efficient implementation of the logarithm unit given an error variance range. Based on this theoretical result, we propose BWOLF (Bit-Width Optimization for Logarithmic Function) to determine the bit-width of operands in the logarithms computation in order to minimize the energy consumption in delivering the required computation precision. We evaluate the efficacy of BWOLF in terms of energy savings on two widely used applications: Kullback-Leibler Divergence and Bayesian Neural Network. The experimental results validate the correctness of our analysis and show significant energy savings, from 27.18% to 95.92%, over the full-precision computation and a baseline approximation method based on uniform truncation. Storage approximation by reducing the supply voltage for dynamic random access memory (DRAM) is effective in saving the power for neural networks. However, this will introduce physical errors in DRAM and could impact the performance of NN models. In the second part of this dissertation, we explore the potential of storage approximation in improving NN system’s security in training data privacy. More specifically, we consider the Model Inversion Attacks (MIAs) that extrapolate the training data from model parameters. Our proposed solution -- MIDAS: Model Inversion Defenses with an Approximate memory System -- intentionally introduces memory faults by overscaling voltage to thwart MIA without compromising the original ML model. We use detailed SPICE simulations to build the DRAM fault model and evaluate MIDAS against state-of-the-art MIAs. Experiments demonstrate that MIDAS can effectively protect training data from run-time MIAs. In terms of the Pearson Correlation Coefficient (PCC) similarity between the original training data and the recovered version, MIDAS reduces the PCC value by 55% and 40% for shallow and deep neural networks under 1.5% accuracy relaxation. Although MIDAS shows promising security benefits through storage approximation, such approximation modifies the neural network parameters and may reduce the NN model’s accuracy. In the third part of this dissertation, we propose model approximation which aims at generating an approximate NN model to correct the errors during training and consequently reduce the possible degradation of NN’s classification results. We demonstrate this concept on gradient inversion attacks which utilize transmitted gradients between the nodes in a federated learning system to reconstruct the training data. Therefore, we propose DAGIA, a Data Augmentation defense against Gradient Inversion Attacks, to deliberately extend the training dataset and report the corresponding gradient updates to protect the original data. For multiple data augmentation techniques, we empirically evaluate the trade-off between test accuracy and information leakage to select the best technique for DAGIA. According to the Structural Similarity (SSIM) between reconstructed training data and the original CIFAR-10 dataset, the experimental results show that DAGIA can reduce the SSIM by 54% with a slightly increased test accuracy for the ConvNet model. In summary, this dissertation focuses on the role of approximation in energy efficiency and security during the implementation of neural networks. We show that computation units for complex operators can be approximated to reduce energy, the storage for neural network weights can be approximated to improve both energy efficiency and security (against information leak), and the NN model itself could be approximated during training for security enhancement. This dissertation work demonstrates that approximation is a promising method to improve the performance of neural networks. It opens the door to applying the principle of approximate computing to the implementation and optimization of neural networks where there are abundant opportunities for approximation.Item Approximation Algorithms for Resource Allocation(2011) Saha, Barna; Khuller, Samir; Computer Science; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)This thesis is devoted to designing new techniques and algorithms for combinatorial optimization problems arising in various applications of resource allocation. Resource allocation refers to a class of problems where scarce resources must be distributed among competing agents maintaining certain optimization criteria. Examples include scheduling jobs on one/multiple machines maintaining system performance; assigning advertisements to bidders, or items to people maximizing profit/social fairness; allocating servers or channels satisfying networking requirements etc. Altogether they comprise a wide variety of combinatorial optimization problems. However, a majority of these problems are NP-hard in nature and therefore, the goal herein is to develop approximation algorithms that approximate the optimal solution as best as possible in polynomial time. The thesis addresses two main directions. First, we develop several new techniques, predominantly, a new linear programming rounding methodology and a constructive aspect of a well-known probabilistic method, the Lov\'{a}sz Local Lemma (LLL). Second, we employ these techniques to applications of resource allocation obtaining substantial improvements over known results. Our research also spurs new direction of study; we introduce new models for achieving energy efficiency in scheduling and a novel framework for assigning advertisements in cellular networks. Both of these lead to a variety of interesting questions. Our linear programming rounding methodology is a significant generalization of two major rounding approaches in the theory of approximation algorithms, namely the dependent rounding and the iterative relaxation procedure. Our constructive version of LLL leads to first algorithmic results for many combinatorial problems. In addition, it settles a major open question of obtaining a constant factor approximation algorithm for the Santa Claus problem. The Santa Claus problem is a $NP$-hard resource allocation problem that received much attention in the last several years. Through out this thesis, we study a number of applications related to scheduling jobs on unrelated parallel machines, such as provisionally shutting down machines to save energy, selectively dropping outliers to improve system performance, handling machines with hard capacity bounds on the number of jobs they can process etc. Hard capacity constraints arise naturally in many other applications and often render a hitherto simple combinatorial optimization problem difficult. In this thesis, we encounter many such instances of hard capacity constraints, namely in budgeted allocation of advertisements for cellular networks, overlay network design, and in classical problems like vertex cover, set cover and k-median.