Theses and Dissertations from UMD
Permanent URI for this communityhttp://hdl.handle.net/1903/2
New submissions to the thesis/dissertation collections are added automatically as they are received from the Graduate School. Currently, the Graduate School deposits all theses and dissertations from a given semester after the official graduation date. This means that there may be up to a 4 month delay in the appearance of a give thesis/dissertation in DRUM
More information is available at Theses and Dissertations at University of Maryland Libraries.
Browse
2 results
Search Results
Item LOW-POWER AND SECURE IMPLEMENTATION OF NEURAL NETWORKS BY APPROXIMATION(2022) Xu, Qian; Qu, Gang; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Neural networks (NN), one type of machine learning (ML) algorithms, have emerged as a powerful paradigm for sophisticated applications such as pattern recognition and natural language processing. In this dissertation, we study how to apply the principle of approximate computing to solve two challenging problems in neural networks, namely energy efficiency and security. More specifically, we investigate approximation across three stacks in the implementation of NN models: computation units, data storage, and the NN model itself. Computation units, such as adders and multipliers, have been one of the main targets for power-efficient implementation of neural networks. Many low-power approximate adders and multipliers have been proposed. NNs also require complex operations like logarithms, despite the heavy usage and high energy consumption on such operations, they are not optimized for energy efficiency. Our first contribution is a truncation-based approximation method that can balance the computation precision and energy consumption of the popular floating-point logarithmic operation. We derive a formula for the most energy-efficient implementation of the logarithm unit given an error variance range. Based on this theoretical result, we propose BWOLF (Bit-Width Optimization for Logarithmic Function) to determine the bit-width of operands in the logarithms computation in order to minimize the energy consumption in delivering the required computation precision. We evaluate the efficacy of BWOLF in terms of energy savings on two widely used applications: Kullback-Leibler Divergence and Bayesian Neural Network. The experimental results validate the correctness of our analysis and show significant energy savings, from 27.18% to 95.92%, over the full-precision computation and a baseline approximation method based on uniform truncation. Storage approximation by reducing the supply voltage for dynamic random access memory (DRAM) is effective in saving the power for neural networks. However, this will introduce physical errors in DRAM and could impact the performance of NN models. In the second part of this dissertation, we explore the potential of storage approximation in improving NN system’s security in training data privacy. More specifically, we consider the Model Inversion Attacks (MIAs) that extrapolate the training data from model parameters. Our proposed solution -- MIDAS: Model Inversion Defenses with an Approximate memory System -- intentionally introduces memory faults by overscaling voltage to thwart MIA without compromising the original ML model. We use detailed SPICE simulations to build the DRAM fault model and evaluate MIDAS against state-of-the-art MIAs. Experiments demonstrate that MIDAS can effectively protect training data from run-time MIAs. In terms of the Pearson Correlation Coefficient (PCC) similarity between the original training data and the recovered version, MIDAS reduces the PCC value by 55% and 40% for shallow and deep neural networks under 1.5% accuracy relaxation. Although MIDAS shows promising security benefits through storage approximation, such approximation modifies the neural network parameters and may reduce the NN model’s accuracy. In the third part of this dissertation, we propose model approximation which aims at generating an approximate NN model to correct the errors during training and consequently reduce the possible degradation of NN’s classification results. We demonstrate this concept on gradient inversion attacks which utilize transmitted gradients between the nodes in a federated learning system to reconstruct the training data. Therefore, we propose DAGIA, a Data Augmentation defense against Gradient Inversion Attacks, to deliberately extend the training dataset and report the corresponding gradient updates to protect the original data. For multiple data augmentation techniques, we empirically evaluate the trade-off between test accuracy and information leakage to select the best technique for DAGIA. According to the Structural Similarity (SSIM) between reconstructed training data and the original CIFAR-10 dataset, the experimental results show that DAGIA can reduce the SSIM by 54% with a slightly increased test accuracy for the ConvNet model. In summary, this dissertation focuses on the role of approximation in energy efficiency and security during the implementation of neural networks. We show that computation units for complex operators can be approximated to reduce energy, the storage for neural network weights can be approximated to improve both energy efficiency and security (against information leak), and the NN model itself could be approximated during training for security enhancement. This dissertation work demonstrates that approximation is a promising method to improve the performance of neural networks. It opens the door to applying the principle of approximate computing to the implementation and optimization of neural networks where there are abundant opportunities for approximation.Item Approximate Computing Techniques for Low Power and Energy Efficiency(2018) Gao, Mingze; Qu, Gang; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Approximate computing is an emerging computation paradigm in the era of the Internet of things, big data and AI. It takes advantages of the error-tolerable feature of many applications, such as machine learning and image/signal processing, to reduce the resources consumption and delivers a certain level of computation quality. In this dissertation, we propose several data format oriented approximate computing techniques that will dramatically increase the power/energy efficiency with the insignificant loss of computational quality. For the integer computations, we propose an approximate integer format (AIF) and its associated arithmetic mechanism with controllable computation accuracy. In AIF, operands are segmented at runtime such that the computation is performed only on part of operands by computing units (such as adders and multipliers) of smaller bit-width. The proposed AIF can be used for any arithmetic operation and can be extended to fixed point numbers. AIF requires additional customized hardware support. We also provide a method that can optimize the bit-width of the fixed point computations that run on the general purpose hardware. The traditional bit-width optimization methods mainly focus on minimizing the fraction part since the integer part is restricted by the data range. In our work, we utilize the dynamic fixed point concept and the input data range as the prior knowledge to get rid of this limitation. We expand the computations into data flow graph (DFG) and propose a novel approach to estimate the error during propagation. We derive the function of energy consumption and apply a more efficient optimization strategy to balance the tradeoff between the accuracy and energy. Next, to deal with the floating point computation, we propose a runtime estimation technique by converting data into the logarithmic domain to assess the intermediate result at every node in the data flow graph. Then we evaluate the impact of each node to the overall computation quality, and decide whether we should perform an accurate computation or simply use the estimated value. To approximate the whole graph, we propose three algorithms to make the decisions at certain nodes whether these nodes can be truncated. Besides the low power and energy efficiency concern, we propose a design concept that utilizes the approximate computing to address the security concerns. We can encode the secret keys into the least significant bits of the input data, and decode the final output. In the future work, the input-output pairs will be used for device authentication, verification, and fingerprint.