Electrical & Computer Engineering Theses and Dissertations
Permanent URI for this collectionhttp://hdl.handle.net/1903/2765
Browse
7 results
Search Results
Item LOW-POWER AND SECURE IMPLEMENTATION OF NEURAL NETWORKS BY APPROXIMATION(2022) Xu, Qian; Qu, Gang; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Neural networks (NN), one type of machine learning (ML) algorithms, have emerged as a powerful paradigm for sophisticated applications such as pattern recognition and natural language processing. In this dissertation, we study how to apply the principle of approximate computing to solve two challenging problems in neural networks, namely energy efficiency and security. More specifically, we investigate approximation across three stacks in the implementation of NN models: computation units, data storage, and the NN model itself. Computation units, such as adders and multipliers, have been one of the main targets for power-efficient implementation of neural networks. Many low-power approximate adders and multipliers have been proposed. NNs also require complex operations like logarithms, despite the heavy usage and high energy consumption on such operations, they are not optimized for energy efficiency. Our first contribution is a truncation-based approximation method that can balance the computation precision and energy consumption of the popular floating-point logarithmic operation. We derive a formula for the most energy-efficient implementation of the logarithm unit given an error variance range. Based on this theoretical result, we propose BWOLF (Bit-Width Optimization for Logarithmic Function) to determine the bit-width of operands in the logarithms computation in order to minimize the energy consumption in delivering the required computation precision. We evaluate the efficacy of BWOLF in terms of energy savings on two widely used applications: Kullback-Leibler Divergence and Bayesian Neural Network. The experimental results validate the correctness of our analysis and show significant energy savings, from 27.18% to 95.92%, over the full-precision computation and a baseline approximation method based on uniform truncation. Storage approximation by reducing the supply voltage for dynamic random access memory (DRAM) is effective in saving the power for neural networks. However, this will introduce physical errors in DRAM and could impact the performance of NN models. In the second part of this dissertation, we explore the potential of storage approximation in improving NN system’s security in training data privacy. More specifically, we consider the Model Inversion Attacks (MIAs) that extrapolate the training data from model parameters. Our proposed solution -- MIDAS: Model Inversion Defenses with an Approximate memory System -- intentionally introduces memory faults by overscaling voltage to thwart MIA without compromising the original ML model. We use detailed SPICE simulations to build the DRAM fault model and evaluate MIDAS against state-of-the-art MIAs. Experiments demonstrate that MIDAS can effectively protect training data from run-time MIAs. In terms of the Pearson Correlation Coefficient (PCC) similarity between the original training data and the recovered version, MIDAS reduces the PCC value by 55% and 40% for shallow and deep neural networks under 1.5% accuracy relaxation. Although MIDAS shows promising security benefits through storage approximation, such approximation modifies the neural network parameters and may reduce the NN model’s accuracy. In the third part of this dissertation, we propose model approximation which aims at generating an approximate NN model to correct the errors during training and consequently reduce the possible degradation of NN’s classification results. We demonstrate this concept on gradient inversion attacks which utilize transmitted gradients between the nodes in a federated learning system to reconstruct the training data. Therefore, we propose DAGIA, a Data Augmentation defense against Gradient Inversion Attacks, to deliberately extend the training dataset and report the corresponding gradient updates to protect the original data. For multiple data augmentation techniques, we empirically evaluate the trade-off between test accuracy and information leakage to select the best technique for DAGIA. According to the Structural Similarity (SSIM) between reconstructed training data and the original CIFAR-10 dataset, the experimental results show that DAGIA can reduce the SSIM by 54% with a slightly increased test accuracy for the ConvNet model. In summary, this dissertation focuses on the role of approximation in energy efficiency and security during the implementation of neural networks. We show that computation units for complex operators can be approximated to reduce energy, the storage for neural network weights can be approximated to improve both energy efficiency and security (against information leak), and the NN model itself could be approximated during training for security enhancement. This dissertation work demonstrates that approximation is a promising method to improve the performance of neural networks. It opens the door to applying the principle of approximate computing to the implementation and optimization of neural networks where there are abundant opportunities for approximation.Item Approximate Computing Techniques for Low Power and Energy Efficiency(2018) Gao, Mingze; Qu, Gang; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Approximate computing is an emerging computation paradigm in the era of the Internet of things, big data and AI. It takes advantages of the error-tolerable feature of many applications, such as machine learning and image/signal processing, to reduce the resources consumption and delivers a certain level of computation quality. In this dissertation, we propose several data format oriented approximate computing techniques that will dramatically increase the power/energy efficiency with the insignificant loss of computational quality. For the integer computations, we propose an approximate integer format (AIF) and its associated arithmetic mechanism with controllable computation accuracy. In AIF, operands are segmented at runtime such that the computation is performed only on part of operands by computing units (such as adders and multipliers) of smaller bit-width. The proposed AIF can be used for any arithmetic operation and can be extended to fixed point numbers. AIF requires additional customized hardware support. We also provide a method that can optimize the bit-width of the fixed point computations that run on the general purpose hardware. The traditional bit-width optimization methods mainly focus on minimizing the fraction part since the integer part is restricted by the data range. In our work, we utilize the dynamic fixed point concept and the input data range as the prior knowledge to get rid of this limitation. We expand the computations into data flow graph (DFG) and propose a novel approach to estimate the error during propagation. We derive the function of energy consumption and apply a more efficient optimization strategy to balance the tradeoff between the accuracy and energy. Next, to deal with the floating point computation, we propose a runtime estimation technique by converting data into the logarithmic domain to assess the intermediate result at every node in the data flow graph. Then we evaluate the impact of each node to the overall computation quality, and decide whether we should perform an accurate computation or simply use the estimated value. To approximate the whole graph, we propose three algorithms to make the decisions at certain nodes whether these nodes can be truncated. Besides the low power and energy efficiency concern, we propose a design concept that utilizes the approximate computing to address the security concerns. We can encode the secret keys into the least significant bits of the input data, and decode the final output. In the future work, the input-output pairs will be used for device authentication, verification, and fingerprint.Item Security and Energy Efficiency in Resource-Constrained Wireless Multi-hop Networks(2016) Paraskevas, Evripidis; Baras, John S; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)In recent decades, there has been a huge improvement and interest from the research community in wireless multi-hop networks. Such networks have widespread applications in civil, commercial and military applications. Paradigms of this type of networks that are critical for many aspects of human lives are mobile ad-hoc networks, sensor networks, which are used for monitoring buildings and large agricultural areas, and vehicular networks with applications in traffic monitoring and regulation. Internet of Things (IoT) is also envisioned as a multi-hop network consisting of small interconnected devices, called ``things", such as smart meters, smart traffic lights, thermostats etc. Wireless multi-hop networks suffer from resource constraints, because all the devices have limited battery, computational power and memory. Battery level of these devices should be preserved in order to ensure reliability and communication across the network. In addition, these devices are not a priori designed to defend against sophisticated adversaries, which may be deployed across the network in order to disrupt network operation. In addition, the distributed nature of this type of networks introduces another limitation to protocol performance in the presence of adversaries. Hence, the inherit nature of this type of networks poses severe limitations on designing and optimizing protocols and network operations. In this dissertation, we focus on proposing novel techniques for designing more resilient protocols to attackers and more energy efficient protocols. In the first part of the dissertation, we investigate the scenario of multiple adversaries deployed across the network, which reduce significantly the network performance. We adopt a component-based and a cross-layer view of network protocols to make protocols secure and resilient to attacks and to utilize our techniques across existing network protocols. We use the notion of trust between network entities to propose lightweight defense mechanisms, which also satisfy performance requirements. Using cryptographic primitives in our network scenario can introduce significant computational overhead. In addition, behavioral aspects of entities are not captured by cryptographic primitives. Hence, trust metrics provide an efficient security metric in these scenarios, which can be utilized to introduce lightweight defense mechanisms applicable to deployed network protocols. In the second part of the dissertation, we focus on energy efficiency considerations in this type of networks. Our motivation for this work is to extend network lifetime, but at the same time maintain critical performance requirements. We propose a distributed sleep management framework for heterogeneous machine-to-machine networks and two novel energy efficient metrics. This framework and the routing metrics are integrated into existing routing protocols for machine-to-machine networks. We demonstrate the efficiency of our approach in terms of increasing network lifetime and maintaining packet delivery ratio. Furthermore, we propose a novel multi-metric energy efficient routing protocol for dynamic networks (i.e. mobile ad-hoc networks) and illustrate its performance in terms of network lifetime. Finally, we investigate the energy-aware sensor coverage problem and we propose a novel game theoretic approach to capture the tradeoff between sensor coverage efficiency and energy consumption.Item Age of Information and Energy Efficiency in Communication Networks(2015) Dutra da Costa, Maice; Ephremides, Anthony; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)This dissertation focuses on two important aspects of communication systems, namely energy efficiency and age of information. Both aspects have received much less attention than traditional performance metrics, such as throughput and delay. The need to improve the energy efficiency in communication networks is apparent, given the high demand for power consuming applications to be implemented in devices with limited energy supplies. Additionally, improvements in energy efficiency are encouraged by possible reductions in network operation costs, and by the increasing awareness of the environmental impact caused by the information and communication technologies. In this dissertation, energy efficiency is studied in the context of a cognitive wireless network, in which users have different priorities to access the network resources, possibly interfering and cooperating among themselves. A new parametrization is proposed to characterize performance trade-offs associated with energy efficiency for non-cooperative and cooperative network models. Additionally, a game theoretic model is proposed to study resource allocation in a cooperative cognitive network, accounting for energy efficiency in the utility functions. Age of information is a relatively new concept, which aims to characterize the timeliness of information. It is relevant to any system concerned with timeliness of information, and particularly relevant when information is used to make decisions, but the value of the information is degraded with time. This is the case in many applications of communications and control systems. In this dissertation, the age of information is first investigated for status update communication systems. The status updates are samples of a random process under observation, transmitted as packets, which also contain the time stamp to identify when the sample was generated. The age of information at the destination node is the time elapsed since the last received update was generated. The status update systems are modeled using queuing theory. We propose models for status update systems capable of managing the packets before transmission, aiming to avoid wasting network resources with the transmission of stale information. In addition to characterizing the average age, we propose a new metric, called peak age, which provides information about the maximum value of the age, achieved immediately before receiving an update. We also propose a new framework, based on the concept of age of information, to analyze the effect of outdated Channel State Information (CSI) on the performance of a communication link in which the source node acquires the CSI through periodic feedback from the destination node. The proposed framework is suitable to analyze the trade-off between performance and timeliness of the CSI, which is a fundamental step to design efficient adaptation functions and feedback protocols.Item HARDWARE AND SOFTWARE ARCHITECTURES FOR ENERGY- AND RESOURCE-EFFICIENT SIGNAL PROCESSING SYSTEMS(2014) Cho, Inkeun; Bhattacharyya, Shuvra S.; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)For a large class of digital signal processing (DSP) systems, design and implementation of hardware and software is challenging due to stringent constraints on energy and resource requirements. In this thesis, we develop methods to address this challenge by proposing new constraint-aware system design methods for DSP systems, and energy- and resource-optimized designs of key DSP subsystems that are relevant across various application areas. In addition to general methods for optimizing energy consumption and resource utilization, we present streamlined designs that are specialized to efficiently address platform-dependent constraints. We focus on two specific aspects in development of energy- and resource-optimized design techniques: (1) Application-specific systems and architectures for energy- and resource- efficient design. First, we address challenges in efficient implementation of wireless sensor network building energy monitoring systems (WSNBEMSs). We develop new energy management schemes in order to maximize system lifetime for WSNBEMSs, and demonstrate that system lifetime can be improved significantly without affecting monitoring accuracy. We also present resource efficient, field programmable gate array (FPGA) architecture for implementation of orthogonal frequency division multiplexing (OFDM) systems. We have demonstrated that our design provides at least 8.8% enhancement in terms of resource efficiency compared to Xilinx FFT v7.1 within the same OFDM configuration. (2) Dataflow-based methods for structured design and implementation of energy- and resource- efficient DSP systems. First, we introduce a dataflow-based design approach based on integrating interrupt-based signal acquisition in context of parameterized synchronous dataflow (PSDF) modeling. We demonstrate that by applying our approach, energy- and resource-efficient embedded software can be derived systematically from high level models of dynamic, data-driven applications systems (DDDASs) functional structure. Also, we present an in-depth development of lightweight dataflow-Verilog (LWDF-V), which is an integration of the LWDF programming model with the Verilog hardware description language (HDL), and we demonstrate the utility of LWDF-V for design and implementation of digital systems for signal processing. We emphasize efficient of LWDF with HDLs, and emphasize the application of LWDF-V to design DSP systems with dynamic parameters on FPGA platforms.Item SCALABLE AND ENERGY EFFICIENT DRAM REFRESH TECHNIQUES(2014) Bhati, Ishwar Singh; Jacob, Bruce; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)A DRAM cell requires periodic refresh operations to preserve data in its leaky capacitor. Previously, the overheads of refresh operations were insignificant. But, as both the size and speed of DRAM chips have increased significantly in the past decade, refresh has become a dominating factor of DRAM performance and power dissipation. The objective of this dissertation is to conduct a comprehensive study of the issues related to refresh operations in modern DRAM devices and thereafter, propose techniques to mitigate refresh penalties. To understand the growing consequences of refresh operations, first we describe various refresh command scheduling schemes; analyze the refresh modes and timings in modern commodity DRAM devices; and characterize the variations in DRAM cells' retention time. Then, we quantify refresh penalties by varying device speed, size, timings, and total memory capacity. Furthermore, we also summarize prior refresh mechanisms and their applicability in future computing systems. Finally, based on our experiments and observations, we propose techniques to improve refresh energy efficiency and mitigate refresh scalability problems. Refresh operations not only introduce performance penalty but also pose energy overheads. In addition to the energy required for refreshing, the background energy component, dissipated by DRAM peripheral circuitry and on-die DLL during refresh command, will become significant in future devices. We propose a set of techniques referred collectively as "coordinated refresh", in which scheduling of low power modes and refresh commands are coordinated so that most of the required refreshes are issued when the DRAM device is in the deepest low power "self refresh" (SR) mode. Our approach saves background power because the peripheral circuitry and clocks are turned off in the SR mode. Moreover, we observe that as the number of rows in DRAM scales, a large body of research on refresh reduction using retention time and access awareness will be rendered ineffective. Because these mechanisms require the memory controller to have fine-grained control over which regions of the memory are refreshed, while in JEDEC DDRx devices, a refresh operation is carried out via an "auto-refresh" command, which refreshes multiple rows from multiple banks simultaneously. The internal implementation of "auto-refresh" is completely opaque outside the DRAM -- all the memory controller can do is tell the DRAM to refresh itself -- the DRAM handles everything else, in particular determining which rows in which banks are to be refreshed. We propose a modification to the DRAM that extends its existing control-register access protocol to include the DRAM's internal refresh counter and also introduce a new "dummy refresh" command that skips refresh operations and simply increments the internal counter. We show that these modifications allow a memory controller to reduce as many refreshes as in prior work, while achieving significant energy and performance advantages by using auto-refresh most of the time.Item Cooperative Detection and Network Coding in Wireless Networks(2009) Baidas, Mohammed Wael; Ray Liu, Prof. K. J.; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)In cooperative communication systems, multiple terminals in wireless networks share their antennas and resources for information exchange and processing. Recently, cooperative communications have been shown to achieve significant performance improvements in terms of transmission reliability, coverage area extension, and network throughput, with respect to existing classical communication systems. This dissertation is focused on two important applications of cooperative communications, namely: (i) cooperative distributed detection in wireless sensor networks, and (ii) many-to-many communications via cooperative space-time network coding. The first application of cooperative communications presented in this dissertation is concerned with the analysis and modeling of the deployment of cooperative relay nodes in wireless sensor networks. Particularly, in dense wireless sensor networks, sensor nodes continuously observe and collect measurements of a physical phenomenon. Such observations can be highly correlated, depending on the spatial separation between the sensor nodes as well as how the physical properties of the phenomenon are evolving over time. This unique characteristic of wireless sensor networks can be effectively exploited with cooperative communications and relays deployment such that the distributed detection performance is significantly improved as well as the energy efficiency. In particular, this dissertation studies the Amplify-and-Forward (AF) relays deployment as a function of the correlation of the observations and analyzes the achievable spatial diversity gains as compared with the classical wireless sensor networks. Moreover, it is demonstrated that the gains of cooperation can be further leveraged to alleviate bandwidth utilization inefficiencies in current sensor networks. Specifically, the deployment of cognitive AF cooperative relays to exploit empty/under-utilized time-slots and the resulting energy savings are studied, quantified and compared. The multiple terminal communication and information exchange form the second application of cooperative communications in this dissertation. Specifically, the novel concept of Space-Time-Network Coding (STNC) that is concerned with formulation of the many-to-many cooperative communications over Decode-and-Forward (DF) nodes is studied and analyzed. Moreover, the exact theoretical analysis as well as upper-bounds on the network symbol error rate performance are derived. In addition, the tradeoff between the number of communicating nodes and the timing synchronization errors is analyzed and provided as a network design guideline. With STNC, it is illustrated that cooperative diversity gains are fully exploited per node and significant performance improvements are achieved. It is concluded that the STNC scheme serves as a potential many-to-many cooperative communications scheme and that its scope goes much further beyond the generic source-relay-destination communications.