Electrical & Computer Engineering Theses and Dissertations
Permanent URI for this collectionhttp://hdl.handle.net/1903/2765
Browse
4 results
Search Results
Item NEW APPROACHES FOR ANALYZING SYSTEMS WITH HISTORY-DEPENDENT EFFICIENCY(2020) Lin, Michael; La, Richard J; Martins, Nuno C; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)In my dissertational work, I propose two novel models for analyzing systems in which the operational efficiency depends on the past history, e.g., systems with human-in-the-loop and energy harvesting sensors. First, I investigate a queuing system with a single server that serves multiple queues with different types of tasks. The server has a state that is affected by the current and past actions. The task completion probability of each kind of task is a function of the server state. A task scheduling policy is specified by a function that determines the probability of assigning a task to the server. The main results with multiple types of tasks include: (i) necessary and sufficient conditions for the existence of a randomized stationary policy that stabilizes the queues; and (ii) the existence of threshold type policies that can stabilize any stabilizable system. For a single type system, I also identify task scheduling policies under which the utilization rate is arbitrarily close to that of an optimal policy that minimizes the utilization rate. Here, the utilization rate is defined to be the long-term fraction of time the server is required to work. Second, I study a remote estimation problem over an activity packet drop link. The link undergoes packet drops and has an (activity) state that is influenced by past transmission requests. The packet-drop probability is governed by a given function of the link's state. A scheduler determines the probability of a transmission request regarding the link's state. The main results include: (i) necessary and sufficient conditions for the existence of a randomized stationary policy that stabilizes the estimation error in the second-moment sense; and (ii) the existence of deterministic policies that can stabilize any stabilizable system. The second result implies that it suffices to search for deterministic strategies for stabilizing the estimation error. The search can be further narrowed to threshold policies when the function for the packet-drop probability is non-decreasing.Item Model Based Optimization and Design of Secure Systems(2013) Malik, Waseem Ansar; Martins, Nuno C; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)ABSTRACT Title of dissertation: MODEL BASED OPTIMIZATION AND DESIGN OF SECURE SYSTEMS Waseem Ansar Malik, Doctor of Philosophy, 2013 Dissertation directed by: Prof. Nuno C. Martins Department of Electrical and Computer Engineering University of Maryland, College Park Dr. Ananthram Swami Computational and Information Sciences Directorate Army Research Laboratory Control systems are widely used in modern industry and find wide applications in power systems, nuclear and chemical plants, the aerospace industry, robotics, communication devices, and embedded systems. All these systems typically rely on an underlying computing and networking infrastructure which has considerable security vulnerabilities. The biggest threat, in this age and time, to modern systems are cyber attacks from adversaries. Recent cyber attacks have practically shut down government websites affecting government operation, undermined financial institutions, and have even infringed on public privacy. Thus it is extremely important to conduct studies on the design and analysis of secure systems. This work is an effort in this research direction and is mainly focused on incorporating security in the design of modern control systems. In the first part of this dissertation, we present a linear quadratic optimal control problem subjected to security constraints. We consider an adversary which can make partial noisy measurements of the state. The task of the controller is to generate control sequences such that the adversary is unable to estimate the terminal state. This is done by minimizing a quadratic cost while satisfying security constraints. The resulting optimization problems are shown to be convex and the optimal solution is computed using Lagrangian based techniques. For the case when the terminal state has a discrete distribution the optimal solution is shown to be nonlinear in the terminal state. This is followed by considering the case when the terminal state has a continuous distribution. The resulting infinite dimensional optimization problems are shown to be convex and the optimal solution is proven to be affine in the terminal state. In the next part of this dissertation, we analyze several team decision problems subjected to security constraints. Specifically, we consider problem formulations where there are two decision makers each controlling a different dynamical system. Each decision maker receives information regarding the respective terminal states that it is required to reach and applies a control sequence accordingly. An adversary makes partial noisy measurements of the states and tries to estimate the respective terminal states. It is shown that the optimal solution is affine in the terminal state when it is identical for both systems. We also consider the general case where the terminal states are correlated. The resulting infinite dimensional optimization problems are shown to convex programs and we prove that the optimal solution is affine in the information available to the decision makers. Next, a stochastic receding horizon control problem is considered and analyzed. Specifically, we consider a system with bounded disturbances and hard bounds on the control inputs. Utilizing a suboptimal disturbance feedback scheme, the optimization problem is shown to be convex. The problem of minimizing the empirical mean of the cost function is analyzed. We provide bounds on the disturbance sample size to compute the empirical minimum of the problem. Further, we consider the problem where there are hard computational constraints and complex on-line optimization is not feasible. This is addressed by randomly generating both the control inputs and the additive disturbances. Bounds on sample sizes are provided which guarantee a notion of a probable near minimum. Model uncertainty is also incorporated into the framework and relevant bounds are provided which guarantee a probable near minimax value. This work finds many applications in miniature devices and miniature robotics. In the final part of this dissertation, we consider a centralized intrusion detection problem with jointly optimal sensor placement. A team of sensors make measurements regarding the presence of an intruder and report their observations to a decision maker. The decision maker solves a jointly optimal detection and sensor placement problem. For the case when the number of sensors is equal to the number of placement points, we prove that uniform placement of sensors is not strictly optimal. We introduce and utilize a majorization based partial order for the placement of sensors. For the case when the number of sensors is less than or equal to six, we show that for a fixed local probability of detection (probability of false alarm) increasing the probability of false alarm (probability of detection) results in optimal placements that are higher on a majorization based partial order.Item Stochastic Systems with Cumulative Prospect Theory(2013) Lin, Kun; Marcus, Steven I.; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Stochastic control problems arise in many fields. Traditionally, the most widely used class of performance criteria in stochastic control problems is risk-neutral. More recent attempts at introducing risk-sensitivity into stochastic control problems include the application of utility functions. The decision theory community has long debated the merits of using expected utility for modeling human behaviors, as exemplified by the Allais paradox. Substantiated by strong experimental evidence, Cumulative Prospect Theory (CPT) based performance measures have been proposed as alternatives to expected utility based performance measures for evaluating human-centric systems. Our goal is to study stochastic control problems using performance measures derived from the cumulative prospect theory. The first part of this thesis solves the problem of evaluating Markov decision processes (MDPs) using CPT-based performance measures. A well-known method of solving MDPs is dynamic programming, which has traditionally been applied with an expected utility criterion. When the performance measure is CPT-inspired, several complications arise. Firstly, when solving a problem via dynamic programming, it is important that the performance criterion has a recursive structure, which is not true for all CPT-based criteria. Secondly, we need to prove the traditional optimality criteria for the updated problems (i.e., MDPs with CPT-based performance criteria). The theorems stated in this part of the thesis answer the question: what are the conditions required on a CPT-inspired criterion such that the corresponding MDP is solvable via dynamic programming? The second part of this thesis deals with stochastic global optimization problems. Using ideas from the cumulative prospect theory, we are able to introduce a novel model-based randomized optimization algorithm: Cumulative Weighting Optimization (CWO). The key contributions of our research are: 1) proving the convergence of the algorithm to an optimal solution given a mild assumption on the initial condition; 2) showing that the well-known cross-entropy optimization algorithm is a special case of CWO-based algorithms. To the best knowledge of the author, there is no previous convergence proof for the cross-entropy method. In practice, numerical experiments have demonstrated that a CWO-based algorithm can find a better solution than the cross-entropy method. Finally, in the future, we would like to apply some of the ideas from cumulative prospect theory to games. In this thesis, we present a numerical example where cumulative prospect theory has an unexpected effect on the equilibrium points of the classic prisoner's dilemma game.Item Simulation-based Methods for Stochastic Control and Global Optimization(2011) Wang, Yongqiang; Marcus, Steven I; Fu, Michael C; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Ideas of stochastic control have found applications in a variety of areas. A subclass of the problems with parameterized policies (including some stochastic impulse control problems) has received significant attention recently because of emerging applications in the areas of engineering, management, and mathematical finance. However, explicit solutions for this type of stochastic control problems only exist for some special cases, and effective numerical methods are relatively rare. Deriving efficient stochastic derivative estimators for payoff functions with discontinuities arising in many problems of practical interest is very challenging. Global optimization problems are extremely hard to solve due to the typical multimodal properties of objective functions. With the increasing availability of computing power and memory, there is a rapid development in the merging of simulation and optimization techniques. Developing new and efficient simulation-based optimization algorithms for solving stochastic control and global optimization problems is the primary goal of this thesis. First we develop a new simulation-based optimization algorithm to solve a stochastic control problem with a parameterized policy that arises in the setting of dynamic pricing and inventory control. We consider a joint dynamic pricing and inventory control problem with continuous stochastic demand and model the problem as a stochastic control problem. An explicit solution is given when a special demand model is considered. For general demand models with a parameterized policy, we develop a new simulation-based method to solve this stochastic control problem. We prove the convergence of the algorithm and show the effectiveness of the algorithm by numerical experiments. In the second part of this thesis, we focus on the problem of estimating the derivatives for a class of discontinuous payoff functions, for which existing methods are either not valid or not efficient. We derive a new unbiased stochastic derivative estimator for performance functions containing indicator functions. One important feature of this new estimator is that it can be computed from a single sample path or simulation, whereas existing estimators in the literature require additional simulations. Finally we propose a new framework for solving global optimization problems by establishing a connection with evolutionary games, and show that a particular equilibrium set of the evolutionary game is asymptotically stable. Based on this connection, we propose a Model-based Evolutionary Optimization (MEO) algorithm, which uses probabilistic models to generate new candidate solutions and uses dynamics from evolutionary game theory to govern the evolution of the probabilistic models. MEO gives new insight into the mechanism of model updating in model-based global optimization algorithms from the perspective of evolutionary game theory. Furthermore, it opens the door to developing new algorithms by using various learning algorithms and analysis techniques from evolutionary game theory.