A. James Clark School of Engineering
Permanent URI for this communityhttp://hdl.handle.net/1903/1654
The collections in this community comprise faculty research works, as well as graduate theses and dissertations.
Browse
2 results
Search Results
Item Stochastic Systems with Cumulative Prospect Theory(2013) Lin, Kun; Marcus, Steven I.; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Stochastic control problems arise in many fields. Traditionally, the most widely used class of performance criteria in stochastic control problems is risk-neutral. More recent attempts at introducing risk-sensitivity into stochastic control problems include the application of utility functions. The decision theory community has long debated the merits of using expected utility for modeling human behaviors, as exemplified by the Allais paradox. Substantiated by strong experimental evidence, Cumulative Prospect Theory (CPT) based performance measures have been proposed as alternatives to expected utility based performance measures for evaluating human-centric systems. Our goal is to study stochastic control problems using performance measures derived from the cumulative prospect theory. The first part of this thesis solves the problem of evaluating Markov decision processes (MDPs) using CPT-based performance measures. A well-known method of solving MDPs is dynamic programming, which has traditionally been applied with an expected utility criterion. When the performance measure is CPT-inspired, several complications arise. Firstly, when solving a problem via dynamic programming, it is important that the performance criterion has a recursive structure, which is not true for all CPT-based criteria. Secondly, we need to prove the traditional optimality criteria for the updated problems (i.e., MDPs with CPT-based performance criteria). The theorems stated in this part of the thesis answer the question: what are the conditions required on a CPT-inspired criterion such that the corresponding MDP is solvable via dynamic programming? The second part of this thesis deals with stochastic global optimization problems. Using ideas from the cumulative prospect theory, we are able to introduce a novel model-based randomized optimization algorithm: Cumulative Weighting Optimization (CWO). The key contributions of our research are: 1) proving the convergence of the algorithm to an optimal solution given a mild assumption on the initial condition; 2) showing that the well-known cross-entropy optimization algorithm is a special case of CWO-based algorithms. To the best knowledge of the author, there is no previous convergence proof for the cross-entropy method. In practice, numerical experiments have demonstrated that a CWO-based algorithm can find a better solution than the cross-entropy method. Finally, in the future, we would like to apply some of the ideas from cumulative prospect theory to games. In this thesis, we present a numerical example where cumulative prospect theory has an unexpected effect on the equilibrium points of the classic prisoner's dilemma game.Item Progressive Source-Channel Coding for Multimedia Transmission over Noisy and Lossy Channels with and without Feedback(2004-07-27) Chande, Vinay; Farvardin, Nariman; Electrical Engineering; Digital Repository at the University of Maryland; University of Maryland (College Park, Md.)Rate-scalable or layered lossy source-coding is useful for progressive transmission of multimedia sources, where the receiver can reconstruct the source incrementally. This thesis considers ``joint source-channel'' schemes for such a progressive transmission, in the presence of noise or loss, with and without the use of a feedback link. First we design image communication schemes for memoryless and finite state channels using limited and explicitly constrained use of the feedback channel in the form of a variable incremental redundancy Hybrid ARQ protocol. Constraining feedback allows a direct comparison with schemes without feedback. Optimized feedback based systems are shown to have useful gains. Second, we develop a controlled Markov chain approach for constrained feedback Hybrid ARQ protocol design. The proposed methodology allows the protocol to be chosen from a collection of signal flow graphs, and also allows explicit control over the tradeoffs in throughput, reliability and complexity. Next we consider progressive image transmission in the absence of feedback. We assign unequal error protection to the bits of a rate-scalable source-coder using rate compatible channel codes. We show that, under the framework, the source and channel bits can be ``scheduled'' in a single bitstream in such a way that operational optimality is retained for different transmission budgets, creating a rate-scalable joint source-channel coder. Next we undertake the design of a joint source-channel decoder that uses ``distortion aware'' ACK/NACK feedback generation. For memoryless channels, and Type-I HARQ, the design of optimal ACK/NACK generation and decoding by packet combining is cast and solved as a sequential decision problem. We obtain dynamic programming based optimal solutions and also propose suboptimal, lower complexity distortion-aware decoders and feedback generation rules which outperform conventional BER based rules such as CRC-check. Finally we design operational rate-distortion optimal ACK/NACK feedback generation rules for transmitting a tree structured quantizer over a memoryless channel. We show that the optimal feedback generation rules are embedded, that is, they allow incremental switching to higher rates during the transmission. Also, we obtain the structure of the feedback generation rules in terms of a feedback threshold function that simplifies the implementation.