Institute for Systems Research Technical Reports
Permanent URI for this collectionhttp://hdl.handle.net/1903/4376
This archive contains a collection of reports generated by the faculty and students of the Institute for Systems Research (ISR), a permanent, interdisciplinary research unit in the A. James Clark School of Engineering at the University of Maryland. ISR-based projects are conducted through partnerships with industry and government, bringing together faculty and students from multiple academic departments and colleges across the university.
Browse
Search Results
Item Stability of Wireless Networks for Mode S Radar(2000) Chawla, Jay P.; Marcus, Steven I.; Shayman, Mark A.; Shayman, Mark; Marcus, Steven; ISRStability issues in a connectionless, one-hop queueing system featuringservers with overlapping service regions (e.g. a Mode Select (Mode S) Radarcommunications network or part of an Aeronautical Telecommunications Network (ATN) network) are considered, and a stabilizing policy is determined in closed-loop form. The cases of queues at the sources (aircraft) and queues at the servers (base stations) are consideredseparately. Stabilizability of the system with exponential service times and Poisson arrival rates is equivalent to the solvability of a linear program and if the system is stabilizable, a stabilizing open loop routingpolicy can be expressed in terms of the coefficients of the solution to thelinear program. We solve the linear program for the case of a single class of packets.The research and scientific content in this material has beenpublished under the same title in the Proceedings of the 32nd Conference onInformation Sciences and Systems; Princeton, NJ; March 1998. Item Solving POMDP by Onolicy Linear Approximate Learning Algorithm(1999) He, Qiming; Shayman, Mark A.; Shayman, Mark A.; ISRThis paper presents a fast Reinforcement Learning (RL) algorithm to solve Partially Observable Markov Decision Processes (POMDP) problem. The proposed algorithm is devised to provide a policyשּׂaking framework for Network Management Systems (NMS) which is in essence an engineering application without an exact model.The algorithm consists of two phases. Firstly, the model is estimated and policy is learned in a completely observable simulator. Secondly, the estimated model is brought into the partially observed realorld where the learned policy is then fineהּuned.
The learning algorithm is based on the onאּolicy linear gradientﬤescent learning algorithm with eligibility traces. This implies that the Qזּalue on belief space is linearly approximated by the Qזּalue at vertex over the belief space where onשּׁine TD method will be applied.
The proposed algorithm is tested against the exact solutions to extensive small/middleדּize benchmark examples from POMDP literature and found near optimal in terms of averageﬤiscountedגּeward and stepהּogoal. The proposed algorithm significantly reduces the convergence time and can easily be adapted to large stateאַumber problems.
Item Using POMDP as Modeling Framework for Network Fault Management(1999) He, Qiming; Shayman, Mark A.; Shayman, Mark A.; ISRFor highדּpeed networks, it is important that fault management be proactive--i.e., detect, diagnose, and mitigate problems before they result in severe degradation of network performance. Proactive fault manageשּׂent depends on monitoring the network to obtain the data on which to base manager decisions. However, monitoring introduces additional overhead that may itself degrade network performance especially when the network is in a stressed state. Thus, a tradeoff must be made between the amount of data collected and transferred on one hand, and the speed and accuracy of fault detection and diagnosis on the other hand. Such a tradeoff can be naturally formulated as a Partially Observable Markov decision process (POMDP).Since exact solution of POMDPs for a realistic number of states is computationally prohibitive, we develop a reinforcementשּׁearningﬢased fast algorithm which learns the decisionגּule in an approximate network simulator and makes it fast deployable to the real network. Simulation results are given to diagnose a switch fault in an ATM network. This approach can be applied to centralized fault management or to construct intelligent agents for distributed fault management.
Item Critical Points of Matrix Least Square Distance Functions(1992) Helmke, Uwe; Shayman, Mark A.; ISRA Classical problem in matrix analysis and total least squares estimation is that of finding a best approximant of a given matrix by lower rank ones. In this paper the critical points and the local minima are determined for the function on varieties of fixed-rank symmetric, skew-symmetric and rectangular matrices representing the distance to a fixed matrix. Our results extend earlier work of Eckart and Young and Higham.Item Dynamics and Control of Constrained Nonlinear Systems with Application to Robotics(1991) Chen, Xiaolin; Shayman, Mark A.; ISRA class of nonlinear constrained dynamic systems is studied. We first characterize the constrained submanifold and the constrained dynamics without using the vector relative degree. Applying the nonlinear feedback and exact linearization techniques to constrained systems, we discuss several control problems for the constrained dynamics such as asymptotically stabilization, asymptotically tracking reference outputs. Our results for the control of constrained nonlinear systems extend previous results, which is based on linear approximation and linear feedback.Item Topology of the Orbit Space of Generalized Linear Systems.(1989) Helmke, Uwe; Shayman, Mark A.; ISRWe investigate the topology of the orbit space of controllable generalized linear (descriptor) systems modulo restricted system equivalence. We compute the singular homology groups in the complex case, and prove that in both the real and complex cases, this space is a smooth compactification of the orbit space of controllable state space systems modulo system similarity.Item A Canonical Form for Controllable Singular Systems.(1988) Helmke, Uwe; Shayman, Mark A.; ISRA new canonical form for the action of restricted system equivalence on controllable singular systems is given. The construction of this form is based on the Weierstrass decomposition of the singular system into a slow and a fast subsystem. Both subsystems are transformed into Hermite canonical form. The resulting Hermite canonical form for singular systems has a particularly simple structure and is expected to be useful for e.g. identification purposes. Continuity properties of the Hermite form are investigated and the nonexistence of a globally defined continuous canonical form for controllable singular systems is shown.