In my dissertational work, I propose two novel models for analyzing systems in which the operational efficiency depends on the past history, e.g., systems with human-in-the-loop and energy harvesting sensors.

First, I investigate a queuing system with a single server that serves multiple queues with different types of tasks. The server has a state that is affected by the current and past actions. The task completion probability of each kind of task is a function of the server state. A task scheduling policy is specified by a function that determines the probability of assigning a task to the server. The main results with multiple types of tasks include: (i) necessary and sufficient conditions for the existence of a randomized stationary policy that stabilizes the queues; and (ii) the existence of threshold type policies that can stabilize any stabilizable system. For a single type system, I also identify task scheduling policies under which the utilization rate is arbitrarily close to that of an optimal policy that minimizes the utilization rate. Here, the utilization rate is defined to be the long-term fraction of time the server is required to work.

Second, I study a remote estimation problem over an activity packet drop link. The link undergoes packet drops and has an (activity) state that is influenced by past transmission requests. The packet-drop probability is governed by a given function of the link's state. A scheduler determines the probability of a transmission request regarding the link's state. The main results include: (i) necessary and sufficient conditions for the existence of a randomized stationary policy that stabilizes the estimation error in the second-moment sense; and (ii) the existence of deterministic policies that can stabilize any stabilizable system. The second result implies that it suffices to search for deterministic strategies for stabilizing the estimation error. The search can be further narrowed to threshold policies when the function for the packet-drop probability is non-decreasing.