DECENTRALIZED AND SCALABLE RESOURCE MANAGEMENT FOR DESKTOP GRIDS

Abstract

The recent growth of the Internet and the CPU power of personal computers and workstations enables desktop grid computing to achieve tremendous computing power with low cost, through opportunistic sharing of resources. However, traditional server-client Grid architectures have inherent problems in robustness, reliability and scalability. Researchers have therefore recently turned to Peer-to-Peer (P2P) algorithms in an attempt to address these issues. I have designed and evaluated a set of protocols that implement a scalable P2P desktop grid computing system for executing Grid applications on widely distributed sets of resources. Such infrastructure must be decentralized, robust, highly available and scalable, while effectively mapping application instances to available resources throughout the system (called matchmaking). First of all, I address the problem of efficient matchmaking of jobs to available system resources by employing customized Content-Addressable Network (CAN) where each resource type corresponds to a distinct dimension. With this approach, incoming jobs are matched with system nodes through proximity in an N-dimensional resource space. Second, I provide comprehensive load balancing mechanisms that can greatly improve overall system throughput and response time without using any centralized control or information about the system. Finally, to remove any hot spots in the system where a small number of nodes are processing a lot of system maintenance work, I have designed a set of optimizations to minimize overall system overheads and distribute them fairly among available system nodes. My ultimate goal is to ensure that no node in the system becomes much more heavily loaded than others, either because of executing jobs or from system maintenance tasks. This is because every node in our system is a peer, so that no node is acting as a pure server or a pure client. Throughout extensive experimental results, I show that the resulting P2P desktop grid computing system is scalable and effective so that it can efficiently match any type of resource requirements for jobs simultaneously, while balancing load among multiple candidate nodes.