Multiple Range Query Optimization with Distributed Cache Indexing

Abstract

MQO is a distributed multiple query processing middleware that can optimize query processing for data analysis applications on the Grid. It has one or more proxies that act as front-end to a collection of backend servers. The basic idea behind this architecture is semantic caching, whereby queries can leverage available cached results in the proxy either directly or through transformations. While this approach has been shown to speed up query evaluation under multi-client workloads, the caching infrastructure in the backend servers is not well used for query planning. In this paper, we describe a distributed multidimensional indexing scheme that enables the proxy to directly consider the cache contents available at the backend servers for planning and scheduling. This approach is shown to produce better query plans and faster query response times. We experimentally demonstrate that system throughput can be improved up to 66%, compared to either load-based or round-robin scheduling.