Optimizing the Execution of Batches of Data Analysis Queries

Abstract

Data analysis applications such as Kronos, a remote sensing application, and the Virtual Microscope, a telepathology application, require operating on and processing large datasets. In such situations, it is important that the storage, retrieval, and manipulation of these datasets is efficiently handled. Past research focused on the creation of database systems that abstract the data analysis process into a framework facilitating the design of algorithms to optimize the execution of scientific queries and batches of queries. These optimizations occur at different levels in the query processing chain in the database system. The present research deals with the optimizations performed by the database system when processing batches of queries. Various algorithms to optimize the memory utilization of multiple data analysis queries are presented and the effect of each on query processing performance as well as their performance are investigated.