Flexible and Efficient Control of Data Transfers for Loosely Coupled Components

Abstract

Allowing loose coupling between the components of complex applications has many advantages, such as flexibility in the components that can participate and making it easier to model multiscale physical phenomena.

To support coupling of parallel and sequential application components, I have designed and implemented a loosely coupled framework which has the following characteristics: (1) connections between participating components are separately identified from the individual components, (2) all data transfers between data exporting and importing components are determined by a runtime-based low overhead method (approximate match), (3) two runtime-based optimization approaches, collective buffering and inverse-match cache, are applied to speed up the applications in many common coupling modes, and (4) a multi-threaded multi-process control protocol that can be systematically constructed by the composition of sub-tasks protocols.

The proposed framework has been applied to two real world applications, and the deployment approach and runtime performance are also studied. Currently the framework runs on x86 Linux clusters, and porting strategies for multicore x86 processors and advanced high performance computer architectures are also explored.