From Computed Tomography Scans to Multiplex Immunofluorescence Slides: Image-based Computational Models to Improve Diagnostics and Treatments

Abstract

Medical images, such as computed tomography (CT) and multiplex immunofluorescence (mIF) slides, play a critical role in the prevention, diagnosis, and treatment of diseases. CT scans are non-invasive and can provide clinicians with detailed images of the heart and blood vessels. These images, accompanied with echocardiograms, offer the ability to simulate blood flow through arteries. On the other hand, mIF slides are cellular-level images used to detect multiple biomarkers within a single tissue section. This tissue is often obtained invasively during a biopsy or the removal of a tumor. This work consists of building and improving computational models that leverage both types of images. First, I study the hemodynamics of Type B aortic dissection (TBAD) through fluid-structure interaction (FSI) models that simulate blood flow in the aorta of three patient-specific geometries. These models are created by (1) manually segmenting CT scans, (2) modeling and meshing the fluid and the solid components, (3) identifying the dissection flap within the solid mesh, (4) running FSI simulations, and (5) analyzing the raw FSI results. To analyze multiple patients simultaneously and achieve large-scale parallelization in FSI results, I developed a new pipeline that automates parts of steps 2 to 5. Second, I discuss the generation of meaningful deep learning features from mIF slides of patients with cancer. Due to their large size, the analysis of mIF slides requires that clinicians select regions of interest (ROI), which themselves contain millions of pixels. Therefore, we focus on generating a smaller, but powerful representation of each ROI using a visual transformer, which allows the analysis of the whole slide. My main goal is to extract features that keep information that can be used to classify the tumor immune microenvironment (TIME) as hot or cold using mIF images. Finally, to address the bottleneck in the first step of the FSI study of TBAD, I propose a semi-supervised method to automatically segment different parts of the aorta when only a few labels are available.