Dimitris Bertsimas
MIT, Sloan School of Management, Cambridge, MA (U.S.A.)

Plenary Lecture

Time Slot: Wednesday Afternoon

Room: AUD_A

Chair: Fabio Schoen

HAIM: Holistic AI for Medicine

Time: 14:30

Artificial intelligence (AI) systems hold great promise to improve healthcare over the nextdecades. Specifically, AI systems leveraging multiple data sources and input modalities arepoised to become a viable method to deliver more accurate results and deployable pipelinesacross a wide range of applications. In this work, we propose and evaluate a unified Holistic AIin Medicine (HAIM) framework to facilitate the generation and testing of AI systems thatleverage multimodal inputs. Our approach uses generalizable data pre-processing and machinelearning modeling stages that can be readily adapted for research and deployment inhealthcare environments. We evaluate our HAIM framework by training and characterizing14,324 independent models based on MIMIC-IV-MM, a multimodal clinical database (N=34,537samples) containing 7,279 unique hospitalizations and 6,485 patients, spanning all possibleinput combinations of 4 data modalities (i.e., tabular, time-series, text, and images), 11 uniquedata sources and 12 predictive tasks. We show that this framework can consistently androbustly produce models that outperform similar single-source approaches across varioushealthcare demonstrations (by 6-33%), including 10 distinct chest pathology diagnoses, alongwith length-of-stay and 48-hour mortality predictions. We also quantify the contribution ofeach modality and data source using Shapley values, which demonstrates the heterogeneity indata modality importance and the necessity of multimodal inputs across different healthcare-relevant tasks. The generalizable properties and flexibility of our Holistic AI in Medicine (HAIM)framework could offer a promising pathway for future multimodal predictive systems in clinicaland operational healthcare settings.