Upcoming Talks

LoRKD: Low-Rank Knowledge Decomposition for Medical Foundation Models

The widespread adoption of large-scale pre-training techniques has significantly advanced the development of medical foundation models, enabling them to serve as versatile tools across a broad range of medical tasks. However, despite their strong generalization capabilities, medical foundation models pre-trained on large-scale datasets tend to suffer from domain gaps between heterogeneous data, leading to suboptimal performance on specific tasks compared to specialist models, as evidenced by previous studies. In this paper, we explore a new perspective called ‘Knowledge Decomposition’ to improve the performance on specific medical tasks, which deconstructs the foundation model into multiple lightweight expert models, each dedicated to a particular anatomical region, with the aim of enhancing specialization and simultaneously reducing resource consumption. To accomplish the above objective, we propose a novel framework named Low-Rank Knowledge Decomposition (LoRKD), which explicitly separates gradients from different tasks by incorporating low-rank expert modules and efficient knowledge separation convolution. The low-rank expert modules resolve gradient conflicts between heterogeneous data from different anatomical regions, providing strong specialization at lower costs. The efficient knowledge separation convolution significantly improves algorithm efficiency by achieving knowledge separation within a single forward propagation. Extensive experimental results on segmentation and classification tasks demonstrate that our decomposed models not only achieve state-of-the-art performance but also exhibit superior transferability on downstream tasks, even surpassing the original foundation models in task-specific evaluations. Moreover, these compact expert models significantly reduce resource consumption, making them more suitable and efficient for practical deployment.

Jiangchao Yao is currently an assistant professor at Cooperative Medianet Innovation Center, Shanghai Jiao Tong University and a research scientist in Shanghai AI Laboratory. Before taking the faculty job, he was an algorithm expert in Data Analytics and Intelligence Lab, DAMO Academy, Alibaba Group, and received the dual PhD degree in Shanghai Jiao Tong University and University of Technology Sydney in 2019. His research mainly focuses on trustworthy machine learning and reasoning with the applications towards AI4Science. He has published more than 80 journal articles and conference papers, and contributed to two book chapters. He served as the area chair of ICML, NeurIPS and ICLR, and action editors of Transactions on Machine Learning Research and the journal Neural Networks. He has been selected as a MSRA Startrack Scholar in 2025.

Coming soon

Please add your questions to the speaker either to this google form or directly under the YouTube video