MedAI #124: SleepFM: Multi-modal Representation Learning for Sleep | Rahul Thapa

Title: SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signals Speaker: Rahul Thapa Abstract: Sleep is a complex physiological process evaluated through various modalities recording electrical brain, cardiac, and respiratory activities. We curate a large polysomnography dataset from over 14,000 participants comprising over 100,000 hours of multi-modal sleep recordings. Leveraging this extensive dataset, we developed SleepFM, the first multi-modal foundation model for sleep analysis. We show that a novel leave-one-out approach for contrastive learning significantly improves downstream task performance compared to representations from standard pairwise contrastive learning. A logistic regression model trained on SleepFM's learned embeddings outperforms an end-to-end trained convolutional neural network (CNN) on sleep stage classification (macro AUROC 0.88 vs 0.72 and macro AUPRC 0.72 vs 0.48) and sleep disordered breathing detection (AUROC 0.85 vs 0.69 and AUPRC 0.77 vs 0.61). Notably, the learned embeddings achieve 48% top-1 average accuracy in retrieving the corresponding recording clips of other modalities from 90,000 candidates. This work demonstrates the value of holistic multi-modal sleep modeling to fully capture the richness of sleep recordings. Speaker Bio: Rahul is a second-year PhD student at Stanford University, working under the guidance of Dr. James Zou, with a primary focus on AI in biomedicine. Currently, he is also a research intern at Together AI, where he concentrates on multimodal models. Rahul's research interests lie in the development and application of multimodal models. He has experience working with sleep signal data and biomedical image, text, and video data. ------ The MedAI Group Exchange Sessions are a platform where we can critically examine key topics in AI and medicine, generate fresh ideas and discussion around their intersection and most importantly, learn from each other. We will be having weekly sessions where invited speakers will give a talk presenting their work followed by an interactive discussion and Q&A. Our sessions are held every Monday from 1pm-2pm PST. To get notifications about upcoming sessions, please join our mailing list: https://mailman.stanford.edu/mailman/listinfo/medai_announce For more details about MedAI, check out our website: https://medai.stanford.edu. You can follow us on Twitter @MedaiStanford Organized by members of the Rubin Lab (http://rubinlab.stanford.edu) and Machine Intelligence in Medicine and Imaging (MI-2) Lab: - Nandita Bhaskhar (https://www.stanford.edu/~nanbhas) - Amara Tariq (https://www.linkedin.com/in/amara-tariq-475815158/) - Avisha Das (https://dasavisha.github.io/)