Medical Imaging AGI's Last Exam

The successful application of AI methods to solve real-world challenges in medical imaging requires validation against real-world data and relevant questions. Currently, benchmark datasets are not sufficiently aligned with the most clinically relevant scientific challenges, as research often uses readily available but clinically limited data. General AI systems (AGI) can flexibly handle a variety of tasks. Existing benchmarks in medical imaging so far mainly deal with basic computer vision tasks and thus provide benchmarks for simple but not for complex tasks. In addition, there is a lack of suitable methods for evaluating the generalization capability across different image modalities and tasks. MEDAL will close these gaps.

The project aims to bring together central clinical issues in medical imaging. Their processing by a medical imaging AGI could achieve performance levels that are comparable to or complement those of human experts and thus potentially have a practical impact on patients and caregivers. To this end, MEDAL will pool international expertise and multimodal data (medical images and optionally other patient information) and focus research more strongly on clinically significant challenges.