Subject

Natural Language Explanations (NLEs) represent a significant advancement in the realm of explainable artificial intelligence, particularly within the context of medical imaging. The primary objective of this research is to develop systems that not only predict outcomes, such as classifying medical images, but also provide detailed textual explanations that elucidate the reasoning behind these predictions. This dual capability aims to enhance the interpretability and trustworthiness of AI models in medical applications. Despite the potential of NLEs, the current availability of relevant datasets is limited, posing a substantial barrier to progress in this field. However, the emergence of large language models (LLMs) and multimodal large models (MLMs) offers promising avenues for generating new datasets and improving model performance through innovative approaches.

Kind of work

The thesis will focus on leveraging the capabilities of LLMs and MLMs to create and utilize new datasets for NLEs in medical imaging. The work will involve two main components. First, the student will need to convert existing medical reports and images into prediction-explanation pairs. This will be achieved by prompting LLMs with medical reports to reformulate them into NLE datasets. Second, the student will explore the potential of MLMs to generate textual annotations directly from medical images, even in the absence of associated textual reports. This task will require the student to fine-tune and evaluate models on the newly formulated datasets, aiming to improve their explanatory power and accuracy in medical imaging applications.

Framework of the Thesis

The framework of this master thesis will be structured in several key phases. Initially, a comprehensive literature review will be conducted to understand the current state of NLEs and the application of LLMs and MLMs in medical imaging. Following this, the student will design a methodology for dataset creation, involving the conversion of medical reports and the use of MLMs for image annotation. The next phase will involve the implementation of these methodologies, including the collection of data, preprocessing, and the development of model training pipelines using Python and PyTorch. The student will then evaluate the performance of the trained models, using metrics that assess both prediction accuracy and the quality of explanations. The final phase will involve documenting the findings, discussing the implications of the results, and proposing future research directions.

Number of Students

1

Expected Student Profile

The ideal candidate for this master thesis should possess a robust background in Python programming, with specific expertise in PyTorch and Transformer models, both in theory and practical implementation. Familiarity with the principles of natural language processing and machine learning is crucial, as the project involves advanced techniques in these areas. Additionally, experience with medical data and an understanding of the challenges associated with medical imaging will be beneficial. The complexity and interdisciplinary nature of this research require a highly motivated individual with strong analytical skills and the ability to independently manage and execute research tasks. Without these foundational skills and knowledge, the student may encounter significant difficulties in successfully completing the thesis.