International Journal of Science, Engineering and TechnologyAuthors: Assistant Professor Raghuvarancheerla, Professor Arivamudhan M
Abstract: Combining vision and language clinical information is important for intelligent diagnosis, but most medical vision-language models are black-boxed in the process of reasoning. This paper proposes an explainable vision-language framework for semantic analysis of medical images along with their diagnostic texts. The framework consists of a multi-scale vision encoder, a clinical text encoder, and a cross-modal alignment layer with an explanation generation pipeline. The model provides diagnostic predictions together with spatial evidence from the image and token-level evidence from text, allowing users to analyze the reasoning process behind every prediction. We experiment with two medical image datasets with their diagnostic texts and compare the framework with other unimodal and multimodal baselines in terms of diagnostic ability and explanation faithfulness. Our experiments show that adding explanation supervision does not hurt the diagnostic ability while greatly enhancing the clinical plausibility of explanations.
