International Journal of Science, Engineering and TechnologyAuthors: Akanksha Aher, Swaraj Rasam, Dr. Jasbir Kaur, Ifrah Kampoo
Abstract: Artificial intelligence today faces a fundamental challenge: modern AI systems are trained on enormous quantities of data, yet most of that data is irrelevant to the task at hand. The Information Bottleneck (IB) principle offers a powerful theoretical answer — it teaches AI to retain only what is genuinely useful for a task while discarding everything else. This survey examines how IB applies to multimodal foundation models that process images and text simultaneously, such as CLIP, BLIP-2, LLaVA, and Flamingo. We explain the theory in accessible terms, review how these models apply IB in practice, and discuss real-world benefits, applications, and open challenges. Our goal is to bridge information-theoretic foundations and engineering practice for the MCA research community.
DOI: http://doi.org/10.5281/zenodo.20826380
