Joakim Lindblad
Centre for Image Analysis, Department of Information Technology, Uppsala University, Uppsala, Sweden
joakim.lindblad [at] it.uu.se
Abstract
Multimodal imaging refers to the capturing of complementary information about a specimen by different imaging techniques (modalities). Such complementary information allows reaching deeper understanding and improved analysis and diagnostics performance. Multimodal imaging combined with correlated analysis of the acquired data can be very useful for both human and AI-based decision making. For successful correlation and fusion of the heterogeneous information, acquired images need to be accurately aligned – a task which is far from easy, given the great diversity of imaging modalities and specimens, combined with the typically very large size of medical and biomedical images.
In this work we present a computationally efficient method that reaches a state-of-art performance for multimodal image registration. The method is based on computing the cross-mutual information function (CMIF) through efficient evaluation of mutual information in the Fourier domain for every possible discrete translation. Utilizing the power of GPU-based processing and performing a search over a limited set of rotation angles, the approach facilitates accurate rigid alignment at high speed.
We demonstrate how this approach can be used for improved deep learning-driven oral cancer detection by practically enabling information fusion from different imaging modalities on whole slide image data, overcoming problems originating from stitching artefacts and microscope drift.
Keywords: mutual information, image alignment, correlated imaging, whole slide imaging, cytology
Acknowledgement: We are grateful for the scientific support of J. Öfverstedt and N. Sladoje. The work was financially supported by the Swedish research council (grants 2017-04385 and 2022-03580), Sweden’s Innovation Agency (VINNOVA) (grants 2017-02447, 2020-03611, 2021-01420), and Cancerfonden (grants 22 2353 Pj and 22 2357 Pj).