Tutorial on Multispectral Imaging in Medicine
Multispectral Imaging in Medicine
Tutorial at MICCAI 2007
29 October 2pm-5pm Brisbane Convention & Exhibition Centre
Co-organisers: Ela Claridge and Iain Styles (University of Birmingham)
Multispectral images of the human retina.
Distribution of retinal blood computed from multispectral image.
Traditional tricolor imaging can provide us with excellent spatial resolution, but has very low spectral resolution, acquiring images in only three broad spectral bands. Much more spectral information can be obtained from point spectroscopy, but this can only obtain information from a small spatial region. Multi-and hyperspectral imaging combines the spatial resolution provided by tricolor imaging with the spectral information available from point spectroscopy. A complete spectrum is recorded at each point in an image, and this can allow us to deduce detailed qualitative and quantitative information about the structure and composition of the tissue, across the whole of the image.
The relatively recent availability of multi-wavelength light sources and programmable optical filters means that images can be acquired in a reasonable timeframe. This means that the use of multi- and hyperspectral imaging in medicine is now a practical reality.
In this tutorial, we will begin by outlining the various ways in which multispectral images can be acquired. We will then present general statistical methods for processing spectroscopic data which will lead into a discussion of how these methods can be used to classify regions of an image from their spectrum, and how to segment multispectral images. Finally, we will discuss how quantitative properties of the tissue itself can be deduced from a multispectral image.
The tutorial is suitable for students and researchers who would like to gain an appreciation of the potential of multispectral imaging, and some insight into the state-of-the-art methods being developed to process the resulting data.
2:00pm - 2:20pm
Introduction and Aims
Colour plays an important role in the clinical diagnosis of many conditions. However, the receptors in the clinician's eye, as well as the sensors in a standard RGB camera, provide only a limited representation of the visible spectrum. Spectral data can yield information beyond what is possible by observation or photography. This talk will demonstrate the shortcomings of the human colour perception and of the RGB-based imaging devices. It will then introduce the basic concepts and terminology associated with light, colour, spectra and multispectral images, to lay foundations for the subsequent topics. Examples of the state-of-the-art clinical applications will provide the motivation for this exciting area of medical imaging.
- Why do we need multispectral imaging?
- Colour perception
- Principles of multispectral imaging
- Key ideas and terminology: light, colour, spectra
- Overview of clinical applications
2:20pm - 2:50pm
Image Acquisition and Preprocessing
Jens Michael Carstensen (Informatics and Mathematical Modelling, Technical University of Denmark/Videometer A/S)
Imaging and machine vision have now for several decades been an obvious choice for the characterization of tissues and organs based on their colour. However, the acquisition and treatment of radiometric measurements using computer vision systems requires effective handling of a number of critical issues that arise from the inherent properties of such systems:
- Pixel values represent a complex combination of many optical effects originating both from the sample (diffuse reflectance, specular reflectance, topography, fluorescence, scattering, etc.) and from the acquisition system (illumination and camera geometry, spectral sensitivity, etc).
- Heterogeneity of tissues and organs makes it difficult to use simplifying assumptions about the sample optical properties such as their spatial or spectral smoothnes.
- The combination of geometry and radiometry in every measurement adds a great deal of complexity, but also offers a huge measurement potential.
An effective way of dealing with these issues is a twofold strategy: to carefully design a system with respect to the task at hand (optimize illumination geometry, focus on reproducibility and traceability of the measurements); and to provide the necessary redundancy in the imaging system to enable meaningful statistical analysis of the image data. Acquiring images at multiple wavelengths is a powerful way of obtaining such an effective redundancy. This presentation will show how this may be done in a way that gives the best starting point for the subsequent analysis and interpretation of multispectral image data.
- Principles of operation of filter, LED and pushbroom systems
- Choice of acquisition parameters: camera, illumination geometry, illuminants, control
- The importance of calibration and registration
- Selection of wavelengths
2:50pm - 3:25pm
Algorithms for Multispectral Image Analysis
AbstractThe wealth of information provided by multispectral images allows for improved understanding of images of tissues and organs. This talk will present a number of representative algorithms that demonstrate the new insights gained by using multispectral data. However, this enhanced awareness does not come without cost. We often reach the point that too much data, often closely correlated or irrelevant to the application at hand, is collected. We will show how this topic has been addressed. Lastly, another important issue that will be discussed is visualization of high dimensional data, inherent in multispectral images, that provides the intuitive understanding of the image contents.
- Enhanced image understanding
- Adaptation of traditional image analysis algorithms
- Novel algorithms for multispectral data (true colour extraction, improved treatment of specular highlights)
- Efficient use of spectrally overdetermined data
- Dimensionality reduction (PCA, LLE, Isomap)
- Spectral band selection (material-specific band design, selection from a set of standard filters)
- Visualization of imperceptible high-dimensional spectral data
- Visualization of the embedded space
- Display of the most prominent spectral bands
- Post-processing presentation of classification results
3:40pm - 4:10pm
Multispectral Image Classification and Segmentation
AbstractIn this tutorial we will present techniques for spectral image clustering and classification. The techniques include computational methods and also examples of optical methods, in which the optimal filters for imaging can be designed and realized optically. Optimally filtered spectral image itself can contain already segmented regions of interest. In the classification, the spectral and spatial domains of the spectral images are taken into account. We will show segmentation results of spectral images.
- Clustering Algorithms
- Optical implementation of optimal filters for classification
4:10pm - 4:45pm
Quantitative Analysis of Multispectral Images
AbstractThe information provided by multispectral imaging can allow us to deduce more more information about the structure and properties of biomedical images than has previously been possible. In this talk, we discuss how to obtain this information, using prior knowledge of the structure and properties of the tissues. In the simplest technique we will show how to use reference spectra derived from images or from libraries to perform "spectral unmixing" on an image, which is especially useful in fluorescence imaging. We will then discuss how to use a detailed prior knowledge of the structure of tissues to construct physical models which can be used to interpret the images in terms of the tissue's constituent parts.
- Key Physical ideas
- Absorption, scattering, reflectance, transmittance
- Spectral Unmixing
- Predictive modelling using biomedical optical models
- Case studies - skin and eye imaging