Three-dimensional Medical Volume Reconstruction from Fluorescent Confocal Laser Scanning Microscopy Imagery

We present a problem of three-dimensional volume reconstruction from fluorescent confocal laser scanning microscopy (CLSM) imagery. We overview a three-dimensional volume reconstruction framework which consists of:

• volume reconstruction procedures using multiple automation levels, feature types, and feature dimensionalities,
• a data-driven registration decision support system,
• an evaluation study of registration accuracy, and
• a novel intensity enhancement technique for 3D CLSM volumes.

The motivation for developing the framework came from the lack of 3D volume reconstruction techniques for CLSM image modality used in cell biology investigations. The 3D volume reconstruction problem is challenging due to significant variations of intensity and shape of cross sectioned structures, unpredictable and inhomogeneous geometrical warping during medical specimen preparation, and an absence of external fiduciary markers.

The objectives of the presented three-dimensional volume reconstruction framework are summarized as follows:

(1) automate alignment of sub-volumes (physical sections) from multiple cross sections,
(2) obtain high resolution image frames by mosaicking (i.e., stitching together),
(3) quantify the accuracy of volume reconstruction using multiple techniques, and
(4) visualize the reconstructed volumes in three-dimensional environments for visual inspection and quantitative interpretation.

In our work, the three-dimensional sub-volume registration problem is viewed primarily as an alignment problem. It is approached by extracting two- or three-dimensional features from each sub-volume and registering the sub-volumes based on the analysis of detected features. We present three sets of techniques classified as pre-processing, main-processing, and post-processing techniques for 3D volume reconstruction.

First, the pre-processing steps include:

• sub-volume intensity analysis for image frame selection and feature detection,
• tile mosaicking using different automation levels and user expertise followed by accuracy evaluation,
• 2D region or 3D volume segmentation using disk/sphere-based region/volume growing technique, and
• feature detection based on 2D or 3D segmentation for accurate feature matching and registration alignment optimization.

Second, the main-processing steps aim at achieving the most accurate sub-volume alignment, and include:

• feature matching (feature correspondence) using different levels of automation and collaborative mechanisms with web services followed by accuracy evaluations,
• registration refinement based on different registration accuracy evaluation criteria,
• optimal global transformation estimation, and (d) sub-volume transformation to construct a 3D volume for visualization.

Finally, the volume post-processing step enhances visual saliency of the reconstructed 3D volume by minimizing distortions of the local image intensities (e.g., gradients of edges), and provides comparative results for enhancement with the existing methods using several image quality assessment metrics.

The primary contribution of this work is the presentation of a new theoretical model for three-dimensional volume reconstruction that includes reconstruction methodology, a data-driven registration decision support, automation, intensity enhancement for processing volumetric image data from fluorescent confocal laser scanning microscopes (CLSM). Researched methods have been fully implemented in the Image to Knowledge (I2K) software package developed at the National Center for Supercomputing Applications (NCSA).

The broader impact of the presented work is in providing the algorithms in a form of web-enabled tools to the medical community so that medical researchers can minimize laborious and time intensive 3D volume reconstructions using the tools and computational resources at NCSA.

Part of the project was to develop tools for registration and alignment of cross section images:

Alignment Tool 1: Transformation Selection (Blood vessels)
Alignment Tool 2: Image Size Selection

• Professor Peter Bajcsy
  Research group ISDA, National Center for Supercomputing Applications, UIUC

• Professor Robert Folberg
  Department of Pathology, College of Medicine, University of Illinois at Chicago (UIC)

• Professor Amy Lin
  Department of Pathology, College of Medicine, University of Illinois at Chicago (UIC)

• Sang-Chul Lee
  ISDA, National Center for Supercomputing Applications, UIUC

• Rob Kooper
  ISDA, National Center for Supercomputing Applications, UIUC

• Andrew Shirk
  ISDA, National Center for Supercomputing Applications, UIUC

• David Clutter
  ISDA, National Center for Supercomputing Applications, UIUC

Publications and Presentations:

Publications

Comparing Vasculogenic Mimicry with Endothelial Cell Lined Vessels: Techniques for 3D Reconstruction and Quantitative Analysis of Tissue Components from Archival Paraffin Blocks.
Amy Y. Lin, Zhuming Ai, Sang-Chul Lee, Peter Bajcsy, Jacob Pe'er, Lu Leach, Andrew J. Maniotis and Robert Folberg
Applied Immunohistochemistry and Molecular Morphology 15, 113-119 (2007)
[pdf (287kB)]  [Publisher]

3D Volume Reconstruction of Extracellular Matrix Proteins in Uveal Melanoma from Fluorescent Confocal Laser Scanning Microscope Images.
Peter Bajcsy, Sang-Chul Lee, Amy Y. Lin and Robert Folberg,
Journal of Microscopy 221, 30-45 (2006)
[Abstract]  [PubMed] [Publisher]

Accuracy Evaluation for Region Centroid-Based Registration of Fluorescent CLSM Imagery.
Sang-Chul Lee, Peter Bajcsy, Amy Y. Lin and Robert Folberg
Journal on Applied Signal Processing, Special Issue on Performance Evaluation in Image Processing, Volume 2006, Article ID 82480, Pages 1-11 (2006)

Presentations

Medical Volume Reconstruction Using Web Services.
Rob Kooper, Andrew Shirk, Sang-Chul Lee, Amy Lin, Robert Folberg and Peter Bajcsy
in Proceedings of IEEE International Conference on Web Services (ICWS 05), Orlando, 2005
[abstract] [pdf]

Acknowledgments

This material is based upon work partially supported by the National Institute of Health under Grant No. R01 EY10457. We also acknowledge NCSA/UIUC support of this work.