Transform-based backprojection for volume reconstruction of large format electron microscope tilt series

https://doi.org/10.1016/j.jsb.2005.12.012Get rights and content

Abstract

Alignment of the individual images of a tilt series is a critical step in obtaining high-quality electron microscope reconstructions. We report on general methods for producing good alignments, and utilizing the alignment data in subsequent reconstruction steps. Our alignment techniques utilize bundle adjustment. Bundle adjustment is the simultaneous calculation of the position of distinguished markers in the object space and the transforms of these markers to their positions in the observed images, along the bundle of particle trajectories along which the object is projected to each EM image. Bundle adjustment techniques are general enough to encompass the computation of linear, projective or nonlinear transforms for backprojection, and can compensate for curvilinear trajectories through the object, sample warping, and optical aberration. We will also report on new reconstruction codes and describe our results using these codes.

Section snippets

Motivation for this study

In recent years, the format of digital image detectors for the electron microscope has been progressively increasing. These resulting high-resolution large format images are playing an important role in determining the three-dimensional (3D) structure and function of cells and sub-cellular organelles at the scale of proteins and protein complexes (Martone et al., 2002a, Martone et al., 2002b). Many of these cellular structures can extend for tens to hundreds of microns. Large format digital

Image formation in an electron microscope

To understand how the distortions affecting tilt-series images arise, we must first consider image formation in a uniform magnetic field B. An electron moving in a magnetic field can generally be understood to experience a force according toF=qv×B,which acts perpendicular to the plane defined by the velocity vector and the magnetic field lines, where q is the charge and v is the instantaneous velocity. Electrons moving in a magnetic field will generally orbit the magnetic field lines, moving

Objectives of the study

Given the problems outlined above, the task of tomographic reconstruction is somewhat akin to the job of accurately mapping the organs of a jellyfish swimming at a distance in the ocean and photographed through the bottom of a soft drink bottle. This is, of course, somewhat of an exaggeration, but given the requirement for high precision in our reconstructions, the simile is apt. To continue with the analogy, the various movements of the animal, the bending of the light rays in the seawater,

Tomographic reconstruction

After image acquisition, the reconstruction process is divided into three phases. The first phase is to identify the precise location of a set of image features consistent across the image series. The second phase of the reconstruction is to develop geometric correspondences between the configurations of features in the images, and if necessary to transform the images geometrically to bring the features into better alignment. This is the alignment process, and can be done automatically once

Development of projection models and application to the alignment problem

To align a series of exposures, one must define a suitable model for the optics. The general projective model, which models the optics of straight-line trajectories, and is a generalization of pinhole camera models, is given by the following formula:λxy=P[G|t]XYZ,where P is a projection, G is a nonsingular linear transform, t is a translation, and λ is a scaling function on the (X, Y, Z) coordinates. In this case, the projection map gives the destination in the imaging plane of a trajectory γ,

Development environment

Because of the depth of capabilities of the IMOD software system our system has been developed, to a large degree, as an extension of the IMOD packages. Wherever possible we have used the same file formats as used by IMOD. In particular image data files, fiducial data files, xyz coordinate models, and re-projection error data are the same as those in IMOD. This allows us to use the 3dmod program and its plugin modules to view data and reconstructions, refine fiducial models, and use the bead

Alignment

Re-projection error is a common measure of alignment accuracy, and gives a good indication of the quality of the subsequent reconstruction (Baldwin et al., 2005, Brandt et al., 2001a, Brandt et al., 2001b, Penczek et al., 1995). To assess the effectiveness of our alignment models, we compared the mean square error produced by the projective, quadratic, and cubic models. Initial tracking was performed on a seed model, in which a number of gold particles were located in the zero-tilt image of the

Bundle adjustment

The conjugate gradient method for computing the XYZ model along with the projective transforms has proved to be quite stable, with rapid convergence to a local minimum. We have observed problems only when the fiducial coordinates used as input to the process contain gross errors such as duplicate or missing points. Use of the projective model, on the other hand, carries some well-known ambiguities, because the bundle adjustment is well defined only up to a projective map. We have some control

Discussion and conclusions

An objective of this work has been the development of a method, which could adapt to the complexity of the electron microscope as an optical instrument without having to untangle each separate phenomenon and estimate its effect on the process of image formation. The model of image formation as a geometrically nonlinear integral transform from a three-dimensional object to a two-dimensional image has proved to be a simple, yet robust vehicle for the inclusion of numerous optical effects. In

Acknowledgments

The authors thank Professor Jose Maria Carazo of Autonoma University, Madrid, for his encouragement of this work and his careful reading and criticism of early drafts of the manuscript. The authors further thank Drs. Kyoshi Hama, Tatsuo Arii, and David Mastronarde for helping to inspire this work. We are especially grateful to Dr. Mastronarde for his detailed explanations of the modules of the IMOD package that were foundational to this effort. We also express our appreciation to Ben Kopek and

References (64)

  • S. Kujawa et al.

    Performance of a low-noise CCD camera adapted to a transmission electron microscope

    Ultramicroscopy

    (1992)
  • M.E. Martone et al.

    A cell-centered database for electron tomographic data

    J. Struct. Biol.

    (2002)
  • D.N. Mastronarde

    Dual-axis tomography: an approach with alignment methods that preserve resolution

    J. Struct. Biol.

    (1997)
  • P. Penczek et al.

    Double-tilt electron tomography

    Ultramicroscopy

    (1995)
  • G.A. Perkins et al.

    Electron tomography of large, multicomponent biological structures

    J. Struct. Biol.

    (1997)
  • P.T.E. Roberts et al.

    A CCD-based image recording system for the CTEM

    Ultramicroscopy

    (1982)
  • G.E. Soto et al.

    Serial section electron tomography: a method for three-dimensional reconstruction of large structures

    Neuroimage

    (1994)
  • A.W. Agar

    Principles and Practice of Electron Microscope Operation

    (1974)
  • R.S. Aikens et al.

    Solid state imagers for microscopy

    Methods Cell Biol.

    (1989)
  • E. Anderson et al.

    LAPACK User’s Guide

    (1999)
  • T. Arii et al.

    Method of extracting three-dimensional information from HVTEM stereo images of biological materials

    J. Electron Microsc. (Tokyo)

    (1987)
  • G. Beylkin

    The inversion problem and applications of the generalized radon transform

    Commun. Pure Appl. Math.

    (1984)
  • E. Boyer et al.

    3D surface reconstruction using occluded contours

    Int. J. Comput. Vision

    (1996)
  • I. Daberkow et al.

    Development and performance of a fast fiber-plate coupled CCD camera at medium energy and image processing system for electron holography

    Ultramicroscopy

    (1996)
  • M. Defrise

    A factorization method for the 3D X-ray transform

    Inverse Problems

    (1995)
  • A.S. Denysuk

    Inversion of the generalized radon transform

    Am. Math. Soc. Transl.

    (1994)
  • L. Ehrenpreis

    The Universality of the Radon Transform

    (2003)
  • G.Y. Fan et al.

    Digital imaging in transmission electron microscopy

    J. Microsc.

    (2000)
  • G.Y. Fan et al.

    Conditions for electron tomographic data acquisition

    J. Electron Microsc. (Tokyo)

    (1995)
  • O. Faugeras et al.

    The Geometry of Multiple Images

    (2001)
  • D.A. Forsyth

    Recognizing algebraic surfaces from their outlines

    Int. J. Comput. Vision

    (1996)
  • J. Frank

    Electron Tomography: Three-dimensional Imaging with the Transmission Electron Microscope

    (1992)
  • Cited by (98)

    • Rational Design of Bioavailable Photosensitizers for Manipulation and Imaging of Biological Systems

      2020, Cell Chemical Biology
      Citation Excerpt :

      The sequence of tilt series followed the multilevel access scheme previously described (Phan et al., 2017), which minimizes errors associated with sample shrinkage evenly across all tilt series. EM tomography reconstruction: The entire set of EMT images were aligned and reconstructed using Transform-based Tracking, Bundle adjustment, and Reconstruction (TxBR) package (Lawrence et al., 2006; Phan et al., 2012, 2017). 3D volume rendering was made with Amira 6.3.

    View all citing articles on Scopus
    View full text