Lecture 7 Analysis of electron micrographs 8th November 2022 Jiri Novacek Content - interaction of electrons with matter, radiation damage - data acquisition, image filtering - projection theorem - image averaging in 2D - principal component analysis Interaction of electrons with specimen Williams et al., TEM, Springer Interaction of electrons with specimen Williams et al., TEM, Springer cryo-TEM Peet et al., (2019) Ultramicroscopy mean free path - mean free path of inelastic scattering in vitrified biological specimens: ~395nm Radiation damage Glaser R. (2016), Meth. Enzym. Radiation damage Glaser R. (2016), Meth. Enzym. Glaser R. (2016), Meth. Enzym. Radiation damage Glaser R. (2016), Meth. Enzym. Grant. (2015), eLife Interaction of electrons with specimen Peet et al., (2019) Ultramicroscopy Data acquisition - data fom each position on the sample stored as a short movie - compensation of sample radiation damage - compensation of the sample motion during exposure Data acquisition - data fom each position on the sample stored as a short movie - compensation of sample radiation damage - compensation of the sample motion during exposure - beam induced motion (sample geometry, local) - drift, vibration (external sources, global) x[i] shift y[i]shift Data acquisition - averaging of the movie into single image – increase S/N - compensation for the global and local motion between the frames – minimize image blur, maximize high-res. Info - dose-weighting – frame filtering based on acquired radiation damage global motion additional local motion Data acquisition - averaging of the movie into single image – increase S/N - compensation for the global and local motion between the frames – minimize image blur, maximize high-res. Info - dose-weighting – frame filtering based on acquired radiation damage aligned image unaligned image aligned image unaligned image Data acquisition - averaging of the movie into single image – increase S/N - compensation for the global and local motion between the frames – minimize image blur, maximize high-res. Info - dose-weighting – frame filtering based on acquired radiation damage - application of adaptive per-frame low pass filter before averaging tim e Image filtering unfiltered image lowpass filtered (50A) lowpass filtered (250A) 130A Image filtering unfiltered image lowpass filtered (50A) bandpass filtered (1000,10A) 130A Image formation Image formation Contrast transfer function A – amplitude contrast s – spatial frequency Cs – spherical abberation λ – electron wavelength z – defocus Contrast transfer function - Finite source size - Energy spread (defocus) - MTF of the camera - Generic envelope (drift, charging, multiple scattering) Envelope function Contrast transfer function Envelope function kV=300,ac=0.07,cs=2.7,z=-1,apix=1,B=30 kV=300,ac=0.07,cs=2.7,z=-1,apix=1,B=300 Contrast transfer function Low defocus High defocus Projection theorem John O’Brien (1991). The New Yorker Projection theorem The 2D Fourier transform of the projection of a 3D density is a central section of the 3D Fourier transform of the density, perpendicular to the direction of projection. Particles (regions of interest) n=1 Particles (regions of interest) n=1 n=2 n=8 n=16 n=64 n=256 Signal to noise ratio increases with square-root of n Image alignment in 2D Sum of unaligned particles Sum of aligned particles Image alignment in 2D In order to align the particles in 2D, we need to determine three parameters: - two translational - one rotational (on of the Euler angles) δx δy ψ ψ δy δx ψ δx Cross correlation - measure of similarity of two data series as a function of displacement of these functions - in 2D optimal overlay of two images - normalized cross-correlation – ccc = <-1,1> Cross correlation function in 1D Cross correlation function in 1D Cross correlation function in 1D Cross correlation function in 1D Cross correlation function in 1D Cross correlation function in 1D Cross correlation function in 1D Cross correlation function in 1D Cross correlation function in 1D Cross correlation function in 1D Cross correlation function in 2D Cross correlation function in 2D ConvolutionCross-correlation Image alignment in 2D Cross correlation Cross correlation Cross correlation Cross correlation function in 2D Convolution FT(F I) = FT(F) . FT(I) FT(F I) = FT(F)* . FT(I) Convolution theorem Cross correlation function Image rotation - the images contain not only shift but also rotation - cross-correlation - image sliding over the template (shift) - (log)-polar transform → image transformation from cartesian to polar coordinates → rotational problem shifted to translational problem → utilization of similar approaches as for image shift determination x y R ψ R ψ Orientation alignment We take a series of rings from each image, unravel them, and compute a series of 1D cross-correlation functions. Shifts along these unraveled CCFs is equivalent to a rotation in Cartesian space. Orientation alignment Orientation alignment Orientation alignment - after rotation Image alignment in 2D - rotation and translation are interdependent – (rot→trans) ≠ (trans→rot) => order of the operation matters shift: (25,45), rotation: 60° shift→rotation rotation→shift Image alignment in 2D - rotation and translation are interdependent – (rot→trans) ≠ (trans→rot) - define reasonable range of shifts (e.g. (-2;+2)) and perform rotational alignment for each shifted image Example: for the shift of +/-2 pixels in x and y → 25 alignment rotational alignments → each alignment results in optimal rotational alignment and ccc → compare ccc and select maximal ccc to determine the final shift and translation => increased complexity Interpolation Suppose we shift the image in x & y. The new pixels will be weighted averages of the old pixels. The more the mix the pixels, the worse the result will be. Shift Interpolation Suppose we shift the image in x & y. The new pixels will be weighted averages of the old pixels. The more the mix the pixels, the worse the result will be. Shift Interpolation Suppose we shift the image in x & y. The new pixels will be weighted averages of the old pixels. The more the mix the pixels, the worse the result will be. Shift Rotation Interpolation The Fourier transform of noise is noise - “White” noise is evenly distributed in Fourier space - “White” means that each pixel is independent Interpolation The Fourier transform of noise is noise - “White” noise is evenly distributed in Fourier space - “White” means that each pixel is independent The degradation of the images means that we should minimize the number of interpolations. Classification Sum of unaligned particles Sum of aligned particles - inherently low signal-to-noise - to estimate the sample quality – summarize same projection images to increase signal-to-noise to evaluate data quality Classification Classification methods are divided into those that are “supervised” and those which are “unsupervised”: - Supervised: divide or categorize according to similarity with “template” or “reference” (e.g. projection matching) - Unsupervised: divide according to intrinsic properties (e.g. find classes of projections representing the same view) Classification Supervised – reference based methods - the reference images to which the experimental data are aligned are known - the number or references determines the number of classes (input parameter – potential bias if the number is too low) - assignment quality score – e.g. cross-correlation coefficient reference images experimental data (ROIs/particles) average after alignment each particles – 2D alignment to each reference, determination of shift, rotation, and reference assignment application of the alignment parameter, particle summation Classification unsupervised – reference-free methods - the reference for the image alignment not required - the number of classes/references – required parameter (input parameter – potential bias if the number is too low) - the initial reference are calculated by summation of a random subset of unaligned particles - usually iterate refinement of the class assignment and alignment parameters reference images experimental data (ROIs/particles) average after alignment each particles – 2D alignment to each reference, determination of shift, rotation, and reference assignment application of the alignment parameter, particle summation Classification unsupervised – reference-free methods - the reference for the image alignment not required - the number of classes/references – required parameter (input parameter – potential bias if the number is too low) - the initial reference are calculated by summation of a random subset of unaligned particles - usually iterate refinement of the class assignment and alignment parameters reference images experimental data (ROIs/particles) average after alignment each particles – 2D alignment to each reference, determination of shift, rotation, and reference assignment application of the alignment parameter, particle summation Classification unsupervised – reference-free methods - utilization of different assignment quality score than ccc – e.g. Bayessian approaches – maximum likelyhood estim. - for each orientation and class – calculate probability for a particle – use this probability when calculating particle sum reference images experimental data (ROIs/particles) average after alignment application of the probability-weighted alignment parameter and class assignment to calculate particle sum Calculation of probability of each particle for all orientations and all classes Classification Frank (1996), J.Microscopy Multivariate data analysis (MDA) or multivariate statistical analysis (MSA) - find new coordinate system tailored to the data - find a space with reduced dimensionality for the representation of the objects. This greatly simplifies classification. eigenvectors Multivariate data analysis (MDA) or multivariate statistical analysis (MSA) - find new coordinate system tailored to the data - find a space with reduced dimensionality for the representation of the objects. This greatly simplifies classification. eigenimages Principle component analysis (PCA), Correspondence analysis (CA) Principle component analysis (PCA), Correspondence analysis (CA) eigenimages For a 4096-pixel image, we will have a 4096x4096 covariance matrix. Row-reduction of the covariance matrix gives us “eigenvectors.” - The eigenvectors describe correlated variations in the data. - The eigenvectors have 4096 elements and can beconverted back into images, called “eigenimages.” - The first eigenvectors will account for the most variation. The later eigenvectors may only describe noise. - Linear combinations of these images will give us approximations of the classes that make up the data. Principle component analysis (PCA), Correspondence analysis (CA) Linear combinations of these images will give us approximations of the classes that make up the data.