Detection of Sub-resolution Dots in Microscopy Images Karel Štěpka, 2011 Centre for Biomedical Image Analysis, FI MU supervisor: doc. RNDr. Michal Kozubek, Ph.D. Outline nFluorescence microscopy nImage degradations nEvaluation of analysis nExisting approaches to dot detection nFurther Work Fluorescence Microscopy n Fluorescence Microscope nLight source nExcitation filter ¨Allows only the excitation part of the spectrum to pass through nSample ¨Absorbs incoming light ¨Emits light with a lower frequency (fluorescence) nEmission filter ¨Allows only the emission part of the spectrum to pass through nSensor Fluorescence In-Situ Hybridization nAllows to stain individual chromosomes or their parts nProbes appear as small dots in the result FISH_(technique) Image courtesy of Wikimedia Commons Observable Parts of a Cell nCytoplasm nCytoskeleton nNucleus nWhole chromosomes ¨Conditions related to the number of chromosomes (e.g. Down syndrome) nTelomeres nKinetochores nIndividual genes ¨Translocations (e.g. BCL/ABR genes and their relation to certain kinds of leukemia) Observable Parts of a Cell – Dots nCytoplasm nCytoskeleton nNucleus nWhole chromosomes ¨Conditions related to the number of chromosomes (e.g. Down syndrome) nTelomeres nKinetochores nIndividual genes ¨Translocations (e.g. BCL/ABR genes and their relation to certain kinds of leukemia) Fluorescence Dots nReal size on the order of 10 nm nIn the resulting image, often 1 pixel > 60 nm nBecause of the diffraction limit of visible light, the magnification cannot be easily improved n nDue to image degradations, the sensor detects a blurred image of the dot nImage of a dot has a few pixels across dots Image Degradations n Types of Image Degradation nNoise ¨Many kinds, with different causes and statistical distributions: nPhoton shot noise (Poisson) nImpulse noise (often fixed pattern) nReadout noise (Gaussian) nDark current noise nLaser speckle noise ¨Can be suppressed using various methods nDark frame subtraction nGaussian blurring nNon-linear filters (median, non-linear diffusion) Types of Image Degradation nDegradation by point spread function (PSF) ¨Every optical system has a characteristic PSF ¨Describes scattering of photons travelling through individual components of the system ¨Even in an ideal optical system, a point light source produces signal equivalent to the Airy disk ¨ ¨ ¨ ¨PSF can be experimentally measured ¨Degradation can be suppressed using deconvolution 500px-Airy-pattern Types of Image Degradation nChromatic aberration ¨Different wavelengths have different refractive index nField curvature ¨Sensor is planar, but the focal area is curved nSpherical aberration ¨Related to the shape of the lens nDegradations related to sensor technology ¨Smear in CCD chips Evaluation of Analysis n Measures to Consider nDetection ¨ ¨precision = ¨ ¨ ¨recall = ¨ ¨ ¨F1 score = n n nDistinguishing between large dots and double-dots ¨To identify chromosomal conditions such as polysomy present not present found TP FP not found FN TN 2 · precision · recall precision + recall TP TP + FP TP TP + FN Measures to Consider nLocalization ¨Absolute position nTo determine the number of dots inside/outside the nucleus ¨Relative position of individual signals nTo identify chromosomal translocations ¨Mean squared error n nOverall intensity ¨To determine the amount of fluorescent dye or protein ¨Mean squared error Evaluation of Analysis nComparison of the results with the ground truth (GT) ¨We can obtain GT by manually annotating real images ¨We can generate synthetic (simulated) images together with their GT nReal testing data, manual GT ¨Different people, or the same person over multiple attempts, generally annotate images differently ¨Time consuming, expensive nSynthetic testing data, generated GT ¨GT is accurate and undebatable (created before the images) ¨The synthetic data must correspond to the real images Existing Approaches to Dot Detection n “Classical” Detection Methods nThresholding ¨Fixed ¨Otsu ¨Unimodal ¨Adaptive nMathematical morphology ¨Top-hat transform Recent “Classical-Based” Methods nEMax ¨Extended maxima transform, size-based filtering n nGué ¨Top-hat, thresholding, region growing, morphological closing and opening n nHDome ¨HDome transformation, mean shift clustering, cluster filtering Recent “Classical-Based” Methods nKozubek ¨Gradual thresholding, size-based filtering n nNetten ¨Top-hat, dot label (“sweep” through all intensity levels) n nRaimondo ¨Top-hat, modified unimodal thresholding, pattern matching (using a model of a dot) Machine Learning Approach nExamine all potential dot locations and classify them as positive/negative ¨Usually using a sliding sub-window nTraining is required, overtraining is undesirable ¨Training set contains image patches from which the classifier learns nPositive examples nNegative examples ¨Test set is used to determine the quality of the classifier ¨Ideally, training_set ∩ test_set = ∅ ¨We train on the training set, until the results on the test set are satisfactory Machine Learning Approach nNeural networks ¨Multilayer perceptron ¨Each input neuron corresponds to one pixel n nAdaBoost ¨Haar-like features used for weak classifiers ¨Combines several weak classifiers into one strong ¨Computationally intensive in 3D n nFischer discriminant analysis ¨Computationally intensive in 3D Recent Survey by I. Smal et al. nCompared performance of several methods (including machine learning) n2D data ¨Real images ¨Simplified synthetic images nDots represented by Gaussian profiles nDid not evaluate the influence of method parameters nGood starting point Ihor Smal et al.: Quantitative Comparison of Spot Detection Methods in Fluorescence Microscopy. IEEE Transactions on Medical Imaging 29(2): 282–301 (2010) Parametrization – No Size Fits All nNo method can be used on all types of images without any adjustments nOn the data/pixel level, images can be very different, even when displaying the same class of objects ¨Noise level ¨Base intensity ¨Dynamic range ¨Contrast ¨Background (non-)uniformity ¨Illumination artifacts ¨Amount of objects of interest Parametrization – Usability nUsability of a method depends on: ¨Number of its parameters ¨Sensitivity to parameter changes ¨Intuitiveness of its parameters for the end user n nA thorough parametric study is required nCurse of dimensionality ¨Some of the methods have 4–6 free parameters Further Work n Further Work nPrepare a set of benchmark data ¨Cover testing of all important measurements nDetection, localization, intensity ¨Possibly make the set publicly available through CBIA web-site nPerform a thorough evaluation of existing methods ¨Test the methods on various images nReal, manually annotated data nSimulated data with known GT ¨Investigate their behavior when used on 3D data ¨Parametric study ¨Publish the results Further Work nIntermediate results performance_for_different_parameters_of_a_single_method performance_for_different_contrast performance_for_different_parameter_combinations_of_all_methods Further Work nInvestigate the conceptual difference between 2D and 3D fluorescence images ¨Dots do not lie in the same focal plane ¨2D images are usually obtained via max. intensity projection ¨Microscopy images exhibit strong anisotropy ¨Per-slice processing or direct extension to 3D do not take any of this into account nDesign a method natively working with 3D images ¨Most of the existing methods are natively 2D (or nD), and use no special approach for 3D data ¨Investigate localization using model fitting ¨Include the new method in the comparison