Image registration methods for zebrafish gene atlas
1. Image Registration Methods for Reconstructing a Gene Expression Atlas of Early Zebrafish Embryogenesis Department Of Electronic Engineering Technical School Of Telecommunications Engineering Technical University Of Madrid Evangelia Balanou Master Thesis European Postgraduate Program On Biomedical Engineering University of Patras – National Technical University of Athens
2. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Outline Introduction Motivation Problem Goal Image Registration Components Design and Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results and Evaluation Comparison of Registration Methods Atlas Construction Conclusions and Future Work
11. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Problem Quantitative spatio-temporal data at cellular level about gene expression required Provided by Fluorescence In Situ Hybridization techniques and Laser Scanning Microscopy Second gene expression pattern x One gene expression pattern y z
12.
13. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Goal Design and implement an image processing framework able to register different datasets with different gene expression patterns to a common template at a given developmental stage “Registration is the process of determining a geometrical transformation that aligns points in one view of an object with corresponding points in another view of that object or another object.” Template One dataset Template + registered image
15. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Image Registration Intensity-based : Calculates the transformation using voxel values alone Input: 2 images – fixed, moving Output: geometrical transformation Optimization problem Decomposed into a set of basic elements (defining different methods) Registration Similarity measure Fixedimage TransformationParameters Movingimage Interpolation Optimization Resampling Registeredimage Transformation Initial Parameters
16. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Transformation Registration Similarity measure Fixedimage TransformationParameters Movingimage Interpolation Optimization Resampling Registeredimage Transformation Initial Parameters Defines the type of parameters whose values align the two images (search space) Spatial mapping of points from the fixed image space to points in the moving image space (inverse mapping)
17. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Interpolation Similarity measure Fixedimage TransformationParameters Movingimage Interpolation Optimization Resampling Registeredimage Transformation Initial Parameters Evaluate moving image intensities at the mapped, non-grid positions
18. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Similarity Measure Similarity measure Fixedimage TransformationParameters Movingimage Interpolation Optimization Resampling Registeredimage Transformation Initial Parameters A measure of “how well” fixed and transformed moving match each other Provides a quantitative criterion to be optimized over the search space (similarity measure function, S(T) ) The desired optimum may be one of the local ones
19.
20. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Optimization Similarity measure Fixedimage TransformationParameters Movingimage Interpolation Optimization Resampling Registeredimage Transformation Parameters Transformation Initial Parameters Most complex component Starting from an initial set of parameters, iteratively searches the optimal solution of the similarity measure function over the parameter space defined by the transformation Stops when stopping criterion is met
21.
22. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Cost Function
23. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Local optimization Start
24. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Local optimization End
25. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Global optimization Start
26. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Global optimization End Capture range of correct optimum (initial parameter range or initialization)
27. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Resampling Similarity measure Fixedimage TransformationParameters Movingimage Interpolation Optimization Resampling Registeredimage Transformation Initial Parameters Once a stopping criterion is met or iteration number has reached, the last transformation parameters are used to produce the registered image
29. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Concept Goal: Design and implement an image processing framework able to register different datasets with different gene expression patterns to a common template at a given developmental stage Partial views Template embryo Nuclei channel Reference gene channel (goosecoid) Another gene channel *All images are 3D and grayscale *Colourmap just for visualization
30. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Concept Partial view of another embryo Partial views Template embryo Nuclei channel Nuclei channel Registration Reference gene channel Reference gene channel Another gene channel *All images are 3D and grayscale *Reference gene (position): goosecoid (gsc) *Colourmap just for visualization Gene expression atlas
32. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Registration Pipeline registration addition initialization Partial embryo view, nuclei channel Initialized Moving image preprocessing Moving image Partial embryo view, gsc channel preprocessing Registered image Rotation centre Transformation Parameters Whole embryo view, nuclei channel addition preprocessing Fixedimage Fixedimage Whole embryo view, gsc channel preprocessing Gravity centres Registration pipeline initialization Partial embryo view, third channel preprocessing transformation Third channel mapped Atlas construction pipeline Purpose: Determine the transformation parameters that bring into spatial alignment the template and one partial view
33. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Preprocessing & Addition Step registration addition initialization Partial embryo view, nuclei channel Initialized Moving image preprocessing Moving image Partial embryo view, gsc channel preprocessing Registered image Rotation centre Transformation Parameters Whole embryo view, nuclei channel addition preprocessing Fixedimage Fixedimage Whole embryo view, gsc channel preprocessing Registration pipeline Purpose: Remove noise, blur, downsample, threshold Combine information from nuclei and gsc channels into a single image
34.
35. Weighted Addition addition preprocessing preprocessing 0 255 Combined image Original channels Preprocessed channels Resolution: 512 x 512 x 465 Voxel size: 1.517 x 1.517 x 1,509μm Resolution: 128 x 128 x 116 Voxel size: 6.068 x 6.068 x 6.036μm
36. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Initialization Step registration addition initialization Partial embryo view, nuclei channel Initialized Moving image preprocessing Moving image Partial embryo view, gsc channel preprocessing Registered image Rotation centre Transformation Parameters Whole embryo view, nuclei channel addition preprocessing Fixedimage Fixedimage Whole embryo view, gsc channel preprocessing Registration pipeline Purpose: Initial positioning of moving to fixed image’s space (no initial parameters in registration) If NOT sufficient overlapping, registration fails
37. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Initialization Step Preprocessed partial embryo view, nuclei channel (binary) Preprocessed partial embryo view, gsc channel initialization Moving image Initialized Moving image Rotation centre Fixedimage Preprocessed whole embryo view, nuclei channel (binary) Preprocessed whole embryo view, gsc channel Based on nature of data (nuclei and gsc channel) For both views one gravity centre from each channel The resulting four points define a spatial transformation that is applied on the moving image
38. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Initialization Step nmoving gscmoving y gscmoving Rotation axis nfixed vF nmoving translation Moving (partial view) Rotation angle gscfixed Translated nmoving vM Translated nmoving nfixed x z gscfixed *Blue/Orange-nuclei Green/Yellow-gsc expression pattern Fixed (template view)
39. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Initialization Step Before initialization Fixed Image Initialized Moving After Initialization Fixed (template) + Initialized Moving (partial) Partial view before and after initialization *Blue/Orange-nuclei Green/Yellow-gsc expression pattern
40. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Registration Step registration addition initialization Partial embryo view, nuclei channel Initialized Moving image preprocessing Moving image Partial embryo view, gsc channel preprocessing Registered image Rotation centre Transformation Parameters Whole embryo view, nuclei channel addition preprocessing Fixedimage Fixedimage Whole embryo view, gsc channel preprocessing Registration pipeline Purpose: Find the transformation parameters that register the initialized moving image to the fixed
41.
42. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Atlas Construction Pipeline registration addition initialization Partial embryo view, nuclei channel Initialized Moving image preprocessing Moving image Partial embryo view, gsc channel preprocessing Registered image Rotation centre Transformation Parameters Whole embryo view, nuclei channel addition preprocessing Fixedimage Fixedimage Whole embryo view, gsc channel preprocessing Gravity centres Registration pipeline initialization Partial embryo view, third channel preprocessing transformation Third channel mapped Atlas construction pipeline Purpose: Transformation of the third channel of the partial view Only transformation step is implemented as a new program
49. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Data Developmental stage: Shield (6 hpf) Framework tested with six datasets (six embryos) One template, one whole embryo view Partial views of five different embryos Animal Dorsal Ventral Vegetal Template embryo Partial view nuclei channel gsc channel co-stained gene expression pattern e.g. snail * Images provided by: DEPSN , France
50. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Questions Does the implemented framework succeed in registering our data? What is the combination of similarity measure and optimization algorithm that results in a successful registration? In other words… What is the most appropriate registration method for our application?
51. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Preprocessing & Addition Original channels Preprocessed channels Combined image preprocessing addition Template preprocessing Slice Volume rendering Fixed image addition preprocessing One partial View preprocessing Slice Volume rendering One moving image Framework works with 2 datasets each time Preprocessing: smoothed, downsampled, nuclei channel turned to binary Addition: nuclei and gsc channels combined into a single image 5 partial -> 5 iterations (6 images in total – 1 fixed, 5 moving)
53. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Method Evaluation Four different methods implemented Evaluation only by visual inspection of the results Optimization algorithms not comparable unless running with optimized parameters Lack of golden standard Point-to-point correspondence does not exist (different embryos)
58. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Summary - Conclusions Goal achieved Designed and implemented an image processing framework able to map different gene expression patterns on a common template (for a given developmental stage) Key points Addition: Combine information from two channels Initialization: Solves the problem of capture range for optimization Registration Method: Correlation Coefficient + Gradient Descent
59. Outline Introduction Motivation Problem Goal ImageRegistration Transformation Interpolation Similarity Measure Optimization Resampling Design & Implementation Concept Overview Registration Pipeline Atlas Construction Pipeline Tools Implementation Results & Evaluation Comparison of Registration Methods Atlas Conclusions & Future Work Conclusions-Future work Advantages Modularity Configurability Semi-automated Future work More datasets -> more gene expression patterns Other developmental stages Validated with known gene regulatory networks
60. Thanks to… Biomedical Image Technologies Laboratory (BIT) Technical School Of Telecommunications Engineering (ETSIT) Technical University of Madrid (UPM)
Tools for studying the spatial characteristics of gene expression FISH + microscopyStained by FISH and imaged by.Imaging: the only way to record spatial location ++structural infoScanning across x+y to acquire optical sliceCan be visualized in projections (slices) or volume rendering! THEY R VOLUMETRIC IMAGES!! to view gene activity=> FISH, attaching a probe to a transcript!!FISH: the assay of choice for localization of specific nucleic acids sequences in native context. Basic principles remain unchanged, now a wide spectrum of detection schemesemission independent of absorption!! Here spectrum with Leica!“capture the relative spatial context of the fluorescently labelled structures”“spatial map of gene expression patterns”Definition of the channel: “ the image data from each fluorescent label” the two or three channels were acquired separately but simultaneously, as the emission spectrum is distinct.The dyes emit light in different parts of the spectrum, so that three separate images of the embryo can be collected with the appropriate color filters (middle)
Tools for studying the spatial characteristics of gene expression FISH + microscopyStained by FISH and imaged by.Imaging: the only way to record spatial location ++structural infoScanning across x+y to acquire optical sliceCan be visualized in projections (slices) or volume rendering! THEY R VOLUMETRIC IMAGES!! to view gene activity=> FISH, attaching a probe to a transcript!!FISH: the assay of choice for localization of specific nucleic acids sequences in native context. Basic principles remain unchanged, now a wide spectrum of detection schemesemission independent of absorption!! Here spectrum with Leica!“capture the relative spatial context of the fluorescently labelled structures”“spatial map of gene expression patterns”Definition of the channel: “ the image data from each fluorescent label” the two or three channels were acquired separately but simultaneously, as the emission spectrum is distinct.The dyes emit light in different parts of the spectrum, so that three separate images of the embryo can be collected with the appropriate color filters (middle)
Image registration is the process of determining the spatial transform that maps points from one image to homologous points in another.
We r talking about the intensity based approach that works directly on the intensity values of the images and does not require any interaction from the user (while the registration is running).The basic input data are two images: one defined as fixed (static) and the other as moving, that will be spatially map to align with the first.Treated as an optimization problem with the goal of finding the spatial mapping that will bring the MOVING into alignment with the FIXED.
We would expect the transformation parameters to map points from the moving to fixed. However this transformation could result in holes or overlaps. Therefore the transformation is done backwards.Inverse mapping= avoid holes, overlaps
The intensities on the transformed grid are taken by interpolating values in the moving.
After the transformation, the images are run thru a similarity measureFor large transformations (total misalignment) only background noise overlaps!The optimum of this function is assumed to correspond to the transformation that successfully registers the images
Based on the Information Theory, that says that the amount of information they contain about each other is maximal.H entropies information they contain about themselves, joint entropy measures the dispersion of the joint probability distribution. The more they match, the clearer the clusters that can be seen on the joint histogram.
The result of the similarity measure is given to ..Goal: is the component that drives the registration. It explores the parameter space of the T in search of a set of values that optimize the similarity measure function!!!!! This is an iterative procedure until reaches…Now a question here is whether we are looking for global or local of the similarity measure function. For intensity-based registration measures, it is possible that a large misregistration of two images results in a better value of the measure than the correct transformation. The desired optimum may not be the global one of the search space and only part of the search space leads to the desired optimum.
GD: Advances parameters in the direction of the gradient where the step size is governed by a learning rate (λ)DE: It is an evolutionary algorithm. Starting from a population vector with size NP, it generates a mutant vector from the existing elements (f is just a weighting factor). To increase diversity of the population, crossing-over with a probability of CR is introduced to construct a trial vector. This trial vector is compared to the population vector and the elements that yield the best similarity measure values are passed on to the next generation (greedy criterion).(Gradient: we can choose the directionPopulation vector initial parameter values randomly from IPR, parameters for next generation vector selected according to the greedy criterion)One difference between them is that GD requires derivative of S whereas DE no!! Another is local, global
That is function to be minimized…
Stochastic, population-based
Stochastic, population-based
Stochastic, population-based
Start the algorithm within the capture range of the desired optimum!Starting estimate of transformation close to the correct solution (initialization)
The output is the transformation. That is given to ..Last step of the registration is to use the resulting transformation to map the moving image onto the fixed image SPACE!!Takes moving and parameters and produces the registered transformed image.
Present yourself
Because cellular resolution was required for this dataset, they cover limited views, restricted to the stained region.
The pipeline that prepares the data before the actual registration, the pipeline that leads all the way to registration
scalar parameter A balances the weight in the registrationprocess of nuclei structural information and gene expression details
the better the images overlap before the registration step, the less displacement theregistration algorithm has to cover and more chances to obtain a useful alignment.
We have already seen the components it is composed of. Here we can see what has been selected for each component.2 and 2 so we can compare their performance.
Command line programsThe parameters under which each program is run can be configured by the user!!!!
Present yourself
“the volume that has been imaged encompasses part of the blastoderm (embryonic cell mass?) and does not get into the yolk (non cellular mass of nutrients)”DEPSN (Development Evolution Plasticity of the Nervous System), Francedatasets dapi, CY5, FITC, imaged by LSM
Many qs concerning configuration of parameters of programs.
Template always enters the algorithmNext comes the step of initialization
(part of the transformation parameters space that includes the desired optimal value
the combination of the correlation coefficient with the gradient descent algorithm presented a coherent and sufficient performance, when the division factor in the addition step was in the [1–6] rangeAlthough the in situ hybridizations of gene X and Y were conducted independently, their expression patterns can be visualized simultaneously.