SlideShare a Scribd company logo
1 of 45
2014 Workshop in 
Metabolomic Data Analysis 
Dmitry Grapov, PhD 
Introduction
Important 
Introduction 
This is an introduction to a series of tutorials for 
metabolomic data analysis 
1. Download all the required files and software at: 
https://sourceforge.net/projects/teachingdemos/files/WCMC%202014%20Summer%20Workshop/ 
2. Make sure you have installed R (v3.1.1, http://cran.us.r-project.org/ ), shiny (v0.10.1, 
http://shiny.rstudio.com/ ) and a modern browser (e.g., Chrome). 
3. Run the code in the folder software the file startup.R to launch all 
accompanying software; or download most current versions at the links 
below 
• DeviumWeb (https://github.com/dgrapov/DeviumWeb) 
• MetaMapR (https://github.com/dgrapov/MetaMapR) 
4. Have a great time!
Goals?
Analysis at the Metabolomic Scale
Statistical and Multivariate Analyses 
Group 1 
Statistics 
+ 
+ 
= 
Multivariate 
Context 
Network Mapping 
Ranked statistically 
significant differences 
within a a biochemical 
context 
Group 2 
What analytes are 
different between the 
two groups of samples? 
Statistical 
t-Test 
significant differences 
lacking rank and 
context 
Multivariate 
O-PLS-DA 
ranked differences 
lacking significance 
and context
Statistical and Multivariate Analyses 
Group 1 
Statistics 
+ 
+ 
= 
Multivariate 
Context 
Network Mapping 
Group 2 
What analytes are 
different between the 
two groups of samples? 
Statistical 
t-Test 
Multivariate 
O-PLS-DA 
To see the big picture it is necessary too view the data from multiple 
different angles
Cycle of Scientific Discovery 
Hypothesis Data Acquisition 
Data Processing 
Hypothesis Generation Data Analysis Data
Sample 
Data Analysis and Visualization 
Variable 
Quality Assessment 
• use replicated mesurements 
and/or internal standards to 
estimate analytical variance 
Statistical and Multivariate 
• use the experimental design 
to test hypotheses and/or 
identify trends in analytes 
Functional 
• use statistical and multivariate 
results to identify impacted 
biochemical domains 
Network 
• integrate statistical and 
multivariate results with the 
experimental design and 
analyte metadata 
Sample Variable 
experimental design 
- organism, sex, age etc. 
analyte description and 
metadata 
- biochemical class, mass 
spectra, etc.
Sample 
Data Analysis and Visualization 
Variable 
Quality Assessment 
• use replicated mesurements 
and/or internal standards to 
estimate analytical variance 
Statistical and Multivariate 
• use the experimental design 
to test hypotheses and/or 
identify trends in analytes 
Functional 
• use statistical and multivariate 
results to identify impacted 
biochemical domains 
Network 
• integrate statistical and 
multivariate results with the 
experimental design and 
analyte metadata 
Network Mapping 
Sample Variable 
experimental design 
- organism, sex, age etc. 
analyte description and 
metadata 
- biochemical class, mass 
spectra, etc.
Data Quality Assessment 
Quality metrics 
•Precision (replicated 
measurements) 
•Accuracy (reference 
samples) 
Common tasks 
•normalization 
•outlier detection 
•missing values 
imputation 
*Finish lab: 1-Data quality
Univariate Qualities 
•length (sample size) 
•center (mean, median, 
geometric mean) 
•dispersion (variance, 
standard deviation) 
•range (min / max), 
•quantiles 
•shape (skewness, kurtosis, 
normality, etc.) 
standard deviation 
mean
Univariate Analyses 
•Identify differences in sample population 
means 
•sensitive to distribution shape 
•parametric = assumes normality 
•error in Y, not in X (Y = mX + error) 
•optimal for long data 
•assumed independence 
•false discovery rate (FDR) 
long 
wide 
n-of-one
False Discovery Rate (FDR) 
Type I Error: False Positives 
•Type II Error: False Negatives 
•Type I risk = 
•1-(1-p.value)m 
m = number of variables tested 
FDR correction 
• p-value adjustment or estimate of FDR (Fdr, q-value) 
Bioinformatics (2008) 24 (12):1461-1462
Statistical Analysis: achieving ‘significance’ 
significance level (α) and power (1-β ) 
effect size (standardized difference in means) 
sample size (n) 
Power analyses can be used to 
optimize future experiments 
given preliminary data 
Example: use experimentally 
derived (or literature estimated) 
effect sizes, desired p-value 
(alpha) and power (beta) to 
calculate the optimal number of 
samples per group 
*finish lab 
2-statistical 
analysis
Outlier Detection 
• 1 variable 
(univariate) 
• 2 variables 
(bivariate) 
• >2 variables 
(multivariate)
bivariate vs. 
multivariate 
(scatter plot) 
outliers? 
mixed up samples 
(PCA scores plot) 
Outlier Detection
Batch Effects 
Drift in >400 replicated measurements across >100 analytical batches for a single analyte 
Principal Component 
Analysis (PCA) of all 
analytes, showing QC 
sample scores 
Acquisition batch 
Abundance 
QCs embedded 
among >5,5000 
samples (1:10) 
collected over 
1.5 yrs 
If the biological effect 
size is less than the 
analytical variance 
then the experiment 
will incorrectly yield 
insignificant results
Analyte specific data quality 
overview 
Sample specific normalization can be used 
to estimate and remove analytical variance 
Raw Data Normalized Data 
low precision 
log mean 
%RSD 
high precision 
Samples 
QCs 
Batch Effects 
*finish lab 3-Data Normalization
Clustering 
Identify 
•patterns 
•group structure 
•relationships 
•Evaluate/refine hypothesis 
•Reduce complexity 
Artist: Chuck Close
Cluster Analysis 
Use the concept similarity/dissimilarity 
to group a collection of samples or 
variables 
Approaches 
•hierarchical (HCA) 
•non-hierarchical (k-NN, k-means) 
•distribution (mixtures models) 
•density (DBSCAN) 
•self organizing maps (SOM) 
Linkage k-means 
Distribution Density
Hierarchical Cluster Analysis 
• similarity/dissimilarity 
defines “nearness” or 
distance 
euclidean 
X 
Y 
manhattan Mahalanobis 
X 
Y 
X 
Y 
* 
non-euclidean
Hierarchical Cluster Analysis 
Agglomerative/linkage algorithm 
defines how points are grouped 
single complete centroid average
Dendrograms 
Similarity 
x 
x 
x 
x
Hierarchical Cluster Analysis 
How does my metadata 
match my data structure? 
Exploration Confirmation 
*finish lab 4-Cluster Analysis
Projection of Data 
The algorithm defines the position of the light source 
Principal Components Analysis (PCA) 
• unsupervised 
• maximize variance (X) 
Partial Least Squares Projection to 
Latent Structures (PLS) 
• supervised 
• maximize covariance (Y ~ X) 
James X. Li, 2009, VisuMap Tech.
Interpreting PCA Results 
Variance explained (eigenvalues) 
Row (sample) scores and column (variable) loadings
How are scores and 
loadings related?
Centering and Scaling 
PMID: 16762068 
*finish lab 5-Principal Components Analysis
Use PLS to test a hypothesis on a 
multivariate level 
Partial Least Squares (PLS) is used to identify maximum modes of 
covariance between X measurements and Y (hypotheses) 
PCA PLS 
time = 0 120 min.
Modeling multifactorial 
relationships 
~two-way ANOVA dynamic changes among groups
PLS Related Objects 
Model 
•dimensions, latent variables (LV) 
•performance metrics (Q2, RMSEP, etc) 
•validation (training/testing, permutation, cross-validation) 
•orthogonal correction 
Samples 
•scores 
•predicted values 
•residuals 
Variables 
•Loadings 
•Coefficients: summary of loadings based on all LVs 
•VIP: variable importance in projection 
•Feature selection
“goodness” of the model is all about the 
perspective 
Determine in-sample (Q2) and out-of- 
sample error (RMSEP) and 
compare to a random model 
•permutation tests 
•training/testing 
*finish lab 6-Partial Least Squares and lab 7-Data Analysis Case Study
Functional Analysis 
Identify changes or enrichment in biochemical domains 
• decrease 
• increase 
Nucl. Acids Res. (2008) 36 (suppl 2): W423-W426.doi: 10.1093/nar/gkn282
Functional Analysis: Enrichment 
Biochemical Pathway Biochemical Ontology 
*finish lab 8-Metabolite Enrichment Analysis
Connections and 
Contexts 
Biochemical (substrate/product) 
•Database lookup 
•Web query 
Chemical (structural or 
spectral similarity ) 
•fingerprint generation 
BMC Bioinformatics 2012, 13:99 doi:10.1186/1471-2105-13-99 
Empirical (dependency) 
•correlation, partial-correlation
Network Mapping 
2. Calculate 
Mappings 
1. Calculate 
Connections 
3. Create 
Mapped Network 
Grapov D., Fiehn O., Multivariate and network tools for analysis and visualization of metabolomic data, ASMS, June 08, 2013, Minneapolis, MN
Mapping Analysis Results to Networks 
Analysis results Network Annotation Mapped Network 
*finish lab 9-Network Mapping I
Biochemical 
Relationships 
http://www.genome.jp/dbget-bin/www_bget?rn:R00975
Structural 
Similarity 
http://pubchem.ncbi.nlm.nih.gov//score_matrix/score_matrix.cgi
Correlation networks 
•simple to calculate 
Complex lipids correlation network in mouse serum
Correlation Networks 
•can be difficult to interpret 
•poorly discriminate between direct and indirect 
associations 
Complex lipids correlation network in mouse heart tissue
Partial correlations can 
help simplify networks 
and preference direct 
over indirect 
associations. 
10.1007/978-1-4614-1689-0_17 
Complex lipids partial correlation 
network in human plasma
Mass Spectral Connections 
Watrous J et al. PNAS 2012;109:E1743-E1752 *finish lab 10-Network Mapping II
Software and Resources 
•DeviumWeb- Dynamic multivariate data analysis and 
visualization platform 
url: https://github.com/dgrapov/DeviumWeb 
•imDEV- Microsoft Excel add-in for multivariate analysis 
url: http://sourceforge.net/projects/imdev/ 
•MetaMapR- Network analysis tools for metabolomics 
url: https://github.com/dgrapov/MetaMapR 
•TeachingDemos- Tutorials and demonstrations 
•url: http://sourceforge.net/projects/teachingdemos/?source=directory 
•url: https://github.com/dgrapov/TeachingDemos 
•Data analysis case studies and Examples 
url: http://imdevsoftware.wordpress.com/
Questions? 
dgrapov@ucdavis.edu 
This research was supported in part by NIH 1 U24 DK097154

More Related Content

What's hot

Normalization of Large-Scale Metabolomic Studies 2014
Normalization of Large-Scale Metabolomic Studies 2014Normalization of Large-Scale Metabolomic Studies 2014
Normalization of Large-Scale Metabolomic Studies 2014Dmitry Grapov
 
Data Normalization Approaches for Large-scale Biological Studies
Data Normalization Approaches for Large-scale Biological StudiesData Normalization Approaches for Large-scale Biological Studies
Data Normalization Approaches for Large-scale Biological StudiesDmitry Grapov
 
4 partial least squares modeling
4  partial least squares modeling4  partial least squares modeling
4 partial least squares modelingDmitry Grapov
 
1 statistical analysis
1  statistical analysis1  statistical analysis
1 statistical analysisDmitry Grapov
 
Mapping to the Metabolomic Manifold
Mapping to the Metabolomic ManifoldMapping to the Metabolomic Manifold
Mapping to the Metabolomic ManifoldDmitry Grapov
 
Data analysis workflows part 2 2015
Data analysis workflows part 2 2015Data analysis workflows part 2 2015
Data analysis workflows part 2 2015Dmitry Grapov
 
Automation of (Biological) Data Analysis and Report Generation
Automation of (Biological) Data Analysis and Report GenerationAutomation of (Biological) Data Analysis and Report Generation
Automation of (Biological) Data Analysis and Report GenerationDmitry Grapov
 
Case Study: Overview of Metabolomic Data Normalization Strategies
Case Study: Overview of Metabolomic Data Normalization StrategiesCase Study: Overview of Metabolomic Data Normalization Strategies
Case Study: Overview of Metabolomic Data Normalization StrategiesDmitry Grapov
 
Multivariate data analysis and visualization tools for biological data
Multivariate data analysis and visualization tools for biological dataMultivariate data analysis and visualization tools for biological data
Multivariate data analysis and visualization tools for biological dataDmitry Grapov
 
Metabolomic data analysis and visualization tools
Metabolomic data analysis and visualization toolsMetabolomic data analysis and visualization tools
Metabolomic data analysis and visualization toolsDmitry Grapov
 
3 data normalization (2014 lab tutorial)
3  data normalization (2014 lab tutorial)3  data normalization (2014 lab tutorial)
3 data normalization (2014 lab tutorial)Dmitry Grapov
 
Data analysis workflows part 1 2015
Data analysis workflows part 1 2015Data analysis workflows part 1 2015
Data analysis workflows part 1 2015Dmitry Grapov
 
Omic Data Integration Strategies
Omic Data Integration StrategiesOmic Data Integration Strategies
Omic Data Integration StrategiesDmitry Grapov
 
5 data analysis case study
5  data analysis case study5  data analysis case study
5 data analysis case studyDmitry Grapov
 
6 metabolite enrichment analysis
6  metabolite enrichment analysis6  metabolite enrichment analysis
6 metabolite enrichment analysisDmitry Grapov
 
The International Journal of Engineering and Science (The IJES)
The International Journal of Engineering and Science (The IJES)The International Journal of Engineering and Science (The IJES)
The International Journal of Engineering and Science (The IJES)theijes
 
Prote-OMIC Data Analysis and Visualization
Prote-OMIC Data Analysis and VisualizationProte-OMIC Data Analysis and Visualization
Prote-OMIC Data Analysis and VisualizationDmitry Grapov
 
Some statistical concepts relevant to proteomics data analysis
Some statistical concepts relevant to proteomics data analysisSome statistical concepts relevant to proteomics data analysis
Some statistical concepts relevant to proteomics data analysisUC Davis
 

What's hot (20)

Normalization of Large-Scale Metabolomic Studies 2014
Normalization of Large-Scale Metabolomic Studies 2014Normalization of Large-Scale Metabolomic Studies 2014
Normalization of Large-Scale Metabolomic Studies 2014
 
7 network mapping i
7  network mapping i7  network mapping i
7 network mapping i
 
Data Normalization Approaches for Large-scale Biological Studies
Data Normalization Approaches for Large-scale Biological StudiesData Normalization Approaches for Large-scale Biological Studies
Data Normalization Approaches for Large-scale Biological Studies
 
4 partial least squares modeling
4  partial least squares modeling4  partial least squares modeling
4 partial least squares modeling
 
1 statistical analysis
1  statistical analysis1  statistical analysis
1 statistical analysis
 
Mapping to the Metabolomic Manifold
Mapping to the Metabolomic ManifoldMapping to the Metabolomic Manifold
Mapping to the Metabolomic Manifold
 
Data analysis workflows part 2 2015
Data analysis workflows part 2 2015Data analysis workflows part 2 2015
Data analysis workflows part 2 2015
 
Automation of (Biological) Data Analysis and Report Generation
Automation of (Biological) Data Analysis and Report GenerationAutomation of (Biological) Data Analysis and Report Generation
Automation of (Biological) Data Analysis and Report Generation
 
Case Study: Overview of Metabolomic Data Normalization Strategies
Case Study: Overview of Metabolomic Data Normalization StrategiesCase Study: Overview of Metabolomic Data Normalization Strategies
Case Study: Overview of Metabolomic Data Normalization Strategies
 
Multivariate data analysis and visualization tools for biological data
Multivariate data analysis and visualization tools for biological dataMultivariate data analysis and visualization tools for biological data
Multivariate data analysis and visualization tools for biological data
 
Metabolomic data analysis and visualization tools
Metabolomic data analysis and visualization toolsMetabolomic data analysis and visualization tools
Metabolomic data analysis and visualization tools
 
3 data normalization (2014 lab tutorial)
3  data normalization (2014 lab tutorial)3  data normalization (2014 lab tutorial)
3 data normalization (2014 lab tutorial)
 
2 cluster analysis
2  cluster analysis2  cluster analysis
2 cluster analysis
 
Data analysis workflows part 1 2015
Data analysis workflows part 1 2015Data analysis workflows part 1 2015
Data analysis workflows part 1 2015
 
Omic Data Integration Strategies
Omic Data Integration StrategiesOmic Data Integration Strategies
Omic Data Integration Strategies
 
5 data analysis case study
5  data analysis case study5  data analysis case study
5 data analysis case study
 
6 metabolite enrichment analysis
6  metabolite enrichment analysis6  metabolite enrichment analysis
6 metabolite enrichment analysis
 
The International Journal of Engineering and Science (The IJES)
The International Journal of Engineering and Science (The IJES)The International Journal of Engineering and Science (The IJES)
The International Journal of Engineering and Science (The IJES)
 
Prote-OMIC Data Analysis and Visualization
Prote-OMIC Data Analysis and VisualizationProte-OMIC Data Analysis and Visualization
Prote-OMIC Data Analysis and Visualization
 
Some statistical concepts relevant to proteomics data analysis
Some statistical concepts relevant to proteomics data analysisSome statistical concepts relevant to proteomics data analysis
Some statistical concepts relevant to proteomics data analysis
 

Similar to Metabolomic Data Analysis Workshop and Tutorials (2014)

Multivariate Analysis and Visualization of Proteomic Data
Multivariate Analysis and Visualization of Proteomic DataMultivariate Analysis and Visualization of Proteomic Data
Multivariate Analysis and Visualization of Proteomic DataUC Davis
 
Using open bioactivity data for developing machine-learning prediction models...
Using open bioactivity data for developing machine-learning prediction models...Using open bioactivity data for developing machine-learning prediction models...
Using open bioactivity data for developing machine-learning prediction models...Sunghwan Kim
 
Workflows, provenance and reporting: a lifecycle perspective at BIH 2013, Rome
Workflows, provenance and reporting: a lifecycle perspective at BIH 2013, RomeWorkflows, provenance and reporting: a lifecycle perspective at BIH 2013, Rome
Workflows, provenance and reporting: a lifecycle perspective at BIH 2013, RomeCarole Goble
 
Review of "Survey Research Methods & Design in Psychology"
Review of "Survey Research Methods & Design in Psychology"Review of "Survey Research Methods & Design in Psychology"
Review of "Survey Research Methods & Design in Psychology"James Neill
 
grizzly - informal overview - pydata boston 2013
grizzly - informal overview - pydata boston 2013 grizzly - informal overview - pydata boston 2013
grizzly - informal overview - pydata boston 2013 adrianheilbut
 
Are we really including all relevant evidence
Are we really including all relevant evidence Are we really including all relevant evidence
Are we really including all relevant evidence cheweb1
 
Data analytics in computer networking
Data analytics in computer networkingData analytics in computer networking
Data analytics in computer networkingStenio Fernandes
 
NEURAL Network Design Training
NEURAL Network Design  TrainingNEURAL Network Design  Training
NEURAL Network Design TrainingESCOM
 
Ijricit 01-002 enhanced replica detection in short time for large data sets
Ijricit 01-002 enhanced replica detection in  short time for large data setsIjricit 01-002 enhanced replica detection in  short time for large data sets
Ijricit 01-002 enhanced replica detection in short time for large data setsIjripublishers Ijri
 
32_Nov07_MachineLear..
32_Nov07_MachineLear..32_Nov07_MachineLear..
32_Nov07_MachineLear..butest
 
An introduction to variable and feature selection
An introduction to variable and feature selectionAn introduction to variable and feature selection
An introduction to variable and feature selectionMarco Meoni
 
Prediction Of Bioactivity From Chemical Structure
Prediction Of Bioactivity From Chemical StructurePrediction Of Bioactivity From Chemical Structure
Prediction Of Bioactivity From Chemical StructureJeremy Besnard
 
RAMSES: Robust Analytic Models for Science at Extreme Scales
RAMSES: Robust Analytic Models for Science at Extreme ScalesRAMSES: Robust Analytic Models for Science at Extreme Scales
RAMSES: Robust Analytic Models for Science at Extreme ScalesIan Foster
 
Automated Machine Learning Applied to Diverse Materials Design Problems
Automated Machine Learning Applied to Diverse Materials Design ProblemsAutomated Machine Learning Applied to Diverse Materials Design Problems
Automated Machine Learning Applied to Diverse Materials Design ProblemsAnubhav Jain
 
RUCK 2017 김성환 R 패키지 메타주성분분석(MetaPCA)
RUCK 2017 김성환 R 패키지 메타주성분분석(MetaPCA)RUCK 2017 김성환 R 패키지 메타주성분분석(MetaPCA)
RUCK 2017 김성환 R 패키지 메타주성분분석(MetaPCA)r-kor
 
презентация за варшава
презентация за варшавапрезентация за варшава
презентация за варшаваValeriya Simeonova
 
Data analytcis-first-steps
Data analytcis-first-stepsData analytcis-first-steps
Data analytcis-first-stepsShesha R
 
Data analysis
Data analysisData analysis
Data analysisamlbinder
 
Paper presentation @IPAW'08
Paper presentation @IPAW'08Paper presentation @IPAW'08
Paper presentation @IPAW'08Paolo Missier
 

Similar to Metabolomic Data Analysis Workshop and Tutorials (2014) (20)

Multivariate Analysis and Visualization of Proteomic Data
Multivariate Analysis and Visualization of Proteomic DataMultivariate Analysis and Visualization of Proteomic Data
Multivariate Analysis and Visualization of Proteomic Data
 
Using open bioactivity data for developing machine-learning prediction models...
Using open bioactivity data for developing machine-learning prediction models...Using open bioactivity data for developing machine-learning prediction models...
Using open bioactivity data for developing machine-learning prediction models...
 
Workflows, provenance and reporting: a lifecycle perspective at BIH 2013, Rome
Workflows, provenance and reporting: a lifecycle perspective at BIH 2013, RomeWorkflows, provenance and reporting: a lifecycle perspective at BIH 2013, Rome
Workflows, provenance and reporting: a lifecycle perspective at BIH 2013, Rome
 
Review of "Survey Research Methods & Design in Psychology"
Review of "Survey Research Methods & Design in Psychology"Review of "Survey Research Methods & Design in Psychology"
Review of "Survey Research Methods & Design in Psychology"
 
grizzly - informal overview - pydata boston 2013
grizzly - informal overview - pydata boston 2013 grizzly - informal overview - pydata boston 2013
grizzly - informal overview - pydata boston 2013
 
Are we really including all relevant evidence
Are we really including all relevant evidence Are we really including all relevant evidence
Are we really including all relevant evidence
 
Data analytics in computer networking
Data analytics in computer networkingData analytics in computer networking
Data analytics in computer networking
 
NEURAL Network Design Training
NEURAL Network Design  TrainingNEURAL Network Design  Training
NEURAL Network Design Training
 
Ijricit 01-002 enhanced replica detection in short time for large data sets
Ijricit 01-002 enhanced replica detection in  short time for large data setsIjricit 01-002 enhanced replica detection in  short time for large data sets
Ijricit 01-002 enhanced replica detection in short time for large data sets
 
32_Nov07_MachineLear..
32_Nov07_MachineLear..32_Nov07_MachineLear..
32_Nov07_MachineLear..
 
An introduction to variable and feature selection
An introduction to variable and feature selectionAn introduction to variable and feature selection
An introduction to variable and feature selection
 
Prediction Of Bioactivity From Chemical Structure
Prediction Of Bioactivity From Chemical StructurePrediction Of Bioactivity From Chemical Structure
Prediction Of Bioactivity From Chemical Structure
 
RAMSES: Robust Analytic Models for Science at Extreme Scales
RAMSES: Robust Analytic Models for Science at Extreme ScalesRAMSES: Robust Analytic Models for Science at Extreme Scales
RAMSES: Robust Analytic Models for Science at Extreme Scales
 
Automated Machine Learning Applied to Diverse Materials Design Problems
Automated Machine Learning Applied to Diverse Materials Design ProblemsAutomated Machine Learning Applied to Diverse Materials Design Problems
Automated Machine Learning Applied to Diverse Materials Design Problems
 
RUCK 2017 김성환 R 패키지 메타주성분분석(MetaPCA)
RUCK 2017 김성환 R 패키지 메타주성분분석(MetaPCA)RUCK 2017 김성환 R 패키지 메타주성분분석(MetaPCA)
RUCK 2017 김성환 R 패키지 메타주성분분석(MetaPCA)
 
презентация за варшава
презентация за варшавапрезентация за варшава
презентация за варшава
 
Delivering The Benefits of Chemical-Biological Integration in Computational T...
Delivering The Benefits of Chemical-Biological Integration in Computational T...Delivering The Benefits of Chemical-Biological Integration in Computational T...
Delivering The Benefits of Chemical-Biological Integration in Computational T...
 
Data analytcis-first-steps
Data analytcis-first-stepsData analytcis-first-steps
Data analytcis-first-steps
 
Data analysis
Data analysisData analysis
Data analysis
 
Paper presentation @IPAW'08
Paper presentation @IPAW'08Paper presentation @IPAW'08
Paper presentation @IPAW'08
 

More from Dmitry Grapov

R programming for Data Science - A Beginner’s Guide
R programming for Data Science - A Beginner’s GuideR programming for Data Science - A Beginner’s Guide
R programming for Data Science - A Beginner’s GuideDmitry Grapov
 
Network mapping 101 course
Network mapping 101 courseNetwork mapping 101 course
Network mapping 101 courseDmitry Grapov
 
Rise of Deep Learning for Genomic, Proteomic, and Metabolomic Data Integratio...
Rise of Deep Learning for Genomic, Proteomic, and Metabolomic Data Integratio...Rise of Deep Learning for Genomic, Proteomic, and Metabolomic Data Integratio...
Rise of Deep Learning for Genomic, Proteomic, and Metabolomic Data Integratio...Dmitry Grapov
 
Dmitry Grapov Resume and CV
Dmitry Grapov Resume and CVDmitry Grapov Resume and CV
Dmitry Grapov Resume and CVDmitry Grapov
 
Machine Learning Powered Metabolomic Network Analysis
Machine Learning Powered Metabolomic Network AnalysisMachine Learning Powered Metabolomic Network Analysis
Machine Learning Powered Metabolomic Network AnalysisDmitry Grapov
 
Complex Systems Biology Informed Data Analysis and Machine Learning
Complex Systems Biology Informed Data Analysis and Machine LearningComplex Systems Biology Informed Data Analysis and Machine Learning
Complex Systems Biology Informed Data Analysis and Machine LearningDmitry Grapov
 
Gene Ontology Enrichment Network Analysis -Tutorial
Gene Ontology Enrichment Network Analysis -TutorialGene Ontology Enrichment Network Analysis -Tutorial
Gene Ontology Enrichment Network Analysis -TutorialDmitry Grapov
 
American Society of Mass Spectrommetry Conference 2014
American Society of Mass Spectrommetry Conference 2014American Society of Mass Spectrommetry Conference 2014
American Society of Mass Spectrommetry Conference 2014Dmitry Grapov
 

More from Dmitry Grapov (9)

R programming for Data Science - A Beginner’s Guide
R programming for Data Science - A Beginner’s GuideR programming for Data Science - A Beginner’s Guide
R programming for Data Science - A Beginner’s Guide
 
Network mapping 101 course
Network mapping 101 courseNetwork mapping 101 course
Network mapping 101 course
 
Rise of Deep Learning for Genomic, Proteomic, and Metabolomic Data Integratio...
Rise of Deep Learning for Genomic, Proteomic, and Metabolomic Data Integratio...Rise of Deep Learning for Genomic, Proteomic, and Metabolomic Data Integratio...
Rise of Deep Learning for Genomic, Proteomic, and Metabolomic Data Integratio...
 
Dmitry Grapov Resume and CV
Dmitry Grapov Resume and CVDmitry Grapov Resume and CV
Dmitry Grapov Resume and CV
 
Machine Learning Powered Metabolomic Network Analysis
Machine Learning Powered Metabolomic Network AnalysisMachine Learning Powered Metabolomic Network Analysis
Machine Learning Powered Metabolomic Network Analysis
 
Complex Systems Biology Informed Data Analysis and Machine Learning
Complex Systems Biology Informed Data Analysis and Machine LearningComplex Systems Biology Informed Data Analysis and Machine Learning
Complex Systems Biology Informed Data Analysis and Machine Learning
 
Modeling poster
Modeling posterModeling poster
Modeling poster
 
Gene Ontology Enrichment Network Analysis -Tutorial
Gene Ontology Enrichment Network Analysis -TutorialGene Ontology Enrichment Network Analysis -Tutorial
Gene Ontology Enrichment Network Analysis -Tutorial
 
American Society of Mass Spectrommetry Conference 2014
American Society of Mass Spectrommetry Conference 2014American Society of Mass Spectrommetry Conference 2014
American Society of Mass Spectrommetry Conference 2014
 

Recently uploaded

How to Add Barcode on PDF Report in Odoo 17
How to Add Barcode on PDF Report in Odoo 17How to Add Barcode on PDF Report in Odoo 17
How to Add Barcode on PDF Report in Odoo 17Celine George
 
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATION
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATIONTHEORIES OF ORGANIZATION-PUBLIC ADMINISTRATION
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATIONHumphrey A Beña
 
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptx
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptxECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptx
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptxiammrhaywood
 
Influencing policy (training slides from Fast Track Impact)
Influencing policy (training slides from Fast Track Impact)Influencing policy (training slides from Fast Track Impact)
Influencing policy (training slides from Fast Track Impact)Mark Reed
 
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfGrade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfJemuel Francisco
 
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxINTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxHumphrey A Beña
 
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptxMULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptxAnupkumar Sharma
 
Food processing presentation for bsc agriculture hons
Food processing presentation for bsc agriculture honsFood processing presentation for bsc agriculture hons
Food processing presentation for bsc agriculture honsManeerUddin
 
USPS® Forced Meter Migration - How to Know if Your Postage Meter Will Soon be...
USPS® Forced Meter Migration - How to Know if Your Postage Meter Will Soon be...USPS® Forced Meter Migration - How to Know if Your Postage Meter Will Soon be...
USPS® Forced Meter Migration - How to Know if Your Postage Meter Will Soon be...Postal Advocate Inc.
 
ENGLISH 7_Q4_LESSON 2_ Employing a Variety of Strategies for Effective Interp...
ENGLISH 7_Q4_LESSON 2_ Employing a Variety of Strategies for Effective Interp...ENGLISH 7_Q4_LESSON 2_ Employing a Variety of Strategies for Effective Interp...
ENGLISH 7_Q4_LESSON 2_ Employing a Variety of Strategies for Effective Interp...JhezDiaz1
 
ROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptxROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptxVanesaIglesias10
 
Activity 2-unit 2-update 2024. English translation
Activity 2-unit 2-update 2024. English translationActivity 2-unit 2-update 2024. English translation
Activity 2-unit 2-update 2024. English translationRosabel UA
 
Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17Celine George
 
Inclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdf
Inclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdfInclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdf
Inclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdfTechSoup
 
ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...
ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...
ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...JojoEDelaCruz
 
4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptxmary850239
 
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)lakshayb543
 
Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4JOYLYNSAMANIEGO
 
Barangay Council for the Protection of Children (BCPC) Orientation.pptx
Barangay Council for the Protection of Children (BCPC) Orientation.pptxBarangay Council for the Protection of Children (BCPC) Orientation.pptx
Barangay Council for the Protection of Children (BCPC) Orientation.pptxCarlos105
 

Recently uploaded (20)

FINALS_OF_LEFT_ON_C'N_EL_DORADO_2024.pptx
FINALS_OF_LEFT_ON_C'N_EL_DORADO_2024.pptxFINALS_OF_LEFT_ON_C'N_EL_DORADO_2024.pptx
FINALS_OF_LEFT_ON_C'N_EL_DORADO_2024.pptx
 
How to Add Barcode on PDF Report in Odoo 17
How to Add Barcode on PDF Report in Odoo 17How to Add Barcode on PDF Report in Odoo 17
How to Add Barcode on PDF Report in Odoo 17
 
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATION
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATIONTHEORIES OF ORGANIZATION-PUBLIC ADMINISTRATION
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATION
 
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptx
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptxECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptx
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptx
 
Influencing policy (training slides from Fast Track Impact)
Influencing policy (training slides from Fast Track Impact)Influencing policy (training slides from Fast Track Impact)
Influencing policy (training slides from Fast Track Impact)
 
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfGrade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
 
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxINTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
 
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptxMULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
 
Food processing presentation for bsc agriculture hons
Food processing presentation for bsc agriculture honsFood processing presentation for bsc agriculture hons
Food processing presentation for bsc agriculture hons
 
USPS® Forced Meter Migration - How to Know if Your Postage Meter Will Soon be...
USPS® Forced Meter Migration - How to Know if Your Postage Meter Will Soon be...USPS® Forced Meter Migration - How to Know if Your Postage Meter Will Soon be...
USPS® Forced Meter Migration - How to Know if Your Postage Meter Will Soon be...
 
ENGLISH 7_Q4_LESSON 2_ Employing a Variety of Strategies for Effective Interp...
ENGLISH 7_Q4_LESSON 2_ Employing a Variety of Strategies for Effective Interp...ENGLISH 7_Q4_LESSON 2_ Employing a Variety of Strategies for Effective Interp...
ENGLISH 7_Q4_LESSON 2_ Employing a Variety of Strategies for Effective Interp...
 
ROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptxROLES IN A STAGE PRODUCTION in arts.pptx
ROLES IN A STAGE PRODUCTION in arts.pptx
 
Activity 2-unit 2-update 2024. English translation
Activity 2-unit 2-update 2024. English translationActivity 2-unit 2-update 2024. English translation
Activity 2-unit 2-update 2024. English translation
 
Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17
 
Inclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdf
Inclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdfInclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdf
Inclusivity Essentials_ Creating Accessible Websites for Nonprofits .pdf
 
ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...
ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...
ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...
 
4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx
 
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
 
Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4
 
Barangay Council for the Protection of Children (BCPC) Orientation.pptx
Barangay Council for the Protection of Children (BCPC) Orientation.pptxBarangay Council for the Protection of Children (BCPC) Orientation.pptx
Barangay Council for the Protection of Children (BCPC) Orientation.pptx
 

Metabolomic Data Analysis Workshop and Tutorials (2014)

  • 1. 2014 Workshop in Metabolomic Data Analysis Dmitry Grapov, PhD Introduction
  • 2. Important Introduction This is an introduction to a series of tutorials for metabolomic data analysis 1. Download all the required files and software at: https://sourceforge.net/projects/teachingdemos/files/WCMC%202014%20Summer%20Workshop/ 2. Make sure you have installed R (v3.1.1, http://cran.us.r-project.org/ ), shiny (v0.10.1, http://shiny.rstudio.com/ ) and a modern browser (e.g., Chrome). 3. Run the code in the folder software the file startup.R to launch all accompanying software; or download most current versions at the links below • DeviumWeb (https://github.com/dgrapov/DeviumWeb) • MetaMapR (https://github.com/dgrapov/MetaMapR) 4. Have a great time!
  • 4. Analysis at the Metabolomic Scale
  • 5. Statistical and Multivariate Analyses Group 1 Statistics + + = Multivariate Context Network Mapping Ranked statistically significant differences within a a biochemical context Group 2 What analytes are different between the two groups of samples? Statistical t-Test significant differences lacking rank and context Multivariate O-PLS-DA ranked differences lacking significance and context
  • 6. Statistical and Multivariate Analyses Group 1 Statistics + + = Multivariate Context Network Mapping Group 2 What analytes are different between the two groups of samples? Statistical t-Test Multivariate O-PLS-DA To see the big picture it is necessary too view the data from multiple different angles
  • 7. Cycle of Scientific Discovery Hypothesis Data Acquisition Data Processing Hypothesis Generation Data Analysis Data
  • 8. Sample Data Analysis and Visualization Variable Quality Assessment • use replicated mesurements and/or internal standards to estimate analytical variance Statistical and Multivariate • use the experimental design to test hypotheses and/or identify trends in analytes Functional • use statistical and multivariate results to identify impacted biochemical domains Network • integrate statistical and multivariate results with the experimental design and analyte metadata Sample Variable experimental design - organism, sex, age etc. analyte description and metadata - biochemical class, mass spectra, etc.
  • 9. Sample Data Analysis and Visualization Variable Quality Assessment • use replicated mesurements and/or internal standards to estimate analytical variance Statistical and Multivariate • use the experimental design to test hypotheses and/or identify trends in analytes Functional • use statistical and multivariate results to identify impacted biochemical domains Network • integrate statistical and multivariate results with the experimental design and analyte metadata Network Mapping Sample Variable experimental design - organism, sex, age etc. analyte description and metadata - biochemical class, mass spectra, etc.
  • 10. Data Quality Assessment Quality metrics •Precision (replicated measurements) •Accuracy (reference samples) Common tasks •normalization •outlier detection •missing values imputation *Finish lab: 1-Data quality
  • 11. Univariate Qualities •length (sample size) •center (mean, median, geometric mean) •dispersion (variance, standard deviation) •range (min / max), •quantiles •shape (skewness, kurtosis, normality, etc.) standard deviation mean
  • 12. Univariate Analyses •Identify differences in sample population means •sensitive to distribution shape •parametric = assumes normality •error in Y, not in X (Y = mX + error) •optimal for long data •assumed independence •false discovery rate (FDR) long wide n-of-one
  • 13. False Discovery Rate (FDR) Type I Error: False Positives •Type II Error: False Negatives •Type I risk = •1-(1-p.value)m m = number of variables tested FDR correction • p-value adjustment or estimate of FDR (Fdr, q-value) Bioinformatics (2008) 24 (12):1461-1462
  • 14. Statistical Analysis: achieving ‘significance’ significance level (α) and power (1-β ) effect size (standardized difference in means) sample size (n) Power analyses can be used to optimize future experiments given preliminary data Example: use experimentally derived (or literature estimated) effect sizes, desired p-value (alpha) and power (beta) to calculate the optimal number of samples per group *finish lab 2-statistical analysis
  • 15. Outlier Detection • 1 variable (univariate) • 2 variables (bivariate) • >2 variables (multivariate)
  • 16. bivariate vs. multivariate (scatter plot) outliers? mixed up samples (PCA scores plot) Outlier Detection
  • 17. Batch Effects Drift in >400 replicated measurements across >100 analytical batches for a single analyte Principal Component Analysis (PCA) of all analytes, showing QC sample scores Acquisition batch Abundance QCs embedded among >5,5000 samples (1:10) collected over 1.5 yrs If the biological effect size is less than the analytical variance then the experiment will incorrectly yield insignificant results
  • 18. Analyte specific data quality overview Sample specific normalization can be used to estimate and remove analytical variance Raw Data Normalized Data low precision log mean %RSD high precision Samples QCs Batch Effects *finish lab 3-Data Normalization
  • 19. Clustering Identify •patterns •group structure •relationships •Evaluate/refine hypothesis •Reduce complexity Artist: Chuck Close
  • 20. Cluster Analysis Use the concept similarity/dissimilarity to group a collection of samples or variables Approaches •hierarchical (HCA) •non-hierarchical (k-NN, k-means) •distribution (mixtures models) •density (DBSCAN) •self organizing maps (SOM) Linkage k-means Distribution Density
  • 21. Hierarchical Cluster Analysis • similarity/dissimilarity defines “nearness” or distance euclidean X Y manhattan Mahalanobis X Y X Y * non-euclidean
  • 22. Hierarchical Cluster Analysis Agglomerative/linkage algorithm defines how points are grouped single complete centroid average
  • 24. Hierarchical Cluster Analysis How does my metadata match my data structure? Exploration Confirmation *finish lab 4-Cluster Analysis
  • 25. Projection of Data The algorithm defines the position of the light source Principal Components Analysis (PCA) • unsupervised • maximize variance (X) Partial Least Squares Projection to Latent Structures (PLS) • supervised • maximize covariance (Y ~ X) James X. Li, 2009, VisuMap Tech.
  • 26. Interpreting PCA Results Variance explained (eigenvalues) Row (sample) scores and column (variable) loadings
  • 27. How are scores and loadings related?
  • 28. Centering and Scaling PMID: 16762068 *finish lab 5-Principal Components Analysis
  • 29. Use PLS to test a hypothesis on a multivariate level Partial Least Squares (PLS) is used to identify maximum modes of covariance between X measurements and Y (hypotheses) PCA PLS time = 0 120 min.
  • 30. Modeling multifactorial relationships ~two-way ANOVA dynamic changes among groups
  • 31. PLS Related Objects Model •dimensions, latent variables (LV) •performance metrics (Q2, RMSEP, etc) •validation (training/testing, permutation, cross-validation) •orthogonal correction Samples •scores •predicted values •residuals Variables •Loadings •Coefficients: summary of loadings based on all LVs •VIP: variable importance in projection •Feature selection
  • 32. “goodness” of the model is all about the perspective Determine in-sample (Q2) and out-of- sample error (RMSEP) and compare to a random model •permutation tests •training/testing *finish lab 6-Partial Least Squares and lab 7-Data Analysis Case Study
  • 33. Functional Analysis Identify changes or enrichment in biochemical domains • decrease • increase Nucl. Acids Res. (2008) 36 (suppl 2): W423-W426.doi: 10.1093/nar/gkn282
  • 34. Functional Analysis: Enrichment Biochemical Pathway Biochemical Ontology *finish lab 8-Metabolite Enrichment Analysis
  • 35. Connections and Contexts Biochemical (substrate/product) •Database lookup •Web query Chemical (structural or spectral similarity ) •fingerprint generation BMC Bioinformatics 2012, 13:99 doi:10.1186/1471-2105-13-99 Empirical (dependency) •correlation, partial-correlation
  • 36. Network Mapping 2. Calculate Mappings 1. Calculate Connections 3. Create Mapped Network Grapov D., Fiehn O., Multivariate and network tools for analysis and visualization of metabolomic data, ASMS, June 08, 2013, Minneapolis, MN
  • 37. Mapping Analysis Results to Networks Analysis results Network Annotation Mapped Network *finish lab 9-Network Mapping I
  • 40. Correlation networks •simple to calculate Complex lipids correlation network in mouse serum
  • 41. Correlation Networks •can be difficult to interpret •poorly discriminate between direct and indirect associations Complex lipids correlation network in mouse heart tissue
  • 42. Partial correlations can help simplify networks and preference direct over indirect associations. 10.1007/978-1-4614-1689-0_17 Complex lipids partial correlation network in human plasma
  • 43. Mass Spectral Connections Watrous J et al. PNAS 2012;109:E1743-E1752 *finish lab 10-Network Mapping II
  • 44. Software and Resources •DeviumWeb- Dynamic multivariate data analysis and visualization platform url: https://github.com/dgrapov/DeviumWeb •imDEV- Microsoft Excel add-in for multivariate analysis url: http://sourceforge.net/projects/imdev/ •MetaMapR- Network analysis tools for metabolomics url: https://github.com/dgrapov/MetaMapR •TeachingDemos- Tutorials and demonstrations •url: http://sourceforge.net/projects/teachingdemos/?source=directory •url: https://github.com/dgrapov/TeachingDemos •Data analysis case studies and Examples url: http://imdevsoftware.wordpress.com/
  • 45. Questions? dgrapov@ucdavis.edu This research was supported in part by NIH 1 U24 DK097154