Analyse quantitative de données de RNA-seq Normalisation ...rssf.i2bc.paris-saclay.fr/X-fer/IFSBM/MADillies-RNAseq-stats.pdf · To consult a statistician after an experiment is ﬁnished

. . . . . .

Analyse quantitative de données de RNA-seqNormalisation

et Analyse Différentielle

Marie-Agnès [email protected]

Institut PasteurPlate-forme Transcriptome et Epigénome

et Centre de Bioinformatique, Biostatistique et Biologie Intégrative

Cancer et Génomique, vers une médecine de précisionBig data moléculaire et son traitement

5 mai 2015

. . . . . .

A la naissance du RNA-seq...

. . . . . .

Quelques années plus tard :-(

. . . . . .

Les grandes étapes du RNA-seq

. . . . . .


. . . . . .


. . . . . .

Thèmes abordés

Caractéristiques des données de comptage

La planification de l’expérience

La normalisation

La recherche de gènes différentiellement exprimés

Et après ? Recherche d’enrichissement en catégoriesfonctionnelles

L’analyse différentielle d’isoformes

. . . . . .

Caractéristiques des données de comptageDescription des données de comptageVariabilités techniques associées aux données de comptage


La normalisationPourquoi normaliser ?Comment normaliser ?

La recherche de gènes différentiellement exprimésLois des données de comptageTests disponibles et performancesDistribution des p-valeurs brutesTenir compte des comparaisons multiples : l’ajustement desp-valeurs



. . . . . .

Comptage des reads par région d’intérêt

. . . . . .

Description des comptages : exemple

. . . . . .

[Marioni et al 2008] : Effets librairie / lane / flow cell

▶ 2 conditions : Foieet Rein (humains)

▶ 7 répétitions parcondition, 2 flowcells

▶ 2 concentrations :3pM (10 éch.) et1.5pM (4 éch.)

. . . . . .

Effet Concentration de la librairie

Comparaison de répétitions techniques, même flow cell

Red dots : genes above the 95th percentile

Blue dots : genes above the 99.5th percentile

. . . . . .

Effets lane / flow cell / concentration

. . . . . .

[Bullard et al 2010] : Effet préparation de la librairie

. . . . . .

Effets librairie / lane / flowcell : résumé

From [Marioni et al 2008] and [Bullard et al 2010]

▶ Les répétitions techniques sont très reproductibles▶ lane < flow cell < concentration ou préparation de la librairie≪ variabilité biologique

▶ La variabilité technique (même librairie, même concentration)(effet lane/flow cell) est bien décrite par une loi de Poisson

Mais ...Les librairies ont été préparées à partir d’ ARN commerciaux. Lavariabilité technique vient aussi de l’étape d’extraction des ARN etde celle de déplétion des ribosomiques / enrichissement desmessagers lors de la préparation des librairies

. . . . . .

Différentes sources de variabilité

Origine Source Conséquencesséquenceur lane nombre de reads

flowcelllibrairie préparation ( jour, expér.) distribution des reads

ribosomiques (déplétion, nombre de reads/ enrichissement messagers)

concentration qualité et nombre de reads

random priming distribution des reads non uniformePCR GC content [Pickrell et al, 2010]

[Benjamini and Speed 2012]ARN extraction variabilité biologique non souhaitée

qualité nombre et qualité des reads

gène longueur nombre de reads

. . . . . .







. . . . . .

Les dictons du jour

To consult a statistician after an experiment is finished is oftenmerely to ask him to conduct a post-mortem examination. He canperhaps say what the experiment died of (Ronald A. Fisher, Indianstatistical congress, 1938, vol. 4, p 17).

While a good design does not guarantee a successful experiment,a suitably bad design guarantees a failed experiment (KathleenKerr, Atelier Inserm 145, 2003)

. . . . . .

Pourquoi un plan d’expérience ?

Pour contrôler l’ensemble des variabilités techniques afinque l’expérience réponde à la question biologique posée

1. Quelle est la question biologique ?

2. Comment estimer les variabilités biologiques associées ?

3. Comment contrôler l’impact des variabilités techniques ?

. . . . . .

1. La question biologique

Trouver les gènes différentiellement exprimés entre les conditionsbiologiques A et B

On fait pousser un sauvage et deux mutants sur deux milieux deculture différents. Les ARN sont extraits en phase exponentielle etstationnaire. Quels sont les gènes différentiellement exprimés ? ? ?

. . . . . .

1. La question biologique

Trouver les gènes différentiellement exprimés entre les conditionsbiologiques A et B

On fait pousser un sauvage et deux mutants sur deux milieux deculture différents. Les ARN sont extraits en phase exponentielle etstationnaire. Quels sont les gènes différentiellement exprimés ? ? ?

. . . . . .

2. Estimer la variabilité biologique

. . . . . .


. . . . . .


. . . . . .


Plus la variance intra-condition est élevée

▶ plus il faut de répétitions pour l’estimer correctement▶ plus la variation d’expression détectable sera élevée

DefinitionLa puissance d’un test est sa capacité à détecter des écarts

. . . . . .

2. Combien de répétitions biologiques ?[Soneson and Delorenzi 2013]

. . . . . .

2. Combien de répétitions biologiques ?[Soneson and Delorenzi 2013]

. . . . . .

3. Contrôler les variabilités techniques

Répétitions techniques ?

▶ une même librairie séquencée plusieurs fois▶ permet d’augmenter la profondeur de séquençage▶ les réplicats techniques sont alors sommés

Il faut privilégier les réplicats biologiques

Multiplexage et autres effets techniques

▶ éviter la confusion d’effets▶ au sein d’une expérience, utiliser toujours le même taux de

multiplexage

. . . . . .

3. Contrôler les variabilités techniques

Répétitions techniques ?

▶ une même librairie séquencée plusieurs fois▶ permet d’augmenter la profondeur de séquençage▶ les réplicats techniques sont alors sommés

Il faut privilégier les réplicats biologiques

Multiplexage et autres effets techniques

▶ éviter la confusion d’effets▶ au sein d’une expérience, utiliser toujours le même taux de

multiplexage

. . . . . .

La confusion d’effets

Confusion entre lane etcondition :

distribution de l’effet lane sur lesdeux conditions

Confusion partielle :distribution de l’effet lane sur toutes les

conditions

. . . . . .

La confusion d’effets

Confusion entre lane etcondition :

distribution de l’effet lane sur lesdeux conditions

Confusion partielle :distribution de l’effet lane sur toutes les

conditions

. . . . . .

Autre (mauvais) exemple : l’effet jour

● ●

●

●

●

●

−40 −30 −20 −10 0 10 20

−20

−10

010

2030

SLX042−03, Principal Component Analysis

PC1 (54.6%)

PC

2 (3

4.7%

)

T0−1 T0−2

T10−1

T10−2

T30−1

T30−2

. . . . . .

Memento pour un beau (bon) plan d’expérience

Avant

▶ Poser précisément la (les) question(s) biologiques(s)▶ Prévoir au moins trois répétitions biologiques par condition▶ Répartir les échantillons sur les lanes / les flowcells

▶ en fonction des comparaisons souhaitées▶ en évitant la confusion d’effets

Pendant

▶ Contrôler au mieux les variabilités techniques à toutes lesétapes humides (effets jour, expérimentateur, protocole, etc)

. . . . . .







. . . . . .

Pourquoi normaliser ?

Pour corriger les biais techniques systématiques et rendre lescomptages comparables entre échantillons

Cadre de la normalisation

▶ Données RNA-seq▶ Expériences d’expression différentielle▶ Données de comptage (brutes, entières)

Nombre total de reads (library size) : Nombre de reads séquencés,alignés et comptés dans un échantillon donné (somme en colonnedu tableau de comptages)

. . . . . .

Les normalisations proposées dans la littérature

Quatre grandes catégories

▶ Ajustement des distributions (Total Read Count, UpperQuartile, Median, Full Quantile)

▶ Concept de ’Nombre de reads effectif’ (TMM, DESeq)▶ Prise en compte de la longueur du gène (RPKM, FPKM)▶ Correction du biais en GC (cqn)

Remarques

▶ Certaines méthodes normalisent les comptages, d’autres leslibrary sizes

▶ Certaines méthodes ont été conçues dans le cadre dans unmodèle statistique pour l’analyse différentielle, d’autres non

. . . . . .

Notations

▶ xij : nombre de reads pour le gène i dans l’échantillon j▶ Nj : nombre de reads dans l’échantillon j (library size of

sample j)▶ n : nombre d’échantillons de l’expérience▶ sj : facteur de normalisation de l’échantillon j▶ Li : longueur du gène i

. . . . . .

Normalisation par le nombre total de reads (TC)[Dudoit 2010]

Corrige les différences sur le nombre total de reads▶ Hypothèse Les comptages sont proportionnels au niveau

d’expression et à la profondeur de séquençage (mêmes ARNen proportions équivalentes)

▶ Méthode Divise le comptage du gène par le nombre total dereads

xij

Nj=

xij

sj, sj = Nj (1)

. . . . . .

▶ Problème rend les fréquences (proportions) comparablesentre lanes, pas les comptages

▶ Solution divise Nj par la moyenne des Nl , l ∈ 1, .., n

sj =Nj

1n∑

l Nl(2)

. . . . . .

Deux variantes de la normalisation Total Count

MotivationLe nombre total de reads est souvent très dépendant d’un petitnombre de gènes très fortement exprimés

Normalisation par le troisième Quartile (UQ)

xij

sj, sj =

Q3j1n∑

l Q3l(3)

Normalisation par la médiane (Med)

xij

sj, sj =

medianj1n∑

l medianl(4)

Q3 et median sont calculés après exclusion des gènes absents detous les échantillons

. . . . . .

Normalisation Quantile (FQ)

Hypothèse Les comptages ont la même distribution dans tous leséchantillons

Données brutes Données normalisées

. . . . . .

Le concept de "Nombre total de reads effectif"

MotivationDes conditions biologiques différentes peuvent exprimer desrépertoires d’ARN différents, associés à des quantités totalesd’ARN différentes

HypothèseLa majorité des transcrits a une expression constante entre lesdeux conditions

. . . . . .

Trimmed Mean of M values (TMM)[Robinson et Oshlack 2010]

. . . . . .

Trimmed Mean of M values (TMM)[Robinson et Oshlack 2010]

Remarques

1. Les facteurs de normalisation fournis par TMM s’appliquentaux nombres totaux de reads, pas aux comptages

2. Méthode disponible dans le package R edgeR

Calcul des comptages normalisés

▶ N′j : Nombre totaux de reads normalisés (N′j = Nj ∗ sj)▶ utilisés comme base d’une normalisation par le nombre total

de reads

sj =N′j

1n∑

j N′j(5)

. . . . . .

Normalisation du package DESeq[Anders et Huber 2010]

sj = medianixij

(∏nν=1 xiν)1/n (6)

. . . . . .

Normalisation RPKM [Mortazavi et al 2008]

Reads Per Kilobase per Million mapped reads

▶ Hypothèse les comptages sont proportionnels au niveaud’expression du gène, à sa longueur et à la profondeur duséquençage (mêmes ARN en proportion égale)

▶ Méthode divise le comptage du gène par le nombre total dereads (en millions) et par la longueur du gène (en kilobases)

xij

Nj ∗ Li∗ 106 ∗ 103 (7)

▶ permet de comparer le niveau d’expression de gènes au seind’un même échantillon

. . . . . .

Conditional Quantile Normalization (CQN)[Hansen et al 2011]

MotivationCorrige les variabilités techniques : biais spécifiques deséchantillons (contenu en GC), longueur du gène, différences dedistribution des comptages

Normalisation en 2 étapes

1. correction des biais (GC et longueur du gène) à l’aide d’unfacteur de correction adapté à chaque gène

2. suivie d’une normalisation quantile pour égaliser lesdistributions des comptages

. . . . . .

Quelle normalisation ? Une comparaison de méthodes

Méthodes incluses dans la comparaison

▶ Ajustement des distributions : Total Count, Median, UpperQuartile, Quantile

▶ Concept de "Nombre total de reads effectif" (Effective librarysize) : TMM, DESeq

▶ Prise en compte de la longueur du gène : RPKM

. . . . . .

Données réelles

Données simulées à partir des données souris

▶ Proportion de gènes différentiels : de 0 à 30%

▶ library sizes équivalentes / non équivalentes

▶ présence / absence de gènes majoritaires

. . . . . .

Résultats sur données réelles

. . . . . .

Erreur de type I et Puissance du test

A partir des données simulées : seuls TMM et DESeq contrôlent letaux d’erreur de type 1 sans altération de la puissance du test

Equivalent library sizes / Presence of majority genes

0.00

0.10

0.20

TC UQ Med DESeq TMM Q RPKM RawCount

Fals

e−po

sitiv

e ra

te

a

0.35

0.45

0.55

0.65

Pow

er

TC UQ Med DESeq TMM Q RPKM RawCount

b

. . . . . .

So the Winner is ... ?

Dans la majorité des casLes méthodes donnent des résultats similaires

Cependant ...Des différences apparaissent en fonction des caractéristiques desdonnées : présence de gènes majoritaires et variabilité desrépertoires exprimés

. . . . . .

So the Winner is ... TMM and DESeq !

Dans la majorité des casLes méthodes donnent des résultats similaires

Cependant ...Des différences apparaissent en fonction des caractéristiques desdonnées : présence de gènes majoritaires et variabilité desrépertoires exprimés

. . . . . .

The Winner is ... TMM and DESeq !

Considérations pratiques

▶ Les deux méthodes sont intégrées dans des packages R pourl’analyse différentielle

▶ Faciles d’utilisation et bien documentés▶ Très discutés sur les forums et les listes de diffusion

spécialisés (http://seqanswers.com/)▶ DESeq(2) fournit des données de comptage normalisées▶ Pour TMM il faut les calculer soi-même▶ TMM normalise les library sizes, DESeq normalise les

comptages▶ Seuls les comptages non nuls peuvent être normalisés

http://seqanswers.com/

. . . . . .







. . . . . .

Trouver les gènes différentiellement exprimés

Position du problème

Test statistique (test d’hypothèse)

▶ sans modèle (non paramétrique) : pas assez de répétitions▶ avec un modèle (paramétrique) : loi des données de

comptage ?

. . . . . .

La loi de Poisson

Loi de probabilité discrète, décrit le nombre d’occurrences d’unévénement donné survenant pendant un laps de temps fixé

DefinitionX suit une loi de Poisson de paramètre λ > 0

P(X = x) = e−λλx

x!(8)

E(X) = V(X) = λ (9)

. . . . . .

La loi binomiale négative

Epreuve de BernoulliExpérience aléatoire à deux issues possibles : succès (S) ouéchec (E)p : probabilité associée au succès

Loi Binomiale NégativeOn répète une épreuve de Bernoulli de paramètre p. La loi BN deparamètres (n, p) décrit la distribution du nombre k d’échecs avantd’obtenir n succès

De Poisson à BNUne Binomiale Négative est un mélange de lois de Poisson deparamètre variable. C’est une alternative robuste à la loi dePoisson en cas de sur-dispersion des données

. . . . . .

Tests disponibles

▶ Transformation + modèle gaussien : limma - voom▶ Poisson : TSPM▶ Binomiale Négative : edgeR, DESeq(2), NBPSeq, baySeq,

ShrinkSeq, ...▶ Approche fréquentiste : edgeR, DESeq(2), NBPSeq, TSPM,

...▶ Approche bayésienne : baySeq, ShrinkSeq, EBSeq, ...▶ Approche non-paramétrique : SAMSeq, NOISeq, ...

. . . . . .

Méthodes pour la comparaison de deux conditions[Soneson and Delorenzi 2013]

▶ comparaison de 11 tests statistiques▶ disponibles sous R▶ adaptés aux données de comptages (analyse par gène)▶ normalisation DESeq ou TMM selon les packages

. . . . . .

Résultats

Les performances des méthodes d’analyse dépendent desfacteurs suivants :▶ nombre de réplicats (et différemment selon les méthodes)

▶ limma, méthodes non paramétriques : manquent de puissance▶ autres méthodes : taux de FP (très) mal contrôlé avec 2

répétitions▶ moins de faux positifs dans les fortes expressions▶ les listes de gènes deviennent plus proches quand on

augmente le nombre de répétitions

▶ (dés)équilibre de l’expression différentielle▶ présence d’outliers

▶ DESeq2 : détection des outliers▶ "robust" edgeR

▶ façon d’estimer la dispersion (edgeR : privilégier tagwise)

. . . . . .

L’estimation de dispersion : le nerf de la guerre

ProblèmeEstimer une dispersion fiable à partir d’un nombre réduit devaleurs (moins de 5)

Le modèle binomial négatif

Xgi ∼ NB(µgi , σ2gi), σ

2gi = µgi + ϕgµ

2gi

CV2 =1µgi

+ ϕg = CV2technique + CV2

biologique

. . . . . .





2gi = µgi + ϕgµ

2gi

CV2 =1µgi


biologique

. . . . . .





2gi = µgi + ϕgµ

2gi

CV2 =1µgi


biologique

. . . . . .

Estimer la dispersion biologique ϕg

Hypothèses

▶ DESeq(2) : la dispersion est une fonction de la moyenne▶ edgeR : la dispersion par gène se rapproche de la dispersion

commune

1e+00 1e+02 1e+04 1e+06

1e−08

1e−06

1e−04

1e−02

1e+00

SLX035−01−comp−edgeR−DESeq2 − Dispersions

Mean of normalized counts

Dis

pers

ion

●

●

●

gene−estfittedfinal

Figure: DESeq2

0 5 10 15

0.10

0.15

0.20

0.25

0.30

0.35

0.40

SLX035−01−comp−edgeR−DESeq2 − BCV plot

Average log CPM

Bio

logi

cal c

oeffi

cien

t of v

aria

tion

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

● ●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●●●

●

●

●

● ●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

● ●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●●●

●

●

●

●

●●

●

●

●

●

●●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●● ●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●● ●

●

●

●●● ●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

● ●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

● ●

●

●

●

●

●

●

● ●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

● ●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

● ●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●●

●●

●

●

● ●

●

●●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

● ●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●● ●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

● ● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●● ●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

● ●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●●

●

●

●

●

●

●

●

●

● ● ●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

● ● ● ●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

● ●

●●●

●

●

●

●

●

●

●●

●

●

●●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●● ●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

● ●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

● ● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

● ●

●

●

●

● ●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

● ●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

● ●

●

●

●

●●

●

●

●

● ●

●●

●

●

●●

●

● ●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

● ●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●● ●●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●● ●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●●●● ●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

● ●●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●●

●

●● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●●

●

●

● ●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

● ●

●

●

●

●●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●●

● ●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●●

●

●●

●

●

●

●●

●

●

● ● ●

●

●

●

●

●

● ●

● ●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

● ●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● TagwiseCommonTrend

Figure: edgeR

. . . . . .





2gi = µgi + ϕgµ

2gi

CV2 =1µgi


biologique

ConséquenceLa variabilité technique est dominante dans les faibles comptages,négligeable dans les forts comptages

. . . . . .

En résumé ...

▶ Avec deux réplicats biologiques, toutes les méthodessont peu performantes (contrôlent mal le FDR, ou manquentde puissance)

▶ Aucune méthode n’est meilleure que les autres en toutecirconstance : le choix dépend du jeu de données

Critères de choix de la méthode

▶ Nombre de réplicats▶ Présence / absence d’outliers▶ Dispersion constante / variable entre conditions▶ Symétrie / dissymétrie de la distribution des gènes DE▶ Plan d’expérience simple / complexe

. . . . . .

edgeR et DESeq(2) : les plus utilisés

DESeqAbandonné au profit de DESeq2

DESeq2 et edgeR : des points communs ...

▶ Packages R, bien documentés, simples à utiliser▶ Une analyse en trois étapes : normalisation, estimation de

dispersion, test statistique▶ Loi binomiale négative et modèle linéaire généralisé (GLM) :

permet l’analyse de plans d’expérience à plusieurs facteurs

et des différences

▶ estimation de la dispersion▶ gestion des outliers▶ filtrage des faibles comptages

. . . . . .

Dans les deux cas ...Bien préciser la version utilisée

SARToolshttps://github.com/PF2-pasteur-fr/SARTools

▶ le contrôle qualité des données▶ la normalisation et l’analyse différentielle▶ l’exportation des listes de gènes différentiels▶ un rapport complet d’analyse en HTML incluant le nom et la

version de tous les packages utilisés pour l’analyse

https://github.com/PF2-pasteur-fr/SARTools

. . . . . .

Distribution des p-valeurs brutes

Le test statistique renvoie une p-valeur par gène (probabilité de setromper en déclarant le gène différentiel)

. . . . . .

Distribution des p-valeurs brutes

Le test statistique renvoie une p-valeur par gène (probabilité de setromper en déclarant le gène différentiel)

. . . . . .

L’ajustement des p-valeurs brutes

Le problème des comparaisons multiples

▶ un gène : risque d’erreur = α▶ m gènes (m tests indépendants) : risque d’erreur = m ∗ α

Différentes méthodes pour contrôler le risque d’erreur

▶ V : nombre de faux positifs, R : nombre de gènes déclarésdifférentiels

▶ FWER (Family Wise Error Rate) : P(V ≥ 1) (ex : Bonferroni)▶ FDR (False Discovery Rate) : E(V/R) (ex : BH

[Benjamini and Hochberg 1995])

. . . . . .







. . . . . .

Biais sur la longueur du gène[Oshlack and Wakefield 2009]

. . . . . .

Recherche d’enrichissement en catégories fonctionnelles

goseq [Young et al, 2010]

▶ Approche : parmi les gènes diférentiellement exprimés,certaines catégories fonctionnelles sont-ellessur-représentées ?

▶ Tient compte du biais sur la longueur des gènes pour établirla sur- ou sous-représentation des catégories fonctionnelles

DESeq2 [Simon Anders, personal communication, 2013]

▶ Approche : les Fold Changes des gènes d’une catégoriedonnée sont-ils particulièrement élevés ?

▶ nécessite une estimation non biaisée des fold changes

. . . . . .







. . . . . .

L’analyse différentielle d’isoformes [Trapnell et al, 2013]

Une alternative :DEXSeq : differential exon usage [Anders et al, 2012]

. . . . . .

DEXSeq : differential exon usage [Anders et al, 2012]

usage of an exon =reads mapping to the exon

reads mapping to any other exon of the same gene(10)

. . . . . .

DEXSeq : exemple

. . . . . .

Take-home message

▶ Les statistiques commencent AVANT l’extraction des ARN▶ Il faut faire des répétitions biologiques▶ La normalisation est indispensable, toutes les méthodes ne

se valent pas▶ Elle doit prendre en compte la variabilité des répertoires

exprimés▶ RPKM n’est pas une bonne normalisation pour l’analyse

différentielle

. . . . . .

Take-home message

▶ Il existe de nombreuses méthodes d’analyse différentielle, lechoix dépend des caractéristiques des données

▶ Il vaut mieux favoriser un plus grand nombre de réplicats audétriment de la profondeur de séquençage [Robles et al 2012](sauf lorsque l’on s’intéresse aux gènes peu exprimés)[Rapaport et al 2013]

▶ L’estimation du niveau d’expression des isoformes ne permetpas d’appliquer les modèles adaptés aux données decomptage

. . . . . .

Marioni JC, Mason CE et al.RNA-seq : An assessment of technical reproducibility and comparison with gene expression arrays.Genome Research 2008, 18 : 1509-1517

Bullard JH, Purdom E, Hansen KD, Dudoit S.Evaluation of statistical methods for normalization and differential expression in mRNA-seq experiments.BMC Bioinformatics 2010, 11 :94

Robinson MD and Smyth, GK.Moderated statistical tests for assessing differences in tag abundance.Bioinformatics 23(21) ; 2881-2887

Robinson MD and Smyth, GK.Small-sample estimation of negative binomial dispersion, with applications to SAGE dataBiostatistics (2008), 9, 2 ; 321-332

Robinson MD, McCarthy DJ, Smyth, GK.edgeR : a Bioconductor package for differential expression analysis of digital gene expression data.Bioinformatics 2009

Mortazavi A, Williams BA, McCue K, Schaeffer L, Wold B.Mapping and quantifying mammalian transcriptomes by RNA-seq.Nature Methods, 2008 Jul ; 5(7) ; 621-628

Robinson MD, Oshlack A.A scaling normalization method for differential expression analysis of RNA-seq data.Genome Biology 2010, 11 :R25

Anders, S and Huber, W.Differential expression analysis for sequence count data.Nature Precedings 2010, march.

Hansen KD, Brenner SE, Dudoit S.Biases in Illumina transcriptome sequencing caused by random hexamer priming.Nucleic Acids Research, 2010, 1-7.

. . . . . .

Pickrell JK, Marioni JC, Pai AA, Degner JF, Engelhardt BE, Nkadori E, Veyrieras JB, Stephens M, Gilad Y,Pritchard JK.Understanding mechanisms underlying human gene expression variation with RNA sequencing.Nature letters, 2010, vol 464.

Dudoit S, Maya O and Jacob L.Short course on RNA seq and CHiP seq data analysis.Valencia, Nov. 2010.

Bolstad, B. M., Irizarry R. A., Astrand, M., and Speed, TP.A comparison of normalization methods for high density oligonucleotide array data based on bias and variance.Bioinformatics 19, 185-193, 2003.

Eisenberg EE and Levanon EY.Human housekeeping genes are compact.Trends Genet, 19(7) :362-365.

Su AI, Wiltshire T, Batalov S, Lapp H, Ching KA, Block D, Zhang J, Soden R, Hayakawa M, Kreiman G, Cooke MP,Walker JR, Hogenesch JB.A gene atlas of the mouse and human protein-encoding transcriptomes.Proc. Natl. Acad. Sci. USA, 101(16) :6062-6067.

Alicia Oshlack and Matthew J WakefieldTranscript length bias in RNA-seq confounds systems biologyBiology Direct 2009, 4 :14.

Yoav Benjamini and Terrence P. SpeedSummurizing and correcting the GC content bias in high throughput sequencingNucleic Acids Research, 2012, 1-14.

Charlotte Soneson and Mauro DelorenziA comparison of methods for differential expression analysis of R ?A-seq dataBMC Bioinformatics 2013, 14 :91

Franck Rapaport, Raya Khanin, Yupu Liang, Azra Krek, Paul Zumbo, Christopher Mason, Nicolas Socci, DoronBetelComprehensive evaluation of differential expression analysis methods for RNA-seq data

. . . . . .

arXiv preprint arXiv :1301 :5277

Benjamini Y and Hochberg YControlling the False Discovery Rate : A Practical and Powerful Approach to Multiple TestingJournal of the Royal Statistical Society, 1995, 57 :1, 289–300

Young, M.D., Wakefield, M.J., Smyth, G.K., Oshlack, A.,Gene ontology analysis for RNA-seq : accounting for selection biasGenome Biology, 11, 2, Feb 2010, R14

Robles J.A., Qureshi S.E., Stephen S.J., Wilson S.R., Burden C.J., Taylor J.M.Efficient experimental design and analysis strategies for the detection of differential expression usingRNA-SequencingBMC Genomics 2012, 13 :484

Simon AndersComparative analysis of RNA-seq data with DESeq and DEXseqhttp ://www.bioconductor.org/help/course-materials/2013/CSAMA2013/tuesday/morning/Anders_DESeq_DEXSeq.pdf

Cole Trapnell, David G Hendrickson, Martin Sauvageau, Loyal Goff, John L Rinn and Lior PachterDifferential analysis of gene regulation at transcript resolution with RNA-seqNature Biotech, Vol 31, n˚1, January 2013

Simon Anders, Alejandro Reyes and Wolfgang HuberDetecting differential usage of exons from RNA-seq dataGenome Research 2012 :22

Documents

Analyse quantitative de données de RNA-seq Normalisation ...rssf.i2bc.paris-saclay.fr/X-fer/IFSBM/MADillies-RNAseq-stats.pdf · To consult a statistician after an experiment is ﬁnished