Validità Esterna. Validità Interna. (come uno studio clinico è stato pianificato) (come uno studio clinico è stato condotto)

Transcript

1 Validità Interna (come uno studio clinico è stato pianificato) Validità Esterna (come uno studio clinico è stato condotto)

2

3 Pappagallo Cinquini - Moschetti

4

5 Liberating Structures in sintesi Favorire la creazione di reti tra Colleghi Ricerca di innovazioni e soluzioni Analisi degli obiettivi Condivisione di idee Debriefing (riflessioni e sintesi) Soluzioni a sfide

6 What?, So What?, Now What? Together, look back on progress to-date and decide what adjustments are needed Partendo da quanto ascoltato, seleziono quanto ritengo più rilevante Quanto selezionato ha un particolare significato per me/noi (perchè lo ritengo/ riteniamo più rilevante di altre cose?) Il fatto che abbia questo particolare significato per me/noi cosa mi/ci fa dedurre Che conclusioni traggo / traiamo da quanto ho / abbiamo dedotto Che cosa credo sia possibile fare Quali azioni posso intraprendere / che ricadute ha sul mio modo di agire, visto quello che credo WHAT? SO WHAT? NOW WHAT?

7 RIFLESSIONI E SINTESI sui temi ogge o del Corso WHAT? Partendo da quanto ascoltato, seleziono quanto ritengo più importante (almeno due argomenti): SO WHAT? Il fatto che io abbia ritenuto alcuni argomenti più importanti è perché per me hanno un particolare significato. Quindi: NOW WHAT? Quali azioni potrei pensare di intraprendere in conseguenza di quanto sopra:

8 Verifica Apprendimento (prova ECM) nome e cognome... Ar colo ogge o della valutazione metodologica... Confidence (rischio di bias - selec on, a ri on, performance, detec on, repor ng, publica on - analisi per so ogruppi, imprecisione delle s me, eterogeneità delle evidenze) Directness (aderenza delle evidenze al quesito clinico sec. P.I.C.O. - presenza di confron indire ) Relevance (adeguatezza del risultato all ipotesi di efficacia proposta dallo studio) Firma...

9 Pappagallo Cinquini - Moschetti Tutti

10 Il percorso verso la decisione terapeutica Una volta definito con chiarezza il quesito clinico sarà necessario verificare: l affidabilità delle evidenze (confidence) la diretta (o meno) trasferibilità delle evidenze disponibili alla tipologia di paziente oggetto del quesito clinico (directness) la rilevanza clinica degli effetti osservati (relevance)

11

12 Strutturazione del Quesito Clinico sec. modello P.I.C.O. P I C O Nei Pazienti con l Intervento (è suscettibile di impiego) in Confronto con riguardo agli Outcome di beneficio/danno Specifiche caratteristiche di malattia (stadio, classe di rischio, ecc.) Intervento terapeutico oggetto del quesito clinico Trattamento altrimenti considerabile in alternativa all intervento in esame Parametri clinico-laboratoristici ritenuti essenziali per la decisio-ne terapeutica

13

14 Strutturazione del Quesito Clinico sec. modello P.I.C.O. P I C O Nei Pazienti con l Intervento (è suscettibile di impiego) in Confronto con riguardo agli Outcome di beneficio/danno Specifiche caratteristiche di malattia (stadio, classe di rischio, ecc.) Intervento terapeutico oggetto del quesito clinico Trattamento altrimenti considerabile in alternativa all intervento in esame Parametri clinico-laboratoristici ritenuti essenziali per la decisio-ne terapeutica

15 Strutturazione del Quesito Clinico sec. modello P.I.C.O. P I C O Nei Pazienti con l Intervento (è suscettibile di impiego) in Confronto con riguardo agli Outcome di beneficio/danno Specifiche caratteristiche di malattia (stadio, classe di rischio, ecc.) Intervento terapeutico oggetto del quesito clinico Trattamento altrimenti considerabile in alternativa all intervento in esame Parametri clinico-laboratoristici ritenuti essenziali per la decisio-ne terapeutica

16 Strutturazione del Quesito Clinico sec. modello P.I.C.O. P I C O Nei Pazienti con l Intervento (è suscettibile di impiego) in Confronto con riguardo agli Outcome di beneficio/danno Specifiche caratteristiche di malattia (stadio, classe di rischio, ecc.) Intervento terapeutico oggetto del quesito clinico Trattamento altrimenti considerabile in alternativa all intervento in esame Parametri clinico-laboratoristici ritenuti essenziali per la decisio-ne terapeutica

17

18 Il percorso verso la decisione terapeutica Una volta definito con chiarezza il quesito clinico sarà necessario verificare: l affidabilità delle evidenze (confidence) la diretta (o meno) trasferibilità delle evidenze disponibili alla tipologia di paziente oggetto del quesito clinico (directness) la rilevanza clinica degli effetti osservati (relevance)

19 Pappagallo Cinquini - Moschetti Tutti

20

21 Direct evidence comes from research that: is conducted in the Population that we are providing answers for; includes the Intervention that we are interested in and compares these interventions with the appropriate Alternatives; measures the Outcomes in which we are interested

22

23 Strutturazione del Quesito Clinico sec. modello P.I.C.O. P Nei Pazienti con Specifiche caratteristiche di malattia (stadio, classe di rischio, ecc.) I C l Intervento (è suscettibile di impiego) in Confronto con Intervento terapeutico oggetto del quesito clinico Non necessariamente corrispondenti ai criteri di selezione delle evidenze disponibili! Trattamento altrimenti considerabile in alternativa all intervento in esame O riguardo agli Outcome di beneficio/danno Parametri clinico-laboratoristici ritenuti essenziali per la decisio-ne terapeutica

24

25

26 Patients with progressive mcrpc chemotherapynaïve and in whom clinically significant cancerrelated symptoms had not developed Docetaxel* Abiraterone** Enzalutamide*** Risk of bias No serious risk Selective crossover Selective crossover Indirectness - Population - Intervention - Comparator - Outcomes 45% symptomatic 23% visceral mets - Mitox-Pred not appropriate to date - 3% symptomatic No visceral mets - Placebo-Pred as comparator - 2% symptomatic 11% visceral mets 13% no prev. AA - Placebo as comparator - * TAX327; ** COU-302; *** PREVAIL

27 Patients with mcrpc previously treated with a docetaxel containing regimen Cabazitaxel Abiraterone Enzalutamide Risk of bias No serious risk (1 ++ ) Selective crossover (1 + ) Selective crossover (1 + ) Indirectness - Population 31% not 2nd line 45% symptomatic 25% visceral mets 30% not 2nd line 44% symptomatic 32% visceral mets 28% not 2nd line 29% symptomatic 23% visceral mets - Intervention Comparator Mitox-Pred not appropriate to date Placebo-Pred as comparator Placebo as comparator * TROPIC; ** COU-301; *** AFFIRM

28

29 Strutturazione del Quesito Clinico sec. modello P.I.C.O. P I Nei Pazienti con l Intervento Specifiche caratteristiche di malattia (stadio, classe di rischio, ecc.) Intervento terapeutico oggetto del quesito clinico C O (è suscettibile di impiego) in Confronto con Non sempre l intervento descritto nelle evidenze corrisponde all intervento oggetto del quesito clinico! riguardo agli Outcome di beneficio/danno Trattamento altrimenti considerabile in alternativa all intervento in esame Parametri clinico-laboratoristici ritenuti essenziali per la decisio-ne terapeutica

30

31

32

33

34

35

36 Strutturazione del Quesito Clinico sec. modello P.I.C.O. P I C Nei Pazienti con l Intervento (è suscettibile di impiego) in Confronto con Specifiche caratteristiche di malattia (stadio, classe di rischio, ecc.) Intervento terapeutico oggetto del quesito clinico Trattamento altrimenti considerabile in alternativa all intervento in esame O riguardo agli Outcome di beneficio/danno Parametri clinico-laboratoristici ritenuti essenziali per la decisio-ne terapeutica Non necessariamente (quasi mai!) il braccio di controllo dello studio RND di riferimento!

37 Patients with mcrpc previously treated with a docetaxel containing regimen Cabazitaxel Abiraterone Enzalutamide Risk of bias No serious risk (1 ++ ) Selective crossover (1 + ) Selective crossover (1 + ) Indirectness - Population 31% not 2nd line 45% symptomatic 25% visceral mets 30% not 2nd line 44% symptomatic 32% visceral mets 28% not 2nd line 29% symptomatic 23% visceral mets - Intervention Comparator Mitox-Pred not appropriate to date Placebo-Pred as comparator Placebo as comparator * TROPIC; ** COU-301; *** AFFIRM

38 Patients with mcrpc previously treated with a docetaxel containing regimen Cabazitaxel Abiraterone Enzalutamide Risk of bias No serious risk (1 ++ ) Selective crossover (1 + ) Selective crossover (1 + ) Indirectness - Population 31% not 2nd line 45% symptomatic 25% visceral mets 30% not 2nd line 44% symptomatic 32% visceral mets 28% not 2nd line 29% symptomatic 23% visceral mets - Intervention Comparator Mitox-Pred not appropriate to date Placebo-Pred as comparator Placebo as comparator * TROPIC; ** COU-301; *** AFFIRM

39 Pappagallo Cinquini - Moschetti Tutti

40 VALIDITA INTERNA La misura in cui uno studio riesce a cogliere la relazione «vera» fra due variabili ERRORE CASUALE ERRORE SISTEMATICO (BIAS)

41 ERRORE CASUALE Errore che si verifica per effetto del caso Replicazioni multiple della stessa misurazione producono differenti risultati in tutte le direzioni per variazioni casuali ma la media dà il risultato corretto ERRORE SISTEMATICO Errore che si verifica per la presenza di un fattore che distorce sistematicamente le osservazioni nella stessa direzione Replicazioni multiple della stessa misurazione producono risultati sempre nella stessa direzione e sbagliati

42 Errore sistematico e validità interna di uno studio I risultati di uno studio sono tanto più validi (probabilmente veri) quanto meno esso è affetto da errori sistematici Gli errori sistematici vanno previsti ed evitati o ridotti in fase di disegno dello studio

43 Bias Systematic distortion of the estimated intervention effect away from the truth, caused by inadequacies in the design, conduct, or analysis of a trial, or in the publication of its results. In other words, in a biased trial, the results observed reflect other factors in addition to (or, in extreme cases, instead of) the effect of the tested therapeutic procedure alone. Altman DG, Schulz KF, Moher D, et al. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 2001;134:

44

45

46

47 Randomizzazione

48 End of a clinical trial Why randomise? We find a difference in outcomes between intervention and control groups Possible explanations: the intervention exhibits a real effect the outcome difference is due to chance there is a systematic difference (or bias) between the groups due to factors other than the intervention Randomisation prevents the third possibility Randomisation ensures similar levels of all risk factors (known and unknown)

49 RANDOMIZATION BIAS Randomisation (coin-toss, computer) Alternate, days of week, record number Allocation schedule? Pre-vedibili Allocation Allocation Intervention Control Intervention Control

50 RANDOMIZATION BIAS Recruiting selected Randomisation (cointoss, computer) individuals due to knowledge of the next allocation Allocation schedule Allocation Manipulating allocations of people based on personal believing Intervention Control Exclusion of certain patients based on their prognosis

51 RANDOMIZATION COMPONENTS Item Sequence generation Allocation concealment Implementation Descriptor Method used to generate the random allocation sequence, including details of any restriction (eg, blocking, stratification) Method used to implement the random allocation sequence (eg, numbered containers or central telephone), clarifying whether the sequence was concealed until interventions were assigned Who generated the allocation sequence, who enrolled participants, and who assigned participants to their groups

52 Selection bias 1. generazione della sequenza di randomizzazione Adequate methods :random number table; computer random number generator; coin tossing; shuffling cards or envelopes; throwing dice. (Low risk of bias) Inadequate methods: odd or even date of birth; date (or day) of admission; hospital or clinic record number; alternation; judgement of the clinician; results of a laboratory test or a series of tests; availability of the intervention (High risk of bias). «quasi randomised studies «

53 Baron Ja et al. A Trial of Calcium and Vitamin D for the Prevention of Colorectal Adenomas. N Engl J Med Oct 15;373(16). Randomization randomization by the coordinating center was performed with the use of computergenerated random numbers with permuted blocks and stratification according to clinical center, sex, anticipated colonoscopic examination at 3 years or 5 years, and full factorial randomization.

54 Selection bias 2) Mascheramento della assegnazione Chi recluta i pazienti e verifica se rispondono ai criteri di inclusione non sa a che gruppo verranno assegnati Chi assegna i pazienti ai gruppi non sa chi sono i pazienti

55 Selection bias 2. Mascheramento della assegnazione Adequate methods: Investigators enrolling participants could not foresee assignment : central allocation (including telephone, web-based, and pharmacycontrolled, randomisation); sequentially numbered drug containers of identical appearance; sequentially numbered, opaque, sealed envelopes. Low risk of bias Inadequate methods: open random allocation schedule (e.g. a list of random numbers); assignment envelopes without appropriate safeguards (e.g. if envelopes were unsealed or nonopaque or not sequentially numbered); alternation or rotation; date of birth; case record number; any other explicitly unconcealed procedure. High risk of bias

56

57 Concelament: randomizzazione telefonica

58 Concealment: drug containers

59 Ratios of odds ratios comparing estimates of intervention effects 532 trials with inadequate or unclear allocation concealment versus 272 trials with adequate concealment Wood, L. et al. BMJ 2008;336:

60 CECITA

61 6 ragioni per introdurre la cecità Se dite al paziente che è stato randomizzato al placebo, non è contento Se dite alle persone che l efficacia del trattamento è dovuto all effetto placebo, si arrabbiano Se dite al clinico che il paziente prende il trattamento, il clinico vedrà un miglioramento (anche in assenza di cambiamento) Se dite al paziente che non si dovrebbe grattare, si gratta uguale, ma vi dice che si gratta di meno (Effetto Rosenthal) Illusione di specifici effetti come le tradizioni millenarie sono molto radicate (agopuntura nei meridiani vs a caso) Avete inventato la panacea che, ogni volta che la somministrate, fallisce miseramente cercate cercate fino a analizzare il beneficio su 100 variabili (così funziona la statistica)

62 COSA POTREBBE FARE Usually reduces differential assessment May improve compliance and retention May reduce biased supplemental care or treatment (co-intervention) [and testing]

63 Blinding or Masking Different terms to describe the same procedures Masking may be more appropriate semantics: Participants with impaired vision Less confusing with blindness an outcome Blinding conveys strong bias prevention Lasagna used the term double blindfold in 1955

64 Blinding or Masking We prefer blinding because: it rests on a long history maintains worldwide recognition If you use masking someone using PubMed in Asia or Africa may not know what you did creates strong visual imagery permeates the ICH guidelines

65 Confused Terminology of Single, Double, and Triple Blinding Permeates the Literature Physicians, textbooks, and journal articles vary greatly in interpretations and definitions [Devereaux et al. JAMA 2001; 285: ] Define double-blind inconsistently Authors frequently fail to report their definitions clearly When I use double-blind, participants, investigators, and assessors are blinded In reporting RCTs, authors should explicitly state what steps were taken to keep whom blinded

66

67

68 Performance bias (cointervention) The interpretation of a randomized controlled trial relies on the assumption that any differences in outcome are the result of either chance (whose effects can be quantified) or of inherent differences between treatments. This assumption is invalid if the treatment groups are not handled equally with regard to all of the study procedures, a part the experimental treatment

69 Performance bias Blinding of participants and providers Rischio di bias dipende dal tipo di outcome!! Low risk of bias : Blinding of participants and providers and unlikely that the blinding could have been broken No blinding or incomplete blinding, but the outcome is not likely to be influenced by lack of blinding (e.g. mortality, cancer incidence) High risk of bias: No blinding or incomplete blinding, and the outcome is likely to be influenced by lack of blinding; Blinding of key study participants and personnel attempted, but likely that the blinding could have been broken, and the outcome is likely to be influenced by lack of blinding

70

71 Detection bias When knowledge of the treatment assignment (by participants already recruited into a trial, investigators, or persons who analyze and report trial results) leads to systematic differences on the way the outcomes are assessed 71

72 Detection bias Blinding of outcome assessor Rischio di bias dipende dal tipo di outcome!! Low risk of bias : Blinding of outcome assessment ensured, and unlikely that the blinding could have been broken No blinding of outcome assessment, but the outcome measurement is not likely to be influenced by lack of blinding High risk of bias: No blinding of outcome assessment, and the outcome measurement is likely to be influenced by lack of blinding; Blinding of outcome assessment, but likely that the blinding could have been broken, and the outcome measurement is likely to be influenced by lack of blinding

73 Detection bias Blinding? Double blinding? Triple blinding? Who needs to be blinded? Is the outcome sensitive to blinding? Blinding: clearly very difficult in many intervention trials (i.e. surgical) Solution: Blinded assessors should be used routinely for measuring outcome

74 Outcome assessor Participants ( subiective outcomes) Investigator who collects outcome data Data manager Statistician Quando l intervento non può essere fatto in cieco ma l outcome è soggettivo è fondamentale cercare di garantire la cecità di chi rileva i dati Non tutela dal detection bias del paziente Non tutela dal performance bias del medico

75 Open studies (unblinded) Quando la cecità non è praticamente realizzabile (chirurgia, interventi educativi, psicosociali, riabilitazione, prevenzione primaria) Quando la cecità non è rilevante per il tipo di outcome ( mortalità, incidenza di tumore, recidiva) Risk of bias: patients might under- or overreport treatment effects and side-effects, based on their confidence on the intervention (detection bias) Providers may give advice or prescribe additional therapy to the control group if they feel that these patients are disadvantaged in comparison to the active group, (performance bias)

76 Single-blinded studies the patient should be unaware of which treatment they are taking the investigators are aware Risk of bias: Providers may give advice or prescribe additional therapy to the control group if they feel that these patients are disadvantaged in comparison to the active group( performance bias)

77 Double-blinded studies neither the patient nor the provider knows the identity of the assigned intervention the validity of the study depends on the providers and participants remaining really blinded throughout the study. A study of a drug is easily unblinded if the medications are not identical in appearance

78 Double blind - double dummy Double dummy is a technique for retaining the blind when administering supplies in a clinical trial, when the two treatments cannot be made identical. Supplies are prepared for Treatment A (active and indistinguishable placebo) and for Treatment B (active and indistinguishable placebo). Subjects then take two sets of treatment; either A (active) and B (placebo), or A (placebo) and B (active). 78

79 Triple-blinded studies Providers blinded Participants blinded All the sponsor s project team (eg, the project clinician, outcome assessor, statistician, and data manager) blinded Triple blinding is appropriate for studies in which the risk of adverse events due to the new or standard treatment is low, and should not be used for treatments where safety is a critical issue 79

80 Assessing trial blindness The degree to which the blinding was maintained in a study can be estimated by asking the patients to guess which group they were assigned to. If the mean result of the guesses is close to being 50% correct, the study was well blinded. A similar enquiry could be done with providers also. 80

81 Ratios of odds ratios comparing intervention effect estimates in 314 non-blinded trials versus 432 blinded trials. Wood, L. et al. BMJ 2008;336:

82 Allocation concealment It prevents selection bias in intervention assignment by protecting the allocation sequence before and until assignment It can always be successfully implemented regardless of the study topic Blinding It seeks to prevent performance and detection bias by protecting the sequence after assignment Not always feasible for example, in trials comparing surgical with medical interventions

83 Pappagallo Cinquini - Moschetti Tutti

84 VARIABILE DI RISPOSTA di tipo quantitativo assume uno spettro continuo di valori e viene misurata in riferimento a una scala a intervalli costanti. di tipo qualitativo esprime categorie di risposta del tipo successo / insuccesso (di un trattamento somministrato). del tipo tempo a evento rappresenta il tempo trascorso fino al verificarsi (o meno) di un evento.

85

86 VARIABILE DI RISPOSTA di tipo quantitativo assume uno spettro continuo di valori e viene misurata in riferimento a una scala a intervalli costanti. di tipo qualitativo esprime categorie di risposta del tipo successo / insuccesso (di un trattamento somministrato). del tipo tempo a evento rappresenta il tempo trascorso fino al verificarsi (o meno) di un evento.

87 Risk Difference: ovvero 8 decessi in MENO (ogni 100 pazienti trattati)

88 VARIABILE DI RISPOSTA di tipo quantitativo assume uno spettro continuo di valori e viene misurata in riferimento a una scala a intervalli costanti. di tipo qualitativo esprime categorie di risposta del tipo successo / insuccesso (di un trattamento somministrato). del tipo tempo a evento rappresenta il tempo trascorso fino al verificarsi (o meno) di un evento.

89 Indicatori riassuntivi di effetto di variabili tempo-a-evento Differenza tra stime della mediana di sopravvivenza (KM) Differenza media di sopravvivenza (restricted means) Differenza tra stime di sopravvivenza (KM) al tempo x Hazard Ratio (KM+Cox)

90 Indicatori riassuntivi di variabili tempo-a-evento Differenza tra stime della mediana di sopravvivenza (KM) Appropriato quando il rapporto tra gli Differenza media di sopravvivenza (restricted hazard dei due gruppi si mantiene means) (relativamente) costante Differenza tra stime di sopravvivenza (KM) al tempo x Hazard Ratio (KM+Cox)

91 Rapporto tra gli hazard dei due gruppi costante nel tempo Hazard Ratio è la misura di effetto più appropriata

92

93

94 Zoledronic acid + docetaxel: Failure-free survival Presented By Nicholas James at 2015 ASCO Annual Meeting

95

96 Reckamp K ORAL02.01 CheckMate 017: Updated OS Data

97 Interpretazione degli studi clinici mediante Forest (Forrest?) Plot

98 Interpretazione degli studi clinici mediante Forest (Forrest?) Plot Stima puntuale Linea di equi-efficacia (di non-effetto) favorito il trattamento sperimentale 1 favorito il trattamento di controllo

99 Interpretazione degli studi clinici mediante Forest (Forrest?) Plot Effetto osservato nel campione Linea di equi-efficacia (di non-effetto) favorito il trattamento sperimentale 1 favorito il trattamento di controllo

100 Interpretazione degli studi clinici mediante Forest (Forrest?) Plot Effetto osservato nel campione Linea di equi-efficacia (di non-effetto) Intervallo di Confidenza* * convenzionalm. 95% favorito il trattamento sperimentale 1 favorito il trattamento di controllo

101 Interpretazione degli studi clinici mediante Forest (Forrest?) Plot Effetto osservato nel campione Linea di equi-efficacia (di non-effetto) Effetto (intervallo presunto*) nella popolazione * convenzionalm. 95% favorito il trattamento sperimentale 1 favorito il trattamento di controllo

102 Statistical Vs Clinical Significance Statistical Significance Is an observed difference likely to be real dependent on the magnitude of the number of patients and/or the magnitude of the difference NOT on whether the difference is meaningful for patients

103 Interpretazione statistica di uno Studio di Superiorità Tutti e tre gli esempi indicano una differenza statisticamente significativa L estremo dx dell intervallo di confidenza NON interseca la linea di non-effetto (P<0.05)

104 Statistical Vs Clinical Significance Statistical Significance Is an observed difference likely to be real dependent on the magnitude of the number of patients and/or the magnitude of the difference NOT on whether the difference is meaningful for patients Clinical Significance Is an observed difference likely to be meaningful for patients dependent on the magnitude of the difference NOT the number of patients

105 Rilevanza Clinica Si ritiene che il trattamento in esame A abbia le potenzialità per migliorare il trattamento standard B almeno di una quantità Δ studio di superiorità A > B di una quantità Δ di interesse clinico studio di non inferiorità A < B non oltre una quantità M di rilevanza clinica

106 Interpretazione degli studi clinici mediante Forest (Forrest?) Plot Beneficio minimo di rilevanza clinica Linea di equi-efficacia (di non-effetto) Δ favorito il trattamento sperimentale 1 favorito il trattamento di controllo

107 Interpretazione clinica di uno Studio di Superiorità Effetto (sempre) clinicamente rilevante? (dato uno specifico Δ di interesse) Δ 1 Δ 1 Δ 1 RILEVANTE e (del tutto) AFFIDABILE RILEVANTE e (ragionevolmente) AFFIDABILE STATISTICAMENTE SIGNIFICATIVO

108 Beneficio minimo preordinato (Target Δ) = 0.75

109

110 Interpretazione clinica di uno Studio di Superiorità Effetto (sempre) clinicamente rilevante? (dato uno specifico Δ di interesse) Δ 1 Δ 1 Δ 1 RILEVANTE e (del tutto) AFFIDABILE RILEVANTE e (ragionevolmente) AFFIDABILE STATISTICAMENTE SIGNIFICATIVO

111

112

113 Interpretazione clinica di uno Studio di Superiorità Effetto (sempre) clinicamente rilevante? (dato uno specifico Δ di interesse) Δ 1 Δ 1 Δ 1 RILEVANTE e (del tutto) AFFIDABILE RILEVANTE e (ragionevolmente) AFFIDABILE STATISTICAMENTE SIGNIFICATIVO

114

115 Target Δ: HR erlotinib:placebo = 0.75 (2 months OS improvement) Analysis after 381 events (450 patients; α 5%, power 80%) Effetto di dimensione inferiore al beneficio minimo preordinato (Δ)

116 Target Δ: HR erlotinib:placebo = 0.75 (2 months OS improvement) Analysis after 381 events (450 patients; α 5%, power 80%) Analysis after 486 events (569 patients) Overpowering (arruolati più pazienti di quanto previsto osservati più eventi significatività statistica [P<0.05] anche in presenza di effetti non clinicamente rilevanti) Actual difference: 0.33 months (10 days)

117 Symptom Endpoints (Patient-Reported Outcomes) Blinding is often difficult Data are often missing or incomplete Clinical significance of small changes unknown Few validated instruments

118 ?

119 Probabilità che l effetto osservato (difference in mean score) sia dovuto al caso (non esprime l entità dell effetto!)

120 Rilevanza dell effetto da rapportare alla M.I.D. specifica

121 The Minimal (Clinical) Interesting Difference (M.I.D. / M.C.I.D.) Easily understood by clinicians as a key concept in the interpretability of PRO scores. The smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient's management

122

123 Rilevanza Clinica Vista la migliore tollerabilità del trattamento in esame A, si è disposti ad accettarne una eventuale minore efficacia rispetto al trattamento standard B purché questa non vada oltre un margine M studio di superiorità A > B di una quantità Δ di interesse clinico studio di non inferiorità A < B non oltre una quantità M di rilevanza clinica

124 Interpretazione degli studi clinici mediante Forest (Forrest?) Plot Linea di equi-efficacia (di non-effetto) Margine di non inferiorità favorito il trattamento sperimentale 1 M favorito il trattamento di controllo

125

126 Interpretazione clinica di uno Studio di Non-Inferiorità (dato uno specifico M di interesse) 1 M Dimostrazione di Non-Inferiorità Il limite superiore dell intervallo di confidenza non interseca la linea di noneffetto indipendentemente da dove si colloca la stima puntuale dell effetto

127

128 Interpretazione clinica di uno Studio di Non-Inferiorità (dato uno specifico M di interesse) 1 M 1 M NON Dimostrazione di Non-Inferiorità

129

130 Margine M 1.17

131 Dimostrata la non-inferiorità relativamente all endpoint primario di efficacia

132 non evidente però il necessario vantaggio di tollerabilità (riguardo all endpoint primario di efficacia)

133 è sufficiente un risparmio di eventi per 100 pazienti/anno per giustificare la scelta del nuovo farmaco?

134

135 Pappagallo Cinquini - Moschetti Tutti

136 Pappagallo Cinquini - Moschetti Tutti

137

138 Uncertainty Estimation When we measure some physical quantity with an instrument and obtain a numerical value, we want to know how close this value is to the true value. The difference between the true value and the measured value is the error. Unfortunately, the true value is unknown and unknowable. If we knew it, we wouldn t need the experiment. Since this is the case, the exact error is never known. We can only estimate it.

139 Imprecision Gli errori casuali condizionano la precisione della stima campionaria

140 imprecision Il controllo della variabilità casuale deve essere effettuato: In fase di pianificazione dello studio minima dimensione campionaria sufficiente per saggiare l ipotesi nulla (prestabilendo α) In fase di analisi - accompagnando la stima puntuale da una misura della sua variabilità casuale

141 O.J. Simpson trial

142 Errors in Trials Truth Jury Decision Innocent Guilty Innocent OK ERROR Guilty ERROR OK If O.J. is innocent, then an error was made in the civil trial. If O.J. is guilty, then an error was made in the criminal trial.

143 Errors in Hypothesis Testing Truth Decision Do not reject null Reject null Null hypothesis OK TYPE I ERROR Alternative hypothesis TYPE II ERROR OK

144 Definitions: Types of Errors Type I error: The null hypothesis is rejected when it is true. Type II error: The null hypothesis is not rejected when it is false. There is always a chance of making one of these errors. We ll want to minimize the chance of doing so!

145 imprecision Il controllo della variabilità casuale deve essere effettuato: In fase di pianificazione dello studio minima dimensione campionaria sufficiente per saggiare l ipotesi nulla (prestabilendo α) In fase di analisi - accompagnando la stima puntuale da una misura della sua variabilità casuale

146 imprecision INTERVALLO DI CONFIDENZA Lower Confidence Limit Point Estimate Width of confidence interval Upper Confidence Limit

147

148 Imprecision small sample size small number of events

149 Imprecision Dependent on the choice of the difference (Δ) you wish to detect and the resulting sample size required

150 Significatività statistica e rilevanza clinica Se con un'opportuna dimensione del campione siamo in grado di ottenere risultati significativi, ciò non ci permette ancora di capire quanto essi lo siano dal punto di vista clinico P<0.05 potrebbe includere differenze clinicamente irrilevanti P>0.05 potrebbe nascondere una differenza reale ed importante, che non è stata evidenziata a causa di una bassa potenza

151 P<0.05 potrebbe includere differenze clinicamente irrilevanti A migliore 0 B migliore

152 P 0.05 potrebbe nascondere differenze clinicamente rilevanti A migliore 0 B migliore

153 1.0 % 0

154 1.0 % 0

155 1.0 % 0

156 Δ 0

157

158 ANALISI PER SOTTOGRUPPI

159 Only one thing is worse than doing subgroup analyses--- believing the results R. Peto

160 Frequency of Subgroup Analyses Approximately 50% of reports of randomized clinical trials contain at least one subgroup analysis (Pocock et al 1987)

161 What are subgroup? An analysis of treatment effects within subgroups of patients enrolled on a clinical trial, based on baseline characteristics, who might be expected to respond to treatment differently Should all patients be given XYZ? Can/should treatment be limited to a selected group?

162 General Assumptions in Subgroup Analysis Hypotheses tested usually address an overall or average treatment effect in the study population No assumption of homogeneity of effect across subgroups - interaction Direction, not magnitude, of the treatment effect is expected be the same in subgroups

163 When multiple subgroup analyses are performed, the probability of a false positive finding can be substantial

164 Error rate as a function of number of subgroups Type I error rate Number of subgroups ODAC May 3,

165 Post-hoc analysis Unplanned analyses (exploratory) Analyses suggested by the data Exhaustive search for differential treatment effects by subgroups (data dredging) Inflated, and generally unknown, error rates

166 Inappropriate subgroup analysis can kill

167 Canadian Co-op Study Group 1978: relative odds of stroke or death in 585 TIA/stroke patients treated longterm with aspirin vs no aspirin Males Females Both Odds ratio 0.7 (95% CI 0.5 to 1.0) Aspirin better Aspirin worse

168 Impact of this result FDA did not licence aspirin for stroke prevention in women Millions of women were denied effective therapy Many avoidable strokes and deaths from vascular disease occurred

169 the question is NOT: Is the treatment effect in this subgroup statistically significantly different from zero? BUT are there any differences in the treatment effect between the various subgroups? The correct statistical procedures are either a test of heterogeneity or a test for interaction

170 Canadian Co-op Study Group 1978: relative odds of stroke or death in 585 TIA/stroke patients treated longterm with aspirin vs no aspirin Males Females Both Odds ratio 0.7 (95% CI 0.5 to 1.0) Aspirin better Aspirin worse

171 ISIS-2: aspirin vs control - effects on vascular death in 17,187 patients with acute myocardial infarction (MI) Relative risk Aspirin Control reduction Overall trial result 9.4% 11.8% 20% P <

172 When this paper was submitted to the Lancet, the editors urged the researchers to include nearly 40 subgroup analyses. The investigators reluctantly agreed, under the condition that they could provide a subgroup analysis of their own to illustrate their unreliability.

173

174

175 Author s conclusions Apparent harm in patients born under star sign of libra or gemini, with prior MI and diabetics, all most likely due to the play of chance All these subgroup analyses should, perhaps, be taken less as evidence about who benefits, than as evidence that such analyses are potentially misleading.

176 Pre-specified Subgroup Analyses Pre-specified analyses (hypothesis driven) Subgroup hypotheses specified in advance in the study protocol Control of error rates can, in principle, be addressed (statistics) - not always done

177 Pre-planned Subgroup Analyses Pre-planned analyses (hypothesis driven) Subgroup hypotheses specified in advance Control of error rates addressed (statistical analysis)

178 Control of Error Rates in Subgroup Analyses For planned subgroup analyses, the overall type I error rate can be controlled. One conservative way is to use α * = α/k in each of the subgroup analyses In this case, the power (probability of detecting real differences when present) is sharply reduced in individual subgroups For unplanned subgroup analyses, k is unknown so the error rates are unknown

179 Error Rates in Subgroup Analyses With k independent subgroups and no difference in treatments, the probability of at least one significant subgroup is: 1- (1- α) k For example, α = 0.05, k = 10 yields 1- (1-0.05) 10 = 0.40

180 Predictivity Selecting more effective drug for a specific patient HR: Breast cancer & tamoxifen HER-2 FISH: Breast cancer & trastuzumab c-kit: GIST V glivec CD-20: LNH e rituximab EGFR e K-ras: CRC V cetuximab Stratified design Test for Target EGFR status: NSCLC C TKIs ALK: crizotinib Test for Target Enrichment design Target + Target - Target + Target - Tx Control Tx Control Tx Control

181 Conclusioni Analisi pre-pianificata di sottogruppi DIMOSTRATIVA Analisi pre-specificata di sottogruppi DUBBIA Analisi post-hoc di sottogruppi SUGGESTIVA

182 Pappagallo Cinquini - Moschetti Tutti

183 Pappagallo Cinquini - Moschetti Tutti

184 Attrition bias Quando non tutti i soggetti randomizzati completano lo studio i soggetti non escono a caso dallo studio: è possibile che quelli che escono siano sistematicamente diversi da quelli che non escono: i gruppi non sono più randomizzati Validità esterna : es: escono tutti i più giovani, o i meno gravi, o i maschi: posso trarre conclusioni solo su quelli che rimangono Validità interna (Bias): se la probabilità di uscire dallo studio è legata all intervento o all outcome, cioè se quelli che escono hanno sistematicamente probabilità più alte o più basse di avere l outcome di quelli che restano

185 Attrition bias Persi al follow up: il soggetto sparisce non si hanno più info Uscito dallo studio il soggetto interrompe il trattamento ma è reperibile ( eventi avversi? Non efficace? ) Bassa compliance: il soggetto riceve il trattamento ma in dosi e modalità diverse da quelle prescritte (eventi avversi? Trattamento poco accettabile?) Missing data: misurazioni ripetute: il soggetto riceve il trattamento ma non è presente a tutte le misurazioni dell outcome (TD non consegnano le urine quando sono positive) 185

186 Low risk of bias Attrition bias Numero di persi (piccolo) ma quanto? (<5-10%) Bilanciati fra i gruppi Riportate le ragioni (non differenti fra gruppi e non attribuibili agli interventi) Intention to treat Imputation of missing data 186

187 Attrition bias Intention to treat analysis: all subjects analysed in the treatment group they were originally randomized, regardless if they actually received the assigned treatment or not Imputation of missing data : es: considerare gli usciti come fallimenti terapeutici (TD); last observation carried forward Per protocol analysis: only patients who received the treatment as described in the prtocol were analysed 187

188 Intention to treat: effectiveness ( efficacia in pratica, efficacia del trattamento prescritto) Tiene conto anche della scarsa compliance, della difficoltà a somministrare il trattamento Tutela da attrition bias (mantiene la similitudine dei gruppi ottenuta con la randomizzazione Per protocol: efficacy (efficacia in condizioni ottimali, efficacia della trattamento ricevuto nelle modalità previste) Può dare stime distorte se la non compliance e l uscita dallo studio è legata al trattamento o all outcome 188

189 Attrition bias Low risk of bias No missing outcome data; the proportion of missing outcomes compared with observed event risk not enough to have a relevant impact on the intervention effect; Missing outcome data balanced in numbers across intervention groups, with similar reasons across groups; Missing data imputed using appropriate methods All patients analysed in the group they were allocated to by randomisation irrespective of non-compliance and cointerventions (intention to treat) High risk of bias: the proportion of missing outcomes compared with observed event risk enough to induce relevant bias in intervention effect estimate Reason for missing outcome data likely to be related to true outcome, with either imbalance in numbers or reasons for missing data across intervention groups; 189

190 What is publication bias (1)? Definition Publication bias refers to the greater likelihood that studies with positive results will be published JAMA 2002;287:

191 What is publication bias (2)? An alternative definition: Publication bias is the selective or multiple publication or suppression of trial results so that the scientific record is distorted Extension: applied to trial parts - outcomes, subgroups, adverse events REPORTING BIAS The likelihood of finding studies is related to the results of those studies (positive vs negative/detrimental)

192 Why does it matter? Distorts the scientific record Hides the truth Influences doctors decision making Misleads policy makers Causes harm to patients Costly for the health service A form of scientific and research misconduct TO U: It will matter if the studies you don t find differ systematically from the ones you have found You might arrive at different answers, or even THE WRONG ANSWER

193 Publication of All Trials

194 Publication Bias Asymmetrical appearance of the funnel plot with a gap in a bottom corner of the graph

195 Funnel plots A funnel plot is a scatter plot of treatment effect against a measure of study size / precision. Precision in the estimation of the true treatment effect increases as the sample size increases. Small studies scatter more widely at the bottom of the graph In the absence of bias the plot should resemble a symmetrical inverted funnel

196 Publication Bias In this situation the effect calculated in a meta-analysis will overestimate the treatment effect The more pronounced the asymmetry, the more likely it is that the amount of bias will be substantial.

197 Outcome reporting bias

198 Reporting bias is selection bias Reporting bias is perhaps the greatest source of selection bias Originally defined as the publication or nonpublication of studies depending on the direction and statistical significance of the results Is a complex phenomenon

199 Reported outcomes Full Partial Qualitative n and effect size, plus precision / p- value for continuous data Effect size or precision (± n or p-value) p-value Incompletely reported outcomes Unreported Hierarchy of the levels of outcome reporting (Chan, 2004)

200 Eterogeneità

201 E efficace?

202 Forest plot (meta-graph) analitico

203 META-ANALYSIS General Number of studies Number of participants (62607) OR (MH) - Fixed effect model Meta-analysis outcome 95% CI lower limit 95% CI upper limit 1,0063 0,9482 1,068 z p-value (two-tailed) 0,2073 0,8358 Heterogeneity Q H p-value (two-tailed) 95% CI lower limit 95% CI upper limit 47,1363 < 0,0001 1,7727 1,3675 2,2979 I^2 95% CI lower limit 95% CI upper limit 68,18% 46,53% 81,06%

204 Could we just add the data from all the trials together? One approach to combining trials would be to add all the treatment groups together, add all the control groups together, and compare the totals This is wrong for several reasons, and it can give the wrong answer

205 If we add up the columns we get 34.3% vs 32.5%, a RR of 1.06, a higher chance of death in the steroids group From a meta-analysis, we get RR=0.96, a lower chance of death in the steroids group

206

207 Va a scua il mar Mettere insieme studi diversi che testano quesiti diversi considerando popolazione diverse usando interventi lievemente diversi ma partendo da protocolli profondamente diversi e dando risultati Eterogeneità

208 What is heterogeneity? Heterogeneity is variation between the studies results

209 What is heterogeneity? Differences between studies with respect to: Clinical heterogeneity (clinical diversity) Participants e.g. conditions under investigation, eligibility criteria for trials, geographical variation Interventions e.g. intensity / dose / duration, sub-type of drug, mode of administration, experience of practitioners, nature of the control (placebo/none/standard care) Outcomes e.g. definition of an event, follow-up duration, ways of measuring outcomes, cut-off points on scales

210 What is heterogeneity? Differences between studies with respect to: Methodological heterogeneity (methodological diversity) Design e.g. randomised vs non-randomised, crossover vs parallel group vs cluster randomised, pre-test and long follow up Conduct e.g. allocation concealment, blinding etc, approach to analysis, imputation methods for missing data

211 What is heterogeneity? What do we do if there is statistical heterogeneity? Variation in the true effects underlying the studies...which may manifest itself in more observed variation than expected by chance alone May be due to clinical diversity (different treatment effects) or methodological diversity (different biases)

212 Come si misura questa eterogeneità?

213 Confidence interval overlapping Eyeball test Cochran s Q: to assess whether observed differences in results are compatible with change alone 2 distribution; low power (small number of studies, small sample size) Q P=<0.10 p=<0.10 (heterogeneity) I 2 quantifing heterogeneity (describes the percentage of variation across studies that is due to heterogeneity rather than chance) I 2 = 20% - 50% 0-40% might not be important 30-60% may represent moderate heterogeneity 50-90% may represent substantial heterogeneity % considerable heterogeneity Tau.

214 How to deal with heterogeneity 1. Do not pool at all 2. Ignore heterogeneity: use fixed effect model 3. Allow for heterogeneity: use random effects model 4. Explore heterogeneity: subgroups analysis or metaregression (tricky)

215 Example: PC Games for intelligence Study Omar Einstein Margie Bart Ombroso Estimates and 95% confidence intervals I 2 = 0 % Standardized mean difference Favours Control Favours PC Games

216 Example: PC Games for intelligence Study Omar Einstein Margie Bart Ombroso Estimates and 95% confidence intervals I 2 = 80 % Standardized mean difference Favours Control Favours PC Games

217 Fixed and random effects

218 The Fixed Effects assumption

219 The Fixed Effects assumption Observed in studies True

220 Fixed effects model In un modello a effetti fissi si assume che tutti gli studi provengano dalla stessa popolazione di studi Si assume che ci sia un parametro (es.media) unico, fisso Il peso degli studi è funzione della variabilità intra-studio Gli intervalli di confidenza del parametro sono ridotti Popolazione di riferimento unica, omogenea

221 The Random Effects assumption

222 The Random Effects assumption Observed in studies True in studies True

223 Popolazioni di riferimento molteplici, eterogenee Random effects model In un modello a effetti random gli studi potrebbero provenire da popolazioni di studi diverse I pesi sono ridistribuiti in modo più omogeneo tra studi grandi e piccoli (il peso non è dovuto solo alla variabilità intrastudio) Gli intervalli di confidenza del parametro sono aumentati

224 Quale modello? Fixed effect Random effect

225 Quale modello? Fixed effect Random effect Potente (IC ristretti) Assume un solo parametro, non facile in ambito biomedico Più facile per sottogruppi Semplicistico Dà luogo a un aggiustamento dei pesi grezzo (ridistribuzione senza tener conto di nessuna co-variata) IC realistici I 2 = 20% - 50% I 2 = 50% - 70% I 2 = > 70%

226 Pappagallo Cinquini - Moschetti Tutti

227 Pappagallo Cinquini - Moschetti Tutti

228 Strutturazione del Quesito Clinico sec. modello P.I.C.O. P Nei Pazienti con Specifiche caratteristiche di malattia (stadio, classe di rischio, ecc.) I C l Intervento Non necessariamente coincidenti con gli outcome di efficacia delle evidenze disponibili considerare anche gli outcome di tollerabilità! (è suscettibile di impiego) in Confronto con Intervento terapeutico oggetto del quesito clinico Trattamento altrimenti considerabile in alternativa all intervento in esame O riguardo agli Outcome di beneficio/danno Parametri clinico-laboratoristici ritenuti essenziali per la decisio-ne terapeutica

229 Surrogate endpoints Issue: Quicker, less expensive, less clinically relevant endpoint or More expensive, clinically definitive endpoint? Mark Conaway, June 2006

230 Surrogate endpoints

231 Surrogate endpoints Issue: Quicker, less expensive, less clinically relevant endpoint or More expensive, clinically definitive endpoint? Hesitate to use the term "surrogate" Has a specific technical definition Mark Conaway, June 2006

232 TRIAL LEVEL CORRELATION BETWEEN EFFECTS threshold for surrogacy Burzykowski and Buyse, Pharmaceutical Statist 2006;5:173

233

234 Adjuvant RT PCa

235 Removal of Patients from Therapy for Disease Progression COU-AA-301 AFFIRM

236

237 Pappagallo Cinquini - Moschetti Tutti

238

239 The best? No head-to-head comparison

240

241

242 Population: previously untreated any age and race histologically proven NSCLC harbouring activating EGFR-mutation Intervention: EGFR-TKIs (Erlotinib, Gefitinib, Afatinib) Comparison: Platinum-based chemotherapy

243 Outcomes: PFS (whenever possible independently reviewed data) PFS in exon 19 deletion PFS in L858R mutation OS ORR (complete and/or partial and/or stable) Treatment related toxic events

244 Search strategy PubMed, Cancer-Lit, Embase-databases and Cochrane-Library were searched for RCTs up to June 2014 with no language or publication status restrictions. Search terms were TKI [Substance Name] and Carcinoma, NSCLC [Substance Name]. The proceedings of the conferences of the American Society of Clinical Oncology(ASCO), European Society of Medical Oncology (ESMO)and International Association for the Study of Lung Cancer (IASLC), World Conference of Lung Cancer were also searched for relevant abstracts. Any unpublished RCTs were considered for inclusion.

245

246

247 Data synthesis: HR for PFS and OS RR for the Others

248 PFS

249

250 OS

251 Skin reactions Diarrhea Hypertransaminasemia

252 So, who s the best?

253 Head to Head vs. Indirect Comparisons Head to Head comparison comes from a trial where A was directly compared to B. Indirect Comparison comes from multiple studies where A and B may have been compared to the same comparator (i.e., C) but have never been compared to each other in the same study, What is indirect comparison? Fujian Song BMed MMed PhD Reader in Research Synthesis, Faculty of Health, University of East Anglia

254 Indirect Comparisons Indirect comparison refers to a comparison of different healthcare interventions using data from separate studies, in contrast to a direct comparison within randomized controlled trials. Indirect comparison is often used because of a lack of, or insufficient, evidence from head-to-head comparative trials. Naive indirect comparison is a comparison of the results of individual arms from different trials as if they were from the same randomized trials. This method provides evidence equivalent to that of observational studies and should be avoided in the analysis of data from randomized trials. Adjusted indirect comparison (including mixed treatment comparison) is an indirect comparison of different treatments adjusted according to the results of their direct comparison with a common control, so that the strength of the randomized trials is preserved. Empirical evidence indicates that results of adjusted indirect comparison are usually, but not always, consistent with the results of direct comparison. What is indirect comparison? Fujian Song BMed MMed PhD Reader in Research Synthesis, Faculty of Health, University of East Anglia

255 Indirect Comparisons Basic assumptions underlying indirect comparisons include: homogeneity assumption for standard meta-analysis, similarity assumption for adjusted indirect comparison and consistency assumption for the combination of direct and indirect evidence. It is essential to fully understand and appreciate these basic assumptions in order to use adjusted indirect and mixed treatment comparisons appropriately. What is indirect comparison? Fujian Song BMed MMed PhD Reader in Research Synthesis, Faculty of Health, University of East Anglia

256 HOMOGENEITY ASSUMPTION When multiple trials are available for a given comparison, the results from multiple trials can be pooled in metaanalyses before an adjusted indirect comparison is conducted. For a meta-analysis to be valid, it is commonly established that results from different trials should be sufficiently homogeneous from a clinical and statistical perspective. This is usually demonstrated by a 2-tailed p value for homogeneity at Pearson chi-squared test or Cochran Q test > 0.10 and a I 2 (inconsistency) < 50%. When homogeneity is unlikely (e.g. I 2 >50%) than heterogeneity and inconsistency are likely. Song, What is? 2009; Higgins et al, BMJ 2003

257 PFS

258 CONSISTENCY ASSUMPTION When both direct and indirect evidence is available, an assumption of evidence consistency is required to quantitatively combine the direct and indirect estimates. It is important to investigate possible causes of discrepancy between the direct and indirect evidence, such as the play of chance, invalid indirect comparison, bias in head-to-head comparative trials, and clinically meaningful heterogeneity When the direct comparison differs from the adjusted indirect comparison, we should usually give more credibility to evidence from head-to-head comparative trials. However, evidence from direct comparative trials may not always be valid. Song, What is? 2009; Song et al, J Clin Epidemiol 2008

259 No head-to head comparisons

260 SIMILARITY ASSUMPTION For an adjusted indirect comparison (A vs B) to be valid, a similarity assumption is required in terms of moderators of relative treatment effect. That is, patients included should be sufficiently similar in the two sets of control arms (C 1 from the trial comparing A vs C 1, and C 2, from the trial comparing B vs C 2 ). This is crucial as only a large theoretical overlap between patients enrolled in C 1 and C 2 enables the relative effect estimated by trials of A versus C 1 to be generalizable to patients in trials of B versus C 1, and the relative effect estimated by trials of B versus C 2 to be generalizable to patients in trials of A versus C 2. Song, What is? 2009

261 Study FIRST-SIGNAL Cisplatin 75 mg/m2 day 1&8 Gemcitabine 1,250 mg/m2 day 1 IPASS Carboplatin (AUC 5.0/6.0) mg/millimeter per minutes Paclitaxel 200mg/m2 day 1 NEJG002 WJTOG3405 EURTAC OPTIMAL Carboplatin(AUC 6.0)mgmm Paclitaxel 200mg/m2 day 1 Cisplatin 80 mg/m2 Docetaxel 60mg/m2 Cisplatin 75 mg/m2 or Carbo Docetaxel 75mg/m2 day 1 or Gemcitabine 1250 day 1&8 Carboplatin(AUC 5.0)mgmm Gemcitabine 1000 mg/m2 day 1&8 TORCH Cisplatin 80 mg/m2 day 1 Gemcitabine 1,200 mg/m2 LUX-LUNG III LUX-LUNG VI Cisplatin 75 mg/m2 Pemetrexed 500mg/m2 Cisplatin 75 mg/m2 Gemcitabine 1000 mg/m2 day 1&8 i.v. every 3 weeks Max 9 cycles i.v. every 3 weeks up to 6 weeks i.v. 3 cycles i.v. every 3 weeks up to 6 weeks i.v. 4 cycles i.v. every 3 weeks up to 6 weeks i.v. 6 cycles i.v. Up to 6 cycles

262 COMPUTATIONS The log relative risk of the adjusted indirect comparison of A and B (lnrr A vs B ) can be estimated by: and its standard error is: ln RR A vs B = ln RR A vs C1 ln RR B vs C2 SE ( ln RR A vs B ) = [ SE ( ln RR A vs C1 ) 2 + SE ( ln RR B vs C2 ) 2 ] Similar computations can be envisioned for odds ratio, absolute risk reductions, weighted mean differences, and standardized mean differences. Higgins et al, BMJ 2003; Song, What is? 2009;

263

264 Interpretation - Quality Rubbish studies = unbelievable results If all the trials in a meta-analysis were of very low quality, then you should be less certain of your conclusions. Instead of Treatment X cures Y disease, try There is some evidence that Treatment X cures Y disease, but the data should be interpreted with caution.

265