Neyman Construction. θ s true. pdf f (x θ) is known for each prospectiveθ generate x construct an int erval in DATA phase space.

Transcript

1 Neyman Construction θ s true x s measured pdf f (x θ) is known for each prospectiveθ generate x construct an int erval in DATA phase space Interval = x h xl repeat for each θ f (x θ)dx = 68% 158!60 Figure from K Cranmer Use the Confidence belt to construct the CI = [θ 1,θ 2 ] ( for a given x obs ) in θ phase space 16/10/18

2 Confidence intervals in practice The recipe to find the interval [a, b] boils down to solving 159 a is hypothetical value of θ such that b is hypothetical value of θ such that 18/10/18

3 Meaning of a confidence interval /10/18

4 Neyman s construction µ 1 (x 0 ) x 1 (µ) µ 2 (x 0 ) x 2 (µ) µ* x 0 x 2 1 ( µ ) f ( x / µ ) dx = α x ( µ ) x 1 (µ*) x 0 x 2 (µ*) Coverage: suppose µ* the true value P(x 1 (µ * ) < x 0 < x 2 (µ * )) = α 161

5 Neyman s construction µ 1 (x 0 ) x 1 (µ) µ 2 (x 0 ) x 2 (µ) µ* x 0 x 2 1 ( µ ) f ( x / µ ) dx = α x ( µ ) x 0 x 1 (µ*) x 2 (µ*) Coverage: suppose µ* the true value P(x 1 (µ * ) < x 0 < x 2 (µ * )) = α 162

6 Logic of an EPP experiment - VI End of the selection: CANDIDATES sample N cand Which relation is there between N cand and N X? Efficiency: not all searched final states are selected and go to the candidates sample.(trigger efficiencies are particularly delicate to treat.) Efficiency includes also the acceptance. Background: few other final states are faking good ones and go in the candidates sample. εn X = N cand N b where: ε = efficiency (0<ε<1); ε = A ε d N b = number of background events Estimate ε and N b is a crucial work for the experimentalist and can be done either using simulation (this is tipically done before the experiment and updated later) or using data themselves. 163

7 Binomial distribution Consider N independent experiments (Bernoulli trials): outcome of each is success or failure, probability of success on any given trial is p. Define discrete r.v. n = number of successes (0 n N). Probability of a specific outcome (in order), e.g. ssfsf is But order not important; there are ways (permutations) to get n successes in N trials, total probability for n is sum of probabilities for each permutation. 164

8 Binomial distribution (2) The binomial distribution is therefore random variable parameters For the expectation value and variance we find: 165

9 Binomial distribution (3) Binomial distribution for several values of the parameters: 166 Example: observe N decays of W ±, the number n of which are W µν is a binomial r.v., p = branching ratio.

10 Multinomial distribution Like binomial but now m outcomes instead of two, probabilities are For N trials we want the probability to obtain: n 1 of outcome 1, n 2 of outcome 2, n m of outcome m. This is the multinomial distribution for 167

11 Multinomial distribution (2) Now consider outcome i as success, all others as failure. all n i individually binomial with parameters N, p i for all i One can also find the covariance to be Example: represents a histogram with m bins, N total entries, all entries independent. 168

12 Poisson distribution Consider binomial n in the limit n follows the Poisson distribution: Example: number of scattering events n with cross section σ found for a fixed integrated luminosity, with 169

13 170

14 171

15 From Binomial to Poisson to Gaussian P(k :n, p) = n k pk (1 p) n k n,np=λ P(k :n, p) Poiss(k;λ) = λ k e k k! k = λ, σ k = k x = k λ Using Stirling Formula prob(x)=g(x,σ = λ ) = 1 2πσ e (x λ ) 2 /2σ 2 This is a Gaussian, or Normal distribution with mean and variance of λ 172

16 Histograms N collisions p(higgs event) = Lσ (pp H ) Aε ff Lσ (pp) Pr ob to see n H obs in N collisions is P(n H obs ) = N n H obs p n obs H (1 p) N n H obs lim N P(n obs H ) = Poiss(n obs H,λ) = e λ λ n obs H n H obs! mass λ = Np = Lσ (pp) Lσ (pp H ) Aε ff Lσ (pp) = n H exp 173

17 Histograms pdf = histogram with infinite data sample, zero bin width, normalized to unit area. 174

18 Uniform distribution Consider a continuous r.v. x with < x <. Uniform pdf is: N.B. For any r.v. x with cumulative distribution F(x), y = F(x) is uniform in [0,1]. Example: for π 0 γγ, E γ is uniform in [E min, E max ], with 175

19 Exponential distribution The exponential pdf for the continuous r.v. x is defined by: Example: proper decay time t of an unstable particle (τ = mean lifetime) Lack of memory (unique to exponential): 176

20 Gaussian distribution The Gaussian (normal) pdf for a continuous r.v. x is defined by: (N.B. often µ, σ 2 denote mean, variance of any r.v., not only Gaussian.) Special case: µ = 0, σ 2 = 1 ( standard Gaussian ): If y ~ Gaussian with µ, σ 2, then x = (y µ) /σ follows φ(x). 177

21 Gaussian pdf and the Central Limit Theorem The Gaussian pdf is so useful because almost any random variable that is a sum of a large number of small contributions follows it. This follows from the Central Limit Theorem: For n independent r.v.s x i with finite variances σ i2, otherwise arbitrary pdfs, consider the sum In the limit n, y is a Gaussian r.v. with Measurement errors are often the sum of many contributions, so frequently measured values can be treated as Gaussian r.v.s. 178

22 Quantities to measure In order to estimate N X we need to measure: N cand ε N b We already know that each of these variables have a fluctuation model: N cand is described by a Poisson process ε is described by a Bernoulli process N b 179

23 N cand : a Poisson variable If events come in a random way (without any time structure) the event count N is a Poisson variable.! if I count N, the best estimate of λ is N itself and the uncertainty is N If N is large enough (N>20) Poisson " Gaussian.! N± N is a 68% probability interval for N. If N is small (close to 0) the Gaussian limit is not ok, a specific treatment is required (see later in the course) /10/18

24 N cand : a Poisson variable If events come in a random way (without any time structure) the event count N is a Poisson variable.! if I count N, the best estimate of λ is N itself ( or better N +1) and the uncertainty is N P(N, λ) = λ N e λ N! P(λ N) = λ N e λ N! [ ] = N +1 [ ] = N +1 E λ var λ If N is large enough (N>20) Poisson " Gaussian.! N± N is a 68% probability interval for N. If N is small (close to 0) the Gaussian limit is not ok, a specific treatment is required (see later in the course). 181

25 Efficiency: a binomial variable - I Bernoulli process: success/failure N proofs, 0<n<N, p = success probability. p == ε P(n / N, p) = ( n N )p n (1 p) N n [ ] = Np [ ] = Np(1 p) E n var n Inference: given n and N which is the best estimate of p? And its uncertainty? (see previous lectures) ε = ˆp = n +1 N + 2 ( ) = σ ( n ) σ ε N = 1 N + 2 ˆp(1 ˆp) /10/18

26 Efficiency: a binomial variable - I Bernoulli process: success/failure N proofs, 0<n<N, p = success probability. p == ε P(n / N, p) = ( n N )p n (1 p) N n [ ] = Np [ ] = Np(1 p) E n var n Inference: given n and N which is the best estimate of p? And its uncertainty? (see previous lectures) ε = ˆp = n N ( ) = σ ( n ) σ ε N = 1 N ˆp(1 ˆp) /10/18

27 Efficiency: a binomial variable - II How measure it? From data: Sample of N true particles and I measure how many, out of these give rise to a signal in my detector From MC: I generate N gen signal events. If I select N sel of these events out of N gen, the efficiency is (assume N gen and N sel large numbers): ε = N sel N gen ( ) = σ ( N sel ) σ ε N gen = 1 N gen N sel N gen " 1 N % sel $ # N ' gen & /10/18

28 Background N b Simulation of N gen bad final states ; N sel are selected. What about N b? We define the rejection factor R = N gen /N sel > 1 We also need a correct normalization in this case: we need to know N exp = total number of expected bad final states in our sample (N exp related to luminosity and cross-section). N N b = N exp sel = N exp N gen R σ (N b ) = σ (N sel ) N exp N gen = N sel N exp N gen = N exp RN gen /10/18

29 Statistical Errors In alla cases there is an unreducible error on N X given by limited statistics. It is a random error, coming from the procedure of sampling that is intrinsic in our experiments. In all cases increasing the statistics, the error decreases σ (N cand ) N cand = σ (ε) 1 N gen 1 N cand σ (N b ) 1 N gen /10/18