Confidence Intervals & Duality

Let $\{P_\theta : \theta \in \Theta\}$ be a parametric family. A $(1-\alpha)$-confidence procedure for $\theta$ is a data-measurable set-valued map $C : \mathcal{X} \to 2^\Theta$ — usually written $X \mapsto C(X)$ — satisfying

$P_\theta(\theta \in C(X)) \ge 1 - \alpha \qquad \text{for every } \theta \in \Theta.$

When $C(X) = [L(X), U(X)]$ is an interval, we call it a $(1-\alpha)$ confidence interval. The probability $P_\theta(\theta \in C(X))$ is the coverage of the procedure at parameter value $\theta$.

The quantity $1 - \alpha$ is the nominal coverage — the level the procedure advertises. The function $\theta \mapsto P_\theta(\theta \in C(X))$ is the actual coverage, and it depends on both $\theta$ and the sample size $n$.

A procedure is exact if actual coverage equals nominal at every $\theta$; anti-conservative (or liberal) if actual coverage is below nominal at some $\theta$; conservative if actual coverage is above nominal at every $\theta$.

The three adjectives name the three failure modes — and the coverage calibration of §19.8 is the diagnostic for which mode a given procedure falls into.

The confidence-interval concept is due to formalML: . Neyman was reacting to Fisher’s fiducial distribution framework, which had produced confusing results in multi-parameter problems. His innovation was to strip probability statements of all posterior-like interpretation and define them purely as frequency properties of the procedure. This is why the probability $P_\theta(\theta \in C(X))$ in Definition 1 is indexed by a fixed $\theta$ — the randomness sits entirely in $X$ and therefore in $C(X)$. The true parameter $\theta$ is a constant, not a random variable.

The philosophical move was radical enough that it took decades to settle into standard teaching. Fiducial intervals lingered in some applied literature until the 1960s. Bayesian credible intervals, built from a genuine posterior distribution over $\theta$, solve the “what is the probability” question differently and are the subject of Track 7 — not a competing frequentist procedure but a different inference paradigm (Rem 3).

Read Definition 1 again. The probability statement is $P_\theta(\theta \in C(X)) \ge 1 - \alpha$, where the subscript tells us that $\theta$ is held fixed and $X$ is the random variable. So the event $\{\theta \in C(X)\}$ is an event about $X$ (whether the random interval catches the fixed true value), not an event about $\theta$.

Once the data are in hand — say $C(X) = [0.20, 0.34]$ — the parameter $\theta$ either lies in $[0.20, 0.34]$ or it doesn’t. The frequentist framework does not assign a probability to that specific fact. What it says is: “The procedure I just used generates intervals that catch the true parameter 95 times out of 100 on repeated experiments.” That’s a guarantee about the procedure’s long-run error rate, not a posterior probability on any one output.

A careful locution: “I am 95% confident that $\theta$ lies in $[0.20, 0.34]$” is a statement about the procedure’s reliability, not about the probability of this one interval being right. Every introductory statistics textbook says this at least once; the fact that practitioners continue to slip into the posterior-probability reading anyway is why we belabor the point here. Every result in the rest of this topic lives or dies with this distinction — in particular, the coverage diagnostics of §19.8 are meaningful only if “coverage” means the procedure’s long-run error rate.

A Bayesian credible interval starts from a posterior distribution $\pi(\theta \mid X)$ — the distribution of $\theta$ given the observed data — and defines a $(1-\alpha)$ credible interval as any set $C$ with $\pi(\theta \in C \mid X) = 1 - \alpha$. Here $\theta$ is treated as a random variable (with a prior that gets updated to a posterior), so the posterior probability of a specific interval containing $\theta$ makes sense directly.

Frequentist coverage and Bayesian credibility answer different questions about different objects. A frequentist CI guarantees a long-run error rate over repeated experiments but says nothing about this specific dataset’s $\theta$. A Bayesian credible interval gives a probability for this specific dataset’s $\theta$ but depends on the chosen prior (with different priors giving different intervals). Neither is “right” — they’re answering different questions — but confusing them is the source of the “95% probability” trap in Remark 2.

Under a flat (improper) prior for a Normal mean with known variance, the two intervals coincide numerically: $\bar X \pm z_{\alpha/2}\sigma/\sqrt n$ is both the 95% frequentist CI and the 95% credible interval. This numerical coincidence is the reason the confusion is so persistent — and why the distinction matters more the further one moves from the symmetric Normal case. Topic 25 develops the Bayesian perspective, including credible intervals and the flat-prior coincidence with z-CIs for Normal means; here we stay frequentist.

Fix $\alpha \in (0, 1)$ and a parametric family $\{P_\theta : \theta \in \Theta\}$. Suppose that for every $\theta_0 \in \Theta$ we have a level-$\alpha$ test $\varphi_{\theta_0}(X) \in \{0, 1\}$ of $H_0: \theta = \theta_0$ (reject iff $\varphi_{\theta_0}(X) = 1$), so that $P_{\theta_0}(\varphi_{\theta_0}(X) = 1) \le \alpha$. Define

$C(X) = \{\theta_0 \in \Theta : \varphi_{\theta_0}(X) = 0\}.$

Then $C(X)$ is a $(1-\alpha)$ confidence set for $\theta$: $P_\theta(\theta \in C(X)) \ge 1 - \alpha$ for every $\theta \in \Theta$.

Conversely, given a $(1-\alpha)$ confidence set $C(X)$, the collection $\{\varphi_{\theta_0}(X) = \mathbf{1}\{\theta_0 \notin C(X)\} : \theta_0 \in \Theta\}$ is a family of level-$\alpha$ tests.

Let $X_1, \ldots, X_n \sim \mathcal{N}(\mu, \sigma^2)$ with $\sigma$ known. The two-sided z-test of $H_0: \mu = \mu_0$ at level $\alpha$ rejects iff $|\sqrt n (\bar X - \mu_0)/\sigma| > z_{\alpha/2}$. By Theorem 1, the CI is the set of null values the test does not reject:

$C(X) = \{\mu_0 : |\sqrt n (\bar X - \mu_0)/\sigma| \le z_{\alpha/2}\} = [\bar X - z_{\alpha/2}\sigma/\sqrt n,\; \bar X + z_{\alpha/2}\sigma/\sqrt n].$

This is the textbook z-interval. The duality makes it an automatic consequence of the z-test’s acceptance region.

Same setup but $\sigma$ unknown; replace $\sigma$ by the sample standard deviation $S$. The two-sided t-test rejects iff $|\sqrt n (\bar X - \mu_0)/S| > t_{n-1, \alpha/2}$. Inverting:

$C(X) = [\bar X - t_{n-1, \alpha/2} S/\sqrt n,\; \bar X + t_{n-1, \alpha/2} S/\sqrt n].$

Exactness — meaning the CI has coverage exactly $1-\alpha$ at every $\mu$ — is inherited from the exactness of the t-test, which comes from formalML: : the pivot $\sqrt n (\bar X - \mu)/S$ is distribution-free because $\bar X$ and $S^2$ are independent. Same Basu in §19.3.

Every CI construction in this topic is a test inversion. The z-CI, t-CI, χ²-CI, F-CI of §19.3 come from the four pivotal tests. The Wald/Score/LRT CIs of §19.4 come from inverting the Topic 18 asymptotic trio. The Wilson interval of §19.5 is the score-test inversion for binomial $p$. The Clopper–Pearson interval of §19.6 is the exact test inversion via the beta–binomial CDF identity. The profile-likelihood CI of §19.7 is the generalized-LRT inversion with Wilks as the asymptotic engine. The TOST equivalence procedure of §19.9 is two one-sided tests run in parallel. Every time we write down a confidence interval, we are cashing in the duality theorem — and the fact that it is the same theorem in every case is the topic’s unifying thread.

Theorem 1 extends without change to vector $\theta \in \mathbb{R}^k$: a level-$\alpha$ test at every $\theta_0$ yields a $(1-\alpha)$ confidence set (not necessarily an interval; possibly a region in $\mathbb{R}^k$). The Wald CI for a vector GLM coefficient is an ellipsoid; the profile-likelihood CI for a vector parameter with nuisance components profiled out is a region whose boundary is the $\chi^2_k$-threshold contour. Simultaneous CIs for multiple parameters and confidence ellipsoids for Hotelling’s $T^2$ are developed in formalML: ; the scalar case of Topic 19 captures every main idea, with the vector-case bookkeeping involving matrix Fisher information in place of scalar $I(\theta_0)$.

A pivot for a parameter $\theta$ is a random function $Q(X, \theta)$ of data $X$ and parameter $\theta$ whose distribution does not depend on $\theta$ — that is, the law of $Q(X, \theta)$ is the same under every $P_\theta$.

Given a pivot with known distribution $F_Q$ and quantiles $q_{\alpha/2}, q_{1-\alpha/2}$, a $(1-\alpha)$ CI for $\theta$ is obtained by solving

$\{\theta : q_{\alpha/2} \le Q(X, \theta) \le q_{1-\alpha/2}\}$

for $\theta$. The inversion is algebra; the probability content is packed into the pivot.

For iid $\mathcal{N}(\mu, \sigma^2)$ data with $\sigma$ known, $Q(X, \mu) = \sqrt n (\bar X - \mu)/\sigma$ is a pivot with $Q \sim \mathcal{N}(0, 1)$. Inverting $-z_{\alpha/2} \le Q \le z_{\alpha/2}$ gives $\bar X - z_{\alpha/2}\sigma/\sqrt n \le \mu \le \bar X + z_{\alpha/2}\sigma/\sqrt n$ — the z-CI of Example 1.

For $\sigma$ unknown, $Q(X, \mu) = \sqrt n (\bar X - \mu)/S$ is a pivot with $Q \sim t_{n-1}$ — this is where Basu’s independence theorem enters: the distribution of $\sqrt n (\bar X - \mu)/S$ is free of both $\mu$ and $\sigma$ precisely because $\bar X \perp\!\!\!\perp S^2$ under normality. Inversion gives the t-CI of Example 2. Worked numerically in formalML: .

For iid $\mathcal{N}(\mu, \sigma^2)$ data with $\mu$ unknown, $Q(X, \sigma^2) = (n-1)S^2/\sigma^2$ is a pivot with $Q \sim \chi^2_{n-1}$. Invert $\chi^2_{n-1, \alpha/2} \le Q \le \chi^2_{n-1, 1-\alpha/2}$:

$\frac{(n-1)S^2}{\chi^2_{n-1, 1-\alpha/2}} \le \sigma^2 \le \frac{(n-1)S^2}{\chi^2_{n-1, \alpha/2}}.$

The interval is asymmetric in $\sigma^2$ — the larger quantile appears in the denominator of the lower endpoint — a direct consequence of the χ²’s skew. Unlike the z/t intervals, there is no “plus or minus” formulation; the asymmetry is real and reflects the distributional shape.

For two independent Normal samples with sample variances $S_1^2, S_2^2$ and sizes $n_1, n_2$, the ratio $(S_1^2/\sigma_1^2)/(S_2^2/\sigma_2^2)$ is distributed as $F_{n_1-1, n_2-1}$ — a pivot for $\sigma_1^2/\sigma_2^2$. Inverting gives a CI for the variance ratio:

$\frac{S_1^2/S_2^2}{F_{n_1-1, n_2-1, 1-\alpha/2}} \le \frac{\sigma_1^2}{\sigma_2^2} \le \frac{S_1^2/S_2^2}{F_{n_1-1, n_2-1, \alpha/2}}.$

This is the two-sample variance-ratio CI — the standard tool for testing equality of Normal variances and, by inversion, for quantifying how different they might be.

The z, t, χ², and F pivots exhaust essentially every classical example of exact small-sample CIs. For non-Normal families — Bernoulli, Poisson, Exponential, Gamma, Weibull, and every GLM — no exact pivot exists for the parameter of interest. That is why the rest of Topic 19 develops the asymptotic and exact-by-inversion constructions: Wald, Score, LRT, Wilson, Clopper–Pearson, profile likelihood. All of them are test inversions via Theorem 1, not pivot manipulations — and test inversion is the general-purpose tool. Pivots are the special case where the algebra gives a closed form.

For a regular parametric family with scalar $\theta$, MLE $\hat\theta_n$, and Fisher information $I(\theta)$, the Wald CI at level $1-\alpha$ is

$C_{\rm Wald}(X) = \left[\hat\theta_n - \frac{z_{\alpha/2}}{\sqrt{n\,I(\hat\theta_n)}},\; \hat\theta_n + \frac{z_{\alpha/2}}{\sqrt{n\,I(\hat\theta_n)}}\right].$

Equivalent description: invert the Wald test $W_n(\theta_0) = n(\hat\theta_n - \theta_0)^2 I(\hat\theta_n) \le z^2_{\alpha/2}$. The interval is symmetric around $\hat\theta_n$ — the quadratic approximation to the log-likelihood at the MLE imposes symmetry regardless of the underlying likelihood’s actual shape.

The Score CI is the set of null values the score (Rao) test does not reject:

$C_{\rm Score}(X) = \{\theta_0 : S_n(\theta_0) \le z^2_{\alpha/2}\}, \qquad S_n(\theta_0) = \frac{U_n(\theta_0)^2}{n\,I(\theta_0)},$

where $U_n(\theta_0) = \partial \ell_n(\theta_0)/\partial \theta$ is the score at the null. Because the variance $I(\theta_0)$ is evaluated at the null, the resulting endpoints solve a quadratic-in-$\theta_0$ inequality — generally asymmetric around $\hat\theta_n$, and always within the parameter space for Bernoulli (§19.5).

The likelihood-ratio confidence interval is the set of null values the LRT does not reject:

$C_{\rm LRT}(X) = \{\theta_0 : -2\log\Lambda_n(\theta_0) \le \chi^2_{1, 1-\alpha}\},$

where $-2\log\Lambda_n(\theta_0) = 2[\ell_n(\hat\theta_n) - \ell_n(\theta_0)]$. Endpoint computation is a bisection on the log-likelihood’s χ²-threshold contour around the MLE — asymmetric whenever the log-likelihood is non-quadratic, which is the generic case at moderate $n$.

For a regular parametric family with scalar $\theta$ and Fisher information $I(\theta)$ continuous in $\theta$, each of $C_{\rm Wald}, C_{\rm Score}, C_{\rm LRT}$ has asymptotic coverage $1 - \alpha$:

$\lim_{n\to\infty} P_{\theta_0}(\theta_0 \in C_\bullet(X)) = 1 - \alpha, \qquad \bullet \in \{\rm Wald, Score, LRT\}.$

Proof. Each CI is the non-rejection region of a test with asymptotic $\chi^2_1$ null distribution. By Topic 18 §18.7 Thm 5, the Wald/Score/LRT statistics each converge in distribution to $\chi^2_1$ under $H_0: \theta = \theta_0$. By the continuous mapping theorem, $P_{\theta_0}(\text{statistic} \le \chi^2_{1, 1-\alpha}) \to 1 - \alpha$. Hence coverage converges to $1 - \alpha$ by Theorem 1 applied at level $\alpha$. ∎

For iid Bernoulli$(p)$ with $\hat p = 0.3$, $n = 50$, at $\alpha = 0.05$:

• Wald: $\hat p \pm z_{0.025}\sqrt{\hat p(1-\hat p)/n} = 0.3 \pm 1.96 \cdot 0.0648 = [0.173, 0.427]$.
• Score (Wilson): closed-form quadratic inversion (§19.5 Proof 2); here $[0.191, 0.438]$.
• LRT: bisection on $-2\log\Lambda_n(p_0) = \chi^2_{1, 0.95}$; here $[0.185, 0.435]$.

All three are close — agreement to ≈ 0.02 in each endpoint — because $n = 50$ is moderate and $\hat p = 0.3$ is not near a boundary. Agreement deteriorates as $n \to 20$ or $\hat p \to 0$; §19.5 and §19.8 quantify.

For iid Poisson$(\lambda)$ with $\hat\lambda = 2$, $n = 30$, $\alpha = 0.05$:

• Wald: $\hat\lambda \pm z_{0.025}\sqrt{\hat\lambda/n} = 2 \pm 1.96 \cdot 0.258 = [1.494, 2.506]$.
• Score: invert $n(\hat\lambda - \lambda_0)^2/\lambda_0 = z^2$; quadratic in $\lambda_0$ gives $[1.527, 2.572]$.
• LRT: bisection on $2n[\hat\lambda\log(\hat\lambda/\lambda_0) - (\hat\lambda - \lambda_0)] = \chi^2_{1, 0.95}$; here $[1.529, 2.572]$.

Score and LRT agree to three decimals here; Wald is visibly different because the Poisson log-likelihood’s curvature at $\hat\lambda = 2$ differs from the quadratic approximation.

The Cramér-Rao lower bound from formalML: says that every unbiased estimator $\hat\theta_n$ satisfies $\text{Var}_\theta(\hat\theta_n) \ge 1/(n I(\theta))$. Dualized: no Wald-type CI can have width less than $2 z_{\alpha/2} / \sqrt{n I(\theta)}$ at its leading-order rate. The CRLB is thus the asymptotic width envelope for confidence intervals — the same Fisher information that bounds estimator variance bounds CI width. All three asymptotic CIs of §19.4 achieve this envelope to leading order; the finite-sample corrections are where they differ.

Topic 18 §18.8 showed that Wald, Score, and LRT differ finite-sample, with the Wald test sensitive to reparameterization: a logit transform and its Wald CI back-transform do not equal the $p$-scale Wald CI. The same is true for Wald CIs; the LRT CI, by contrast, is invariant — $\{\theta_0 : -2\log\Lambda_n(\theta_0) \le \chi^2_{1, 1-\alpha}\}$ transforms to $\{g(\theta_0) : -2\log\Lambda_n(g(\theta_0)) \le \chi^2_{1, 1-\alpha}\}$ under a bijection $g$, because $-2\log\Lambda_n$ depends only on the likelihood values, not on how $\theta$ is parameterized. This is why production GLM libraries default to LRT (aka “deviance”) CIs for coefficients whose null effect is at a boundary (logistic regression on rare events, Poisson on small counts); the reparameterization-dependent Wald CI gives qualitatively different answers depending on the scale chosen. formalML: has the proof for the test-statistic version; dualization via Theorem 1 transfers it verbatim to CIs.

Let $X_1, \ldots, X_n$ be iid Bernoulli$(p)$ with $\hat p_n = \bar X_n$. The asymptotic level-$\alpha$ score test of $H_0: p = p_0$ rejects iff $|Z_n(p_0)| > z_{\alpha/2}$, where $Z_n(p_0) = \sqrt n (\hat p_n - p_0)/\sqrt{p_0(1-p_0)}$. The test-inversion CI is the Wilson interval

$C_{\rm Wilson}(X) = \frac{\hat p_n + \frac{z^2}{2n} \pm z\sqrt{\frac{\hat p_n(1-\hat p_n)}{n} + \frac{z^2}{4n^2}}}{1 + z^2/n},$

where $z = z_{\alpha/2}$.

At $n = 20, x = 0, \alpha = 0.05$: $\hat p_n = 0$, $z = 1.96$.

• Wald: $[0, 0]$ — a point, coverage 0 at every $p > 0$.
• Wilson: center $= (0 + 1.96^2/40)/(1 + 1.96^2/20) = 0.0805$; half-width $= 1.96\sqrt{0 + 1.96^2/1600}/(1.192) = 0.0805$. Lower $= \max(0, 0.0805 - 0.0805) = 0$; upper $= 0.0805 + 0.0805 \cdot \text{(corrected)} \approx 0.161$. Proper interval.
• Clopper–Pearson (next section): $[0, 0.168]$.

The Wilson and Clopper–Pearson upper endpoints agree to $\approx 5\%$; Wald is catastrophic. This is the concrete content of Topic 18 §18.8 Rem 16: for rare-event A/B tests, Wald under-covers at the boundary, and Wilson is the practical default.

formalML: proposed an easy-to-remember approximation to Wilson: add $z^2/2 \approx 2$ successes and $z^2/2 \approx 2$ failures to the observed counts, then apply the Wald formula to the inflated sample. At $\alpha = 0.05$, $z^2/2 \approx 1.92 \approx 2$, so the popular form is “add 2 successes, add 2 failures, Wald the result.” The resulting interval usually matches Wilson within $0.01$ and is a reasonable hand-calculator substitute. The pedagogical slogan — approximate is better than exact for interval estimation of binomial proportions — captures the main takeaway of BRO2001 in five words.

formalML: gave the systematic diagnostic for binomial CI coverage: actual coverage as a function of true $p$ across a grid of $n$. Their Table 1 is the authority for test cases (§19.8 Ex 12); their key finding is that Wald’s actual coverage oscillates around $1-\alpha$ but dips below nominal — sometimes by $0.1$ or more — at moderate $p$ and small $n$. Wilson’s coverage oscillates around $1-\alpha$ with much smaller amplitude; Clopper–Pearson is always at or above nominal (conservative) but often by $0.03$ or more. The paper’s recommendation for practical work: Wilson as default; Agresti–Coull as hand-calculator substitute; Clopper–Pearson only when a strict lower bound on coverage matters (regulatory submissions, conservative monitoring).

Let $X \sim \text{Binomial}(n, p)$ with observed value $x \in \{0, 1, \ldots, n\}$. The Clopper–Pearson $(1-\alpha)$ confidence interval for $p$ is

$C_{\rm CP}(x) = [p_L, p_U], \qquad p_L = B^{-1}(\alpha/2;\, x, n-x+1), \qquad p_U = B^{-1}(1-\alpha/2;\, x+1, n-x),$

where $B^{-1}(q; a, b)$ is the $q$-quantile of the Beta$(a, b)$ distribution, with the conventions $p_L = 0$ when $x = 0$ and $p_U = 1$ when $x = n$. Coverage $P_p(p \in C_{\rm CP}(X)) \ge 1 - \alpha$ for every $p \in [0, 1]$.

At $\alpha = 0.05$:

• $p_L = B^{-1}(0.025;\, 3, 18) \approx 0.032$,
• $p_U = B^{-1}(0.975;\, 4, 17) \approx 0.379$.

So the 95% Clopper–Pearson CI is $[0.032, 0.379]$. The Wilson CI at the same $(n, x, \alpha)$ is $[0.054, 0.360]$; Wald is $[-0.007, 0.307]$ before clamping — the lower endpoint is negative, reflecting the same pathology as Example 8 at a less extreme level. Clopper–Pearson’s conservatism shows up as the widest of the three intervals: it buys guaranteed coverage at the cost of width. Worked numerically in formalML: .

A discrete CDF cannot assign mass exactly $\alpha/2$ to a tail unless $\alpha/2$ happens to coincide with a cumulative PMF value at some integer cutoff. Generically it doesn’t, so the exact tail test must reject only when the cumulative mass is at or below $\alpha/2$, which gives actual size strictly less than $\alpha$ at most $p$. Dualized via Theorem 1, this yields actual coverage strictly above $1 - \alpha$ — conservatism. The same phenomenon reappears for Poisson, negative binomial, hypergeometric — every discrete family. The only way to achieve exact coverage from a discrete test is to use a randomized test, which no one does in practice because it delivers different answers on the same data.

Default choice for binomial CIs is Wilson (Rem 10). Clopper–Pearson is preferred when:

1. Regulatory requirement of strict coverage. FDA submissions, clinical trial monitoring, and other contexts where “actual coverage $\ge 1 - \alpha$” must be certifiable regardless of $p$.
2. Very small $n$ or $x = 0$ / $x = n$. The boundary conventions of Theorem 4 extend to the extremes; Wilson’s closed form degenerates at $x = 0$ (returns $[0, z^2/(n + z^2)]$, well-defined but conservatism-free).
3. Low tolerance for any coverage dip. In monitoring applications where even a 2% under-coverage is unacceptable, Clopper–Pearson’s always-at-or-above-nominal guarantee is worth the width penalty.

The trade-off is explicit: buy coverage guarantees with width. Wilson gives tighter intervals at roughly-nominal (but sometimes slightly-under) coverage.

Let $\{P_{\theta, \psi} : \theta \in \Theta \subset \mathbb{R}, \psi \in \Psi \subset \mathbb{R}^{k-1}\}$ be a regular parametric family with scalar parameter of interest $\theta$ and nuisance $\psi$. The profile log-likelihood is

$\ell_P(\theta) = \sup_\psi \ell(\theta, \psi) = \ell(\theta, \hat\psi(\theta)),$

where $\hat\psi(\theta)$ is the conditional MLE of $\psi$ at fixed $\theta$. The profile-likelihood confidence interval at level $1 - \alpha$ is

$C_P(X) = \{\theta : 2\,[\ell_P(\hat\theta_n) - \ell_P(\theta)] \le \chi^2_{1, 1-\alpha}\},$

where $\hat\theta_n$ is the profile MLE (equivalently, the $\theta$-coordinate of the joint MLE).

Under Wilks’ regularity conditions (see Topic 18 §18.6),

$P_{(\theta_0, \psi_0)}(\theta_0 \in C_P(X)) \to 1 - \alpha \qquad \text{as } n \to \infty,$

for every $(\theta_0, \psi_0) \in \Theta \times \Psi$.

For $X_i \sim \mathcal{N}(\mu, \sigma^2)$ iid, the conditional MLE of $\sigma$ at fixed $\mu$ is $\hat\sigma(\mu) = \sqrt{n^{-1}\sum (X_i - \mu)^2}$. Plugging into the Normal log-likelihood and simplifying, the profile for $\mu$ becomes

$\ell_P(\mu) = -\frac{n}{2}\log\left(\frac{2\pi}{n}\sum(X_i - \mu)^2\right) - \frac{n}{2}.$

The profile MLE is $\hat\mu_n = \bar X$. Expanding $-2[\ell_P(\bar X) - \ell_P(\mu)] \le \chi^2_{1, 1-\alpha}$ and rearranging:

$\frac{n(\bar X - \mu)^2}{\hat\sigma^2(\bar X)} \le \exp\!\left(\frac{\chi^2_{1, 1-\alpha}}{n}\right) \cdot n - n.$

At $n$ large this is nearly $n(\bar X - \mu)^2/S^2 \le \chi^2_{1, 1-\alpha}$ — the asymptotic Wald-CI form. The exact t-CI is $\sqrt n(\bar X - \mu)/S \le t_{n-1, \alpha/2}$; the ratio $t_{n-1, \alpha/2}^2 / \chi^2_{1, 1-\alpha}$ is the source of the asymptotic-vs-exact gap. At $n = 30$, $t_{29, 0.025}^2 \approx 4.18$ vs $\chi^2_{1, 0.95} \approx 3.84$, so the profile CI is slightly wider than the t-CI — an 8% gap that shrinks as $n^{-1}$.

For $X_i \sim \text{Gamma}(\alpha, \beta)$ iid, the conditional MLE of $\beta$ at fixed $\alpha$ has a closed form: the rate MLE score equation is $n\alpha/\beta - \sum X_i = 0$, giving $\hat\beta(\alpha) = \alpha/\bar X$. The profile for $\alpha$ is

$\ell_P(\alpha) = n\alpha\log(\alpha/\bar X) - n\alpha - n\log\Gamma(\alpha) + (\alpha - 1)\sum \log X_i.$

The profile MLE $\hat\alpha_n$ is the solution of $\ell_P'(\alpha) = 0$ — no closed form; solve numerically. The profile CI is then the $\chi^2_1$-threshold contour around $\hat\alpha_n$. For a seeded sample of $n = 30$ from $\text{Gamma}(2, 1)$, the profile CI for $\alpha$ contains $\alpha = 2.0$ with the expected $1 - \alpha$ probability — confirmed empirically in ProfileLikelihoodExplorer and test 19J of the testing harness.

Proof 4 is three lines long. The heavy lifting — the Taylor expansion of the log-likelihood, remainder control, the asymptotic-normality-to-chi-squared step — all lives in the Wilks proof at Topic 18 §18.6, which Topic 19 consumes as a black-box engine. This is characteristic of how testing-theoretic machinery propagates: once Wilks is established, every composite-LRT and every profile-CI coverage fact follows from a one-line invocation. The pedagogical cost of not re-deriving Wilks at each use is zero; the reader who wants the details goes back to Topic 18, and the new material of Topic 19 stays about CIs rather than recycling Wilks.

In generalized linear models — logistic, Poisson, gamma regression, log-linear models — the workhorse CI for a single coefficient is the profile-likelihood CI, computed by refitting the model with the coefficient fixed at a grid of values and tracing the deviance drop. R’s confint() on a glm object defaults to this; the car::Confint extension makes it the explicit recommendation. The reason is the combination of (i) reparameterization invariance (Rem 8), (ii) asymptotic efficiency via Wilks (this proof), and (iii) correct coverage at boundaries (no Wald-type pathology). The cost is computational: each CI endpoint requires a refit. For large models this can be prohibitive, and practitioners fall back to Wald with sandwich variance estimators — a deliberate trade of statistical precision for compute.

When $\theta$ is vector-valued and we want a CI for a scalar function $g(\theta)$, the profile is over $\{\psi : g(\psi) = c\}$ at each target value $c$. Differentiability of $\ell_P$ in $c$ follows from the Danskin envelope theorem: $d\ell_P/dc$ equals $\partial \ell / \partial c$ evaluated at the conditional MLE, under regularity. The vector-θ profile-CI theory is developed in formalML: ; for Topic 19 we stay scalar.