Sample Geometry Lab

From a data matrix to \(\bar{\mathbf x}\), \(\mathbf S\) and \(\mathbf R\)

Rows are observational units and columns are variables, so \(x_{jk}\) is variable \(k\) for unit \(j\). Each row is one point in the scatter plot. Drag a point, click empty space to add one, or double-click a point to remove it. You can also type into the table.

\[ \bar x_k=\frac1n\sum_{j=1}^n x_{jk} \qquad s_{ik}=\frac1n\sum_{j=1}^n(x_{ji}-\bar x_i)(x_{jk}-\bar x_k) \qquad r_{ik}=\frac{s_{ik}}{\sqrt{s_{ii}}\sqrt{s_{kk}}} \qquad \mathbf R=\mathbf D^{-1/2}\mathbf S\,\mathbf D^{-1/2} \]

Chapter 1 uses divisor \(n\) (written \(\mathbf S_n\)). R's cov() and Chapter 3's \(\mathbf S\) divide by \(n-1\). Switch between them below: \(\mathbf S\) changes, \(\mathbf R\) doesn't.

\((x_{j1}-\bar x_1)(x_{j2}-\bar x_2)>0\) product \(<0\) × \(\bar{\mathbf x}\)

Data matrix \(\mathbf X\)

show signed rectangles

Summaries

Three rulers: Euclidean, standardised and Mahalanobis

How far is the point \(P\) from the centre of the cloud? Each distance draws the set of points at the same distance as \(P\). Drag \(P\), reshape the cloud, or change the units of \(x_1\).

\[ d_E=\sqrt{(\mathbf x-\mathbf y)'(\mathbf x-\mathbf y)} \qquad d_S=\sqrt{\sum_k\frac{(x_k-y_k)^2}{s_{kk}}}=\sqrt{(\mathbf x-\mathbf y)'\mathbf D^{-1}(\mathbf x-\mathbf y)} \qquad d_M=\sqrt{(\mathbf x-\mathbf y)'\mathbf S^{-1}(\mathbf x-\mathbf y)} \]

Special cases: \(\mathbf S=\mathbf I\) gives \(d_M=d_E\), and a diagonal \(\mathbf S\) gives \(d_M=d_S\). Uses Ch 2's \(\mathbf S^{-1/2}\): \(d_M\) is the ordinary length after whitening.

Euclidean: circle standardised: axis-aligned ellipse Mahalanobis: tilted ellipse

The same picture after multiplying by \(\mathbf S^{-1/2}\). The cloud becomes round and the Mahalanobis ellipse becomes a circle, so \(d_M\) is just the length of the whitened \(P\).

Shape of the cloud and units

\(k\) rescales \(x_1\), e.g. metres → centimetres.

\(\sqrt{s_{11}}\)

\(\sqrt{s_{22}}\)

\(r_{12}\)

\(k\) (units)

Euclidean standardised Mahalanobis

Distance from the centre to \(P\)

Ruler	At \(k=1\)	Now
Euclidean \(d_E\)
Standardised \(d_S\)
Mahalanobis \(d_M\)

Vectors measure, matrices move

On the left, drag the tips of x and y to see length, inner product, angle and projection. On the right, a matrix \(\mathbf A\) acts on the whole plane. Its determinant is how much it scales area, and its eigenvectors are the directions it doesn't turn.

\[ L_{\mathbf x}=\sqrt{\mathbf x'\mathbf x} \qquad \cos\theta=\frac{\mathbf x'\mathbf y}{L_{\mathbf x}L_{\mathbf y}} \qquad \operatorname{proj}_{\mathbf x}\mathbf y=\frac{\mathbf y'\mathbf x}{\mathbf x'\mathbf x}\,\mathbf x \qquad \mathbf A\mathbf e=\lambda\mathbf e \qquad |\mathbf A-\lambda\mathbf I|=0 \qquad \mathbf A^{-1}\text{ exists}\iff|\mathbf A|\ne0 \]

These are the tools the Chapter 3 geometry is built on: the mean as a projection, correlation as \(\cos\theta\), and \(|\mathbf S|\) as a squared volume.

\(\mathbf x\) (drag the tip) \(\mathbf y\) (drag the tip) \(\operatorname{proj}_{\mathbf x}\mathbf y\)

\(\mathbf I\to\mathbf A\)

\(\mathbf A\mathbf e_1\) (column 1)\(\mathbf A\mathbf e_2\) (column 2)image of the unit square, area \(|\det\mathbf A|\)eigenvector lines

The two vectors

snap to a 0.5 grid (easy to check by hand)

Quantity	Value
\(\mathbf x,\ \mathbf y\)
lengths \(L_{\mathbf x},\ L_{\mathbf y}\)
inner product \(\mathbf x'\mathbf y\)
\(\cos\theta\) and \(\theta\)
\(\operatorname{proj}_{\mathbf x}\mathbf y\)
dependent?

Make \(\mathbf x'\mathbf y=0\) for a right angle. Line the vectors up to make them linearly dependent: \(\mathbf y=c\,\mathbf x\), and \(|\cos\theta|=1\).

The matrix \(\mathbf A\)

Read off from the picture

Quadratic forms and positive definiteness

A symmetric \(\mathbf A\) turns every vector into one number, \(Q(\mathbf x)=\mathbf x'\mathbf A\mathbf x\). The shading shows its sign over the plane and the lines are level sets. If \(\mathbf A\) is positive definite, every level set is an ellipse. Drag the point to evaluate \(Q\).

\[ Q(\mathbf x)=\mathbf x'\mathbf A\mathbf x=a_{11}x_1^2+2a_{12}x_1x_2+a_{22}x_2^2=\lambda_1(\mathbf e_1'\mathbf x)^2+\lambda_2(\mathbf e_2'\mathbf x)^2 \qquad \mathbf A\text{ p.d.}\iff Q(\mathbf x)>0\ \forall\,\mathbf x\ne\mathbf 0\iff\lambda_1,\lambda_2>0 \]

Spectral decomposition \(\mathbf A=\lambda_1\mathbf e_1\mathbf e_1'+\lambda_2\mathbf e_2\mathbf e_2'\) rewrites \(Q\) as a weighted sum of squares. The signs of the weights decide everything. Maximisation lemma: on the unit circle, \(\lambda_2\le Q\le\lambda_1\).

\(Q>0\) \(Q<0\) \(Q=1\) eigenvector axes

Choose a symmetric \(\mathbf A\)

\(a_{11}\)

\(a_{12}\)

\(a_{22}\)

\(Q(\mathbf x)\) for unit vectors \(\mathbf x=(\cos\phi,\sin\phi)\). The curve always lies between \(\lambda_2\) and \(\lambda_1\). If it dips below zero, \(\mathbf A\) is not positive definite.

Transforming a random vector: \(\mathbf Y=\mathbf A\mathbf X+\mathbf b\)

AI Exploration 1, done live. 400 draws of \(\mathbf X\) are pushed through \(\mathbf A\) and shifted by \(\mathbf b\). The formulas predict the new mean and covariance without any data, and the simulation checks them.

\[ E(\mathbf c'\mathbf X)=\mathbf c'\boldsymbol\mu \qquad \operatorname{Var}(\mathbf c'\mathbf X)=\mathbf c'\boldsymbol\Sigma\mathbf c \qquad E(\mathbf A\mathbf X+\mathbf b)=\mathbf A\boldsymbol\mu+\mathbf b \qquad \operatorname{Cov}(\mathbf A\mathbf X+\mathbf b)=\mathbf A\boldsymbol\Sigma\mathbf A' \]

\(\mathbf b\) moves the mean and does nothing to the covariance. If \(\mathbf A\) is rank-deficient, \(\mathbf A\boldsymbol\Sigma\mathbf A'\) is singular and the cloud collapses onto a line. The same rules drive Chapter 3's \(\mathbf b'\bar{\mathbf x}\) and \(\mathbf b'\mathbf S\mathbf b\).

\(\mathbf X\): ellipse of \(\boldsymbol\Sigma\) around \(\boldsymbol\mu\)

\(\mathbf Y\): predicted ellipse of \(\mathbf A\boldsymbol\Sigma\mathbf A'\) around \(\mathbf A\boldsymbol\mu+\mathbf b\)× sample mean of the 400 \(\mathbf y\)'s

\(\mathbf X\sim(\boldsymbol\mu,\boldsymbol\Sigma)\), \(\boldsymbol\mu=(2,1)'\)

\(\sigma_1\)

\(\sigma_2\)

\(\rho_{12}\)

\(\mathbf A\) and \(\mathbf b\)

Theory vs 400 simulated draws

Two pictures of one data matrix

An \(n\times p\) matrix \(\mathbf X\) is either n points in \(\mathbb R^p\) (rows) or p vectors in \(\mathbb R^n\) (columns). With \(n=3,\ p=2\) the column picture fits in 3-D. Drag the points on the left.

\[ \underbrace{\frac{\mathbf y_i'\mathbf 1}{\mathbf 1'\mathbf 1}\,\mathbf 1 = \bar x_i\mathbf 1}_{\text{mean = projection}} \qquad \underbrace{\mathbf d_i=\mathbf y_i-\bar x_i\mathbf 1}_{\text{deviation vector}} \qquad \underbrace{\mathbf d_i'\mathbf d_k=(n-1)\,s_{ik}}_{\text{inner product = covariance}} \qquad \underbrace{\cos\theta_{ik}=r_{ik}}_{\text{angle = correlation}} \]

Uses Ch 2: projection \(\mathrm{proj}_{\mathbf y}\mathbf x=\frac{\mathbf x'\mathbf y}{\mathbf y'\mathbf y}\mathbf y\), length \(\|\mathbf x\|=\sqrt{\mathbf x'\mathbf x}\), angle \(\cos\theta=\frac{\mathbf x'\mathbf y}{\|\mathbf x\|\|\mathbf y\|}\).

observation \(\mathbf x_j\) (drag me)× \(\bar{\mathbf x}\), the balance point

\(\mathbf y_1,\ \mathbf d_1\) \(\mathbf y_2,\ \mathbf d_2\) line of \(\mathbf 1\) \(\bar x_i\mathbf 1\) Drag to rotate ·

Data set

Show in 3-D

\(\mathbf y_i\) projections \(\mathbf d_i\) and \(\theta\) parallelogram

Divisor

Geometry = statistics

Geometry	Statistic	Live value
\(\mathbf y_1'\mathbf 1/n\)	\(\bar x_1\)
\(\mathbf y_2'\mathbf 1/n\)	\(\bar x_2\)
\(\\|\mathbf d_1\\|^2\)	\(k\,s_{11}\)
\(\\|\mathbf d_2\\|^2\)	\(k\,s_{22}\)
\(\mathbf d_1'\mathbf d_2\)	\(k\,s_{12}\)
\(\mathbf d_1'\mathbf 1,\ \mathbf d_2'\mathbf 1\)	always \(0\)
\(\cos\theta\)	\(r_{12}\)
\(\text{area}^2\)	\(k^2\,\|\mathbf S\|\)

Correlation is the cosine of an angle

AI Exploration-1. \(\mathbf d_1,\mathbf d_2\in\mathbb R^5\) are centred (each sums to zero). Turn the angle and watch the five observations.

\[ r_{12}=\frac{\mathbf d_1'\mathbf d_2}{\|\mathbf d_1\|\,\|\mathbf d_2\|}=\cos\theta \qquad\text{and}\qquad |\mathbf d_1'\mathbf d_2|\le\|\mathbf d_1\|\,\|\mathbf d_2\| \ \Rightarrow\ -1\le r_{12}\le 1 \]

Uses Ch 2: the Cauchy–Schwarz inequality \((\mathbf a'\mathbf b)^2\le(\mathbf a'\mathbf a)(\mathbf b'\mathbf b)\), with equality only when one vector is a multiple of the other (\(r=\pm1\)).

The flat plane spanned by \(\mathbf d_1,\mathbf d_2\) inside \(\mathbb R^5\). The purple segment is the projection of \(\mathbf d_2\) onto \(\mathbf d_1\), with length \(\|\mathbf d_2\|\cos\theta\).

The same ten numbers as five points \((d_{1j},d_{2j})\). The dashed line is the least-squares line.

Controls

\(\theta\)

\(\|\mathbf d_2\|/\|\mathbf d_1\|\)

\(\bar{\mathbf x}\) and \(\mathbf S\) are random too

Before the data are collected, each row \(\mathbf X_j\) is a random vector, so \(\bar{\mathbf X}\) and \(\mathbf S\) are random as well. Draw many samples of size \(n\) from \(N_2(\boldsymbol\mu,\boldsymbol\Sigma)\) and check the theorem.

\[ E(\bar{\mathbf X})=\boldsymbol\mu \qquad \operatorname{Cov}(\bar{\mathbf X})=\tfrac1n\boldsymbol\Sigma \qquad E(\mathbf S)=\boldsymbol\Sigma \qquad E(\mathbf S_n)=\tfrac{n-1}{n}\boldsymbol\Sigma \]

Uses Ch 2: linearity \(E(\mathbf A\mathbf X\mathbf B)=\mathbf A\,E(\mathbf X)\,\mathbf B\) and \(\operatorname{Cov}(\sum a_j\mathbf X_j)=\sum_j\sum_l a_ja_l\operatorname{Cov}(\mathbf X_j,\mathbf X_l)\).

population contour (\(\boldsymbol\Sigma\)) latest sample every \(\bar{\mathbf x}\) so far contour of \(\boldsymbol\Sigma/n\)

average \(s_{11}\) with divisor \(n-1\) → \(\sigma_{11}\) with divisor \(n\) → \(\tfrac{n-1}{n}\sigma_{11}\)

Population and sample size

\(n\)

\(\sigma_1\)

\(\sigma_2\)

\(\rho_{12}\)

Changing a slider starts a fresh experiment. With n = 2 the divisor-n estimate is off by a factor of ½.

Simulation vs theory · 0 samples

Quantity	Average over samples	Theory
\(\bar{\mathbf x}\)		\(\boldsymbol\mu\) =
\(\mathbf S\)		\(\boldsymbol\Sigma\) =
\(\mathbf S_n\)		\(\tfrac{n-1}{n}\boldsymbol\Sigma\) =
spread of \(\bar{\mathbf x}\)		\(\boldsymbol\Sigma/n\) =

One number for spread: \(|\mathbf S|\) and \(\operatorname{tr}(\mathbf S)\)

Both numbers compress the whole covariance matrix into one value. The ellipse is \((\mathbf x-\bar{\mathbf x})'\mathbf S^{-1}(\mathbf x-\bar{\mathbf x})=c^2\) with \(c=2\). The axes are fixed, so you can compare areas between settings.

\[ |\mathbf S| = s_{11}s_{22}(1-r_{12}^2)=\lambda_1\lambda_2 \qquad \operatorname{tr}(\mathbf S)=s_{11}+s_{22}=\lambda_1+\lambda_2 \qquad \text{area}=\pi c^2\sqrt{|\mathbf S|} \qquad |\mathbf S|=\frac{(\text{volume of }\mathbf d\text{'s})^2}{(n-1)^p} \]

Uses Ch 2: determinant \(|\mathbf A|=ad-bc\), and eigenvalues from \(|\mathbf A-\lambda\mathbf I|=0\). The determinant is the product of the eigenvalues and the trace is their sum.

ellipse of \(\mathbf S\), axes of half-length \(c\sqrt{\lambda_1},\ c\sqrt{\lambda_2}\) 30 observations with exactly this \(\mathbf S\)

Shape the covariance matrix

\(\sqrt{s_{11}}\)

\(\sqrt{s_{22}}\)

\(r_{12}\)

Textbook Example 3.8 (all have \(|\mathbf S|=9\))

overlay all three Ex. 3.8 ellipses show data

Quantity	Formula	Live value
\(\|\mathbf S\|\)	\(s_{11}s_{22}-s_{12}^2\)
	\(s_{11}s_{22}(1-r_{12}^2)\)
	\(\lambda_1\lambda_2\)
\(\operatorname{tr}(\mathbf S)\)	\(s_{11}+s_{22}\)
\(\operatorname{tr}(\mathbf S)\)	\(\lambda_1+\lambda_2\)
area	\(\pi c^2\sqrt{\|\mathbf S\|}\)

Inside \(\mathbf S\): rotate, stretch, rotate back

This tab brings in Chapter 2. The spectral decomposition shows what \(\mathbf S\) does geometrically. Its square root turns a circle into the ellipse from the Generalized variance tab, and its inverse square root turns the data back into a circle.

\[ \mathbf S=\mathbf P\boldsymbol\Lambda\mathbf P'=\lambda_1\mathbf e_1\mathbf e_1'+\lambda_2\mathbf e_2\mathbf e_2' \qquad \mathbf S^{1/2}=\mathbf P\boldsymbol\Lambda^{1/2}\mathbf P' \qquad \mathbf S^{-1/2}=\mathbf P\boldsymbol\Lambda^{-1/2}\mathbf P' \qquad \mathbf R=\mathbf D^{-1/2}\mathbf S\,\mathbf D^{-1/2} \]

From Ch 2 (spectral decomposition, square-root matrices, \(\boldsymbol\rho=\mathbf V^{-1/2}\boldsymbol\Sigma\mathbf V^{-1/2}\)) and 3.3 (\(\mathbf R\) from \(\mathbf S\)). The covariance of \(\mathbf A\mathbf x\) is \(\mathbf A\mathbf S\mathbf A'\) (3.4), and that drives everything below.

Step

Choose \(\mathbf S\)

\(\sqrt{s_{11}}\)

\(\sqrt{s_{22}}\)

\(r_{12}\)

The pieces, live

Linear combinations inherit \(\bar{\mathbf x}\) and \(\mathbf S\)

\(y_j=\mathbf b'\mathbf x_j\) turns each observation into one number. For a unit \(\mathbf b\), that number is where the point's shadow lands on the line in direction \(\mathbf b\). Drag the b and c handles.

\[ \bar y=\mathbf b'\bar{\mathbf x} \qquad s_y^2=\mathbf b'\mathbf S\mathbf b \qquad s_{yz}=\mathbf b'\mathbf S\mathbf c \qquad \text{many at once: } \bar{\mathbf y}=\mathbf A\bar{\mathbf x},\ \mathbf S_y=\mathbf A\mathbf S\mathbf A' \qquad \lambda_2\le\frac{\mathbf b'\mathbf S\mathbf b}{\mathbf b'\mathbf b}\le\lambda_1 \]

Sample version of Ch 2's \(E(\mathbf c'\mathbf X)=\mathbf c'\boldsymbol\mu,\ \operatorname{Var}(\mathbf c'\mathbf X)=\mathbf c'\boldsymbol\Sigma\mathbf c\). The bound on the right is the Rayleigh quotient from Ch 2.

projections onto \(\mathbf b\) projections onto \(\mathbf c\) spread curve: \(2\sqrt{\mathbf b'\mathbf S\mathbf b}\) in every direction

show \(\mathbf c\) show spread curve

Quantity	Formula	Live value
direction b	unit vector
mean of \(y_b\)	\(\mathbf b'\bar{\mathbf x}\)
variance of \(y_b\)	\(\mathbf b'\mathbf S\mathbf b\)
direction c	unit vector
variance of \(y_c\)	\(\mathbf c'\mathbf S\mathbf c\)
covariance	\(\mathbf b'\mathbf S\mathbf c\)
correlation	\(\frac{\mathbf b'\mathbf S\mathbf c}{\sqrt{\mathbf b'\mathbf S\mathbf b}\sqrt{\mathbf c'\mathbf S\mathbf c}}\)
range of variances	\([\lambda_2,\ \lambda_1]\)

As \(\mathbf b\) turns through 180°, \(\mathbf b'\mathbf S\mathbf b\) swings between \(\lambda_2\) and \(\lambda_1\) and never goes outside them. The peak is at \(\mathbf e_1\), the first principal component direction.

The bivariate normal: contours, probability ellipses and conditioning

The density is constant on the ellipses \((\mathbf x-\boldsymbol\mu)'\boldsymbol\Sigma^{-1}(\mathbf x-\boldsymbol\mu)=c^2\), the Mahalanobis circles from Chapter 1. Pick \(c^2\) from \(\chi^2_2\) and the ellipse holds a known probability. Drag the vertical slice to condition on \(X_1\).

\[ f(\mathbf x)=\frac{1}{2\pi|\boldsymbol\Sigma|^{1/2}}e^{-\frac12(\mathbf x-\boldsymbol\mu)'\boldsymbol\Sigma^{-1}(\mathbf x-\boldsymbol\mu)} \qquad (\mathbf X-\boldsymbol\mu)'\boldsymbol\Sigma^{-1}(\mathbf X-\boldsymbol\mu)\sim\chi^2_p \qquad X_2\mid X_1=x_1\ \sim\ N\!\Big(\mu_2+\tfrac{\sigma_{12}}{\sigma_{11}}(x_1-\mu_1),\ \sigma_{22}-\tfrac{\sigma_{12}^2}{\sigma_{11}}\Big) \]

Axes of the ellipse: eigenvectors \(\mathbf e_i\) of \(\boldsymbol\Sigma\), half-lengths \(c\sqrt{\lambda_i}\) (Ch 2). For \(p=2\), \(\chi^2_2(\alpha)=-2\ln\alpha\) exactly. Conditional result: Theorem 4.6 with \(\boldsymbol\Sigma_{12}\boldsymbol\Sigma_{22}^{-1}\) reduced to scalars.

probability ellipse \(c^2=\chi^2_2(\alpha)\) slice \(X_1=x_1\) regression line \(E(X_2\mid X_1)\)

marginal \(X_2\sim N(\mu_2,\sigma_{22})\) conditional \(X_2\mid X_1=x_1\): the slice, rescaled to area 1

Parameters \((\boldsymbol\mu=\mathbf 0)\)

\(\sigma_1\)

\(\sigma_2\)

\(\rho_{12}\)

slice \(x_1\)

Probability ellipse

1000 simulated points density shading

Live numbers

Quantity	Value
cut-off \(c^2=\chi^2_2(\alpha)=-2\ln\alpha\)
half-axes \(c\sqrt{\lambda_1},\ c\sqrt{\lambda_2}\)
share inside, \(\approx 1-\alpha\)
\(E(X_2\mid x_1)=\frac{\sigma_{12}}{\sigma_{11}}x_1\)
\(\operatorname{Var}(X_2\mid x_1)=\sigma_{22}(1-\rho^2)\)

Is it normal? Q-Q plots, \(r_Q\) and the chi-square plot

Pick a data set and run the lecture's checks on it. Drag any point in the scatter plot to make your own outlier, and watch where it lands in each plot.

\[ \text{Q-Q: plot } \Big(\Phi^{-1}\big(\tfrac{j-1/2}{n}\big),\ x_{(j)}\Big) \qquad r_Q=\operatorname{corr}\big(q_j,\,x_{(j)}\big) \qquad d_j^2=(\mathbf x_j-\bar{\mathbf x})'\mathbf S^{-1}(\mathbf x_j-\bar{\mathbf x})\ \dot\sim\ \chi^2_p \]

Reject normality at level \(\alpha\) if \(r_Q\) is below the critical point for \(n\) (Johnson & Wichern Table 4.2). For \(p=2\), \(\chi^2_2\) quantiles are \(-2\ln(1-\text{prob})\). About half the \(d_j^2\) should be below \(\chi^2_2(0.5)=1.39\).

\(n\)

\(d^2=\chi^2_2(.5)\): 50% ellipse \(d^2=\chi^2_2(.005)=10.6\)

Q-Q plot of

Chi-square plot: ordered \(d_j^2\) against \(\chi^2_2\) quantiles. The line is \(y=x\).

Verdicts

Critical points at \(\alpha=0.05\), interpolated in \(n\).

Box-Cox: choosing a power to make data look normal

Powers below 1 pull in a long right tail, and powers above 1 stretch it out. Slide \(\lambda\) and watch the histogram and Q-Q plot. The likelihood curve underneath picks the best \(\lambda\) for you.

\[ x^{(\lambda)}=\begin{cases}\dfrac{x^\lambda-1}{\lambda}&\lambda\ne0\\[4pt] \ln x&\lambda=0\end{cases} \qquad \ell(\lambda)=-\frac n2\ln\Big[\frac1n\sum_{j=1}^n\big(x_j^{(\lambda)}-\overline{x^{(\lambda)}}\big)^2\Big]+(\lambda-1)\sum_{j=1}^n\ln x_j \]

\(\hat\lambda\) maximises \(\ell(\lambda)\). The last term is the Jacobian of the transformation. Without it, small \(\lambda\) would always win by shrinking the variance. The best \(\lambda\) often improves normality but doesn't guarantee it, so always re-check.

Histogram of \(x^{(\lambda)}\) with the normal curve that has the same mean and SD.

Q-Q plot of \(x^{(\lambda)}\) with \(r_Q\) and the 5% critical point.

\(\ell(\lambda)\) for \(-1\le\lambda\le2\). Click or drag on the chart to set \(\lambda\).

Choose \(\lambda\)

\(\lambda\)

Live numbers

Hotelling's \(T^2\): testing a mean vector

Drag \(\boldsymbol\mu_0\) (the cross). The joint test rejects \(H_0:\boldsymbol\mu=\boldsymbol\mu_0\) exactly when the cross leaves the purple ellipse. The dashed box is where two separate \(t\)-tests both accept, so the corners of the box are where one-at-a-time tests and the joint test disagree.

\[ t^2=n(\bar x-\mu_0)(s^2)^{-1}(\bar x-\mu_0) \qquad T^2=n(\bar{\mathbf x}-\boldsymbol\mu_0)'\mathbf S^{-1}(\bar{\mathbf x}-\boldsymbol\mu_0)\ \sim\ \frac{(n-1)p}{n-p}\,F_{p,\,n-p}\ \text{ under } H_0 \]

Reject at level \(\alpha\) when \(T^2>\frac{(n-1)p}{n-p}F_{p,n-p}(\alpha)\). \(T^2\) is the statistical distance from Ch 1 between \(\bar{\mathbf x}\) and \(\boldsymbol\mu_0\), measured with \(\widehat{\mathrm{Cov}}(\bar{\mathbf X})=\frac1n\mathbf S\) from Ch 3.

\(\boldsymbol\mu_0\) values \(T^2\) accepts \(\boldsymbol\mu_0\) values both \(t\)-tests accept observations

density of \(\frac{(n-1)p}{n-p}F_{p,n-p}\) rejection region, area \(\alpha\) observed \(T^2\) simulated \(T^2\) under \(H_0\)

Controls

Live numbers

The likelihood ratio test and Wilks' lambda

Fit the data twice: with \(\boldsymbol\mu\) free (spread \(\hat{\boldsymbol\Sigma}\) about \(\bar{\mathbf x}\)) and with \(\boldsymbol\mu\) forced to \(\boldsymbol\mu_0\) (spread \(\hat{\boldsymbol\Sigma}_0\) about \(\boldsymbol\mu_0\)). The wrong centre inflates the spread along \(\bar{\mathbf x}-\boldsymbol\mu_0\). Drag \(\boldsymbol\mu_0\) and compare the two ellipses.

\[ \Lambda=\frac{\max_{\boldsymbol\Sigma}L(\boldsymbol\mu_0,\boldsymbol\Sigma)}{\max_{\boldsymbol\mu,\boldsymbol\Sigma}L(\boldsymbol\mu,\boldsymbol\Sigma)}=\left(\frac{|\hat{\boldsymbol\Sigma}|}{|\hat{\boldsymbol\Sigma}_0|}\right)^{n/2} \qquad \hat{\boldsymbol\Sigma}_0=\hat{\boldsymbol\Sigma}+(\bar{\mathbf x}-\boldsymbol\mu_0)(\bar{\mathbf x}-\boldsymbol\mu_0)' \]

\[ \Lambda^{2/n}=\Big(1+\frac{T^2}{n-1}\Big)^{-1} \qquad -2\ln\Lambda\ \dot\sim\ \chi^2_{\nu-\nu_0}\ \text{ for large } n \]

Theorem 5.1: small \(\Lambda\) means large \(T^2\), so the LRT and Hotelling's test are the same test. Theorem 5.2: for large \(n\), \(-2\ln\Lambda\) is roughly \(\chi^2\) with \(\nu-\nu_0=p\) degrees of freedom, since \(H_0\) fixes the \(p\) means. \(\hat{\boldsymbol\Sigma}\) and \(\hat{\boldsymbol\Sigma}_0\) are the MLEs (divisor \(n\)) from Ch 4.

\(\hat{\boldsymbol\Sigma}\) about \(\bar{\mathbf x}\) \(\hat{\boldsymbol\Sigma}_0\) about \(\boldsymbol\mu_0\) deviations \(\mathbf x_j-\boldsymbol\mu_0\)

\(\Lambda^{2/n}\) falls as \(T^2\) grows, so "\(\Lambda\) small" and "\(T^2\) large" mark out the same samples.

exact density of \(-2\ln\Lambda\) under \(H_0\) \(\chi^2_p\) approximation (Theorem 5.2) \(\chi^2_p(\alpha)\) cut-off and the exact tail area beyond it

Theorem 5.2: how large is large \(n\)?

\(n\)

\(p\)

show deviations from \(\boldsymbol\mu_0\)