Midterm Exam: Study Guide

STAT3530 Applied Linear Models

Scope

Everything through week 4: the simple linear model and least squares estimation, inference for coefficients and for new observations, diagnostics, transformations, and unusual observations. To review, work from:

  • homeworks 1 through 3
  • the first four sets of slides
  • the first four handouts
  • the scripts, especially the diagnose-and-fix activity

Format

  • in class, open note, calculator allowed, no software
  • around five questions, each with a few short parts
  • written to take about an hour, so the period leaves room to think and check
  • answers are short: a sentence or two, a small calculation, or a few lines of algebra
  • expect a mix of small derivations, plots and output to interpret, and questions posed in the context of a dataset
  • ask me during the exam if a prompt is unclear

What you should be able to do

The model and its estimates

  • write a simple linear model in matrix form, and identify \(X\) for a given setup
  • compute \(X^TX\), its inverse, and \(\hat\beta\) in small cases
  • interpret an intercept and a slope in the context of the data
  • say what centering a predictor changes and what it leaves alone
  • describe what \(\hat\sigma^2\) estimates and where its degrees of freedom come from
  • describe what correlation does and does not tell you about a relationship

Properties of estimators

  • recognize an estimator of the form \(a^TY\)
  • check unbiasedness using \(\mathbb{E}[a^TY] = a^T\mathbb{E}[Y]\)
  • find a variance using \(\text{Var}[a^TY] = \sigma^2a^Ta\) or \(\text{Var}[\hat\beta] = \sigma^2(X^TX)^{-1}\)
  • explain why least squares is preferred among linear unbiased estimators, and when that argument stops applying
  • say how a standard error responds to sample size, spread of the predictor, and error variance
  • read a variance-covariance matrix, including the sign of the off-diagonal entry

Inference

  • read a summary() table
  • state hypotheses that answer a question posed in words
  • build a \(t\) statistic from an estimate and its standard error, including against a value other than zero
  • compute a confidence interval by hand, and interpret it in context
  • distinguish a confidence interval for a mean response from a prediction interval
  • say why intervals widen away from the center of the predictor values

Diagnostics

  • read residual and QQ plots, and name what is wrong
  • check independence when observations come in a meaningful order
  • say what each violation threatens and what it leaves intact
  • judge whether a model is good enough for the question being asked, which is not the same as perfect
  • decide among leaving a problem alone, transforming, and modeling the departure

Transformations

  • recognize when a transformation helps and when it makes things worse
  • choose between transforming the response and transforming the predictor
  • read a Box-Cox plot to choose a power
  • use a centered and squared predictor for a relationship that turns
  • interpret a fitted model on the log scale, including per unit and per doubling
  • say what a back-transformed fitted value estimates
  • explain why \(R^2\) and \(\hat\sigma\) do not compare across models with different responses

Unusual observations

  • distinguish leverage, outliers, and influence, and spot each in a plot
  • compute Cook’s distance from a studentized residual and a leverage
  • judge influence by comparing the fit with and without the point
  • say when removing an observation is defensible

Interpretation and its limits

  • answer the question that was asked, in the units of the data
  • say what an observational study does and does not support
  • recognize when a prediction would extrapolate beyond the data

How much algebra to expect

  • arithmetic small enough for a calculator
  • matrices no bigger than \(2 \times 2\), apart from a single column
  • short derivations of the kind done in class and on homework, a few lines each

One deliberate exception. The heavier matrix material from homeworks 1 and 2 is not examined. You will not be asked to:

  • prove a matrix is symmetric or idempotent
  • work with the hat matrix or with projections
  • argue about rank or linear dependence
  • reproduce the vector geometry of least squares
  • manipulate random vectors beyond \(\mathbb{E}[a^TY]\) and \(\text{Var}[a^TY]\)

You should still know the results that material produces, and be able to use them: \(\text{Var}[\hat\beta] = \sigma^2(X^TX)^{-1}\), the residual variance estimate and its degrees of freedom, and leverage as a diagonal element of the hat matrix.

How to prepare

  • rework the homework problems, especially where you read output and write an interpretation
  • do the practice set under exam conditions, with a calculator and an hour on the clock, before looking at anything else
  • rerun the diagnose-and-fix script and say out loud what each residual plot shows and what you would do about it
  • write yourself a one-page summary of the formulas you expect to need; assembling it is most of the studying, and you can bring it with you