Correlation and Regression
Master the least squares regression line, make predictions using interpolation and extrapolation, and understand the limitations of regression analysis.
The Least Squares Regression Line
The least squares regression line (line of best fit) minimises the sum of the squared vertical distances (residuals) from each data point to the line. The equation is:
ŷ = a + bx
where b = slope (gradient) = r x (sy / sx), and a = y-intercept = ȳ - b x̄
The regression line always passes through the point (x̄, ȳ), the mean of x and mean of y. The slope b tells us the predicted change in y for each unit increase in x.
Important Properties
- ●The regression line of y on x is used to predict y from a given x value.
- ●The sign of the slope b is always the same as the sign of r.
- ●In the HSC, you will typically use a calculator or given values to find a and b.
Interpolation vs Extrapolation
Using a regression line to make predictions requires understanding the difference between interpolation and extrapolation:
Interpolation
Making predictions within the range of the original data. These predictions are generally reliable.
Example: Data ranges from x = 10 to x = 50. Predicting y when x = 30 is interpolation.
Extrapolation
Making predictions outside the range of the original data. These predictions are less reliable and should be made with caution.
Example: Data ranges from x = 10 to x = 50. Predicting y when x = 80 is extrapolation.
Residuals and Model Fit
A residual is the difference between the actual y-value and the predicted y-value from the regression line:
Residual = yactual - ŷpredicted
A residual plot graphs residuals against x-values. If the linear model is appropriate, residuals should show a random pattern with no clear trend. A pattern in the residuals suggests a non-linear model may be better.
Good fit (random residuals)
Poor fit (curved pattern)
Key Vocabulary
Regression Line
The line of best fit that minimises the sum of squared residuals, used to model the relationship between x and y.
Interpolation
Predicting a y-value for an x-value within the range of the observed data.
Extrapolation
Predicting a y-value for an x-value outside the range of the observed data; less reliable.
Residual
The difference between an observed value and the value predicted by the regression line: y - ŷ.
Worked Examples
A regression line is ŷ = 12.5 + 3.2x. Predict y when x = 8.
Step 1: Substitute x = 8 into the equation: ŷ = 12.5 + 3.2(8).
Step 2: ŷ = 12.5 + 25.6 = 38.1.
Answer: The predicted value of y is 38.1.
Data: x̄ = 5, ȳ = 24, r = 0.85, sx = 2, sy = 6. Find the regression equation.
Step 1: b = r x (sy / sx) = 0.85 x (6/2) = 0.85 x 3 = 2.55.
Step 2: a = ȳ - b x̄ = 24 - 2.55(5) = 24 - 12.75 = 11.25.
Answer: The regression equation is ŷ = 11.25 + 2.55x.
The regression line ŷ = 50 - 2.5x is based on data where x ranges from 3 to 15. A student uses it to predict y at x = 25. Comment on the reliability.
Step 1: x = 25 is outside the observed range (3 to 15), so this is extrapolation.
Step 2: ŷ = 50 - 2.5(25) = 50 - 62.5 = -12.5.
Step 3: The prediction is unreliable because extrapolation assumes the linear trend continues beyond the data, which may not hold.
Answer: The prediction of -12.5 is unreliable because it involves extrapolation well beyond the observed data range.
Knowledge Check
Select the correct answer for each question. Click "Check Answer" to see if you are right.
Question 1
The regression line ŷ = 20 + 4x passes through which special point?
Question 2
Given ŷ = 100 - 5.5x, what is the predicted y when x = 12?
Question 3
If the actual y-value is 42 and the predicted y-value is 38, what is the residual?
Question 4
Data is collected for x values from 5 to 30. Using the regression line to predict y at x = 50 is an example of:
Question 5
In a regression equation ŷ = a + bx, the slope b = -2.3. What does this mean?
Key Concepts Summary
- ●The least squares regression line ŷ = a + bx minimises the sum of squared residuals.
- ●The slope b represents the predicted change in y for each unit increase in x.
- ●Interpolation (within the data range) is more reliable than extrapolation (outside the data range).
- ●Residuals (actual - predicted) help assess the quality of the model; random residuals indicate a good fit.
- ●The regression line always passes through the point of means (x̄, ȳ).