Ch. 12: Simple Linear Regression

Introduction to Simple Linear Regression

Simple linear regression is a statistical method used to model the relationship between a single explanatory variable and a response variable by fitting a linear equation to observed data.

• Explanatory variable (predictor): The variable used to explain or predict changes in the response variable.

• Response variable: The outcome variable whose variation is being studied.

• Least squares regression method: Finds the line that minimizes the sum of squared differences between observed and predicted values.

Regression Equation:

The regression line is given by:

• Intercept (): The predicted value of y when x = 0.

• Slope (): The change in y for a one-unit increase in x.

Correlation (): Measures the strength and direction of the linear relationship between two variables. .

Coefficient of Determination (): Represents the proportion of variance in the response variable explained by the explanatory variable.

• Residuals: The differences between observed and predicted values ().

Example: If the regression equation is , then for , the predicted is .

Ch. 12: Inference for Simple Linear Regression

Statistical Inference in Regression

Inference in regression involves making conclusions about the population regression parameters based on sample data.

• Hypothesis Test for Slope: Tests whether the explanatory variable is significantly associated with the response variable.

Null Hypothesis (): (no association)

Alternative Hypothesis (): (association exists)

Test Statistic:

• Where is the standard error of the slope estimate.

Confidence Interval for Slope:

• Where is the critical value from the t-distribution.

Assumptions:

• Linearity

• Independence

• Normality of residuals

• Equal variance (homoscedasticity)

Example: Testing if the slope is significantly different from zero to determine if the predictor variable has a meaningful effect on the response.

Ch. 13: Multiple Regression

Introduction to Multiple Regression

Multiple regression extends simple linear regression by modeling the relationship between a response variable and two or more explanatory variables.

• Model:

• Response variable: The outcome being predicted.

• Explanatory variables: The predictors used to explain variation in the response.

ANOVA for Regression: Used to test the overall significance of the regression model.

Individual Variable Testing: Each coefficient is tested to determine if it significantly contributes to the model.

Assumptions:

• Linearity

• Independence

• Normality of residuals

• Equal variance

Model Selection: Variables can be added or removed based on their statistical significance and contribution to .

Quadratic Regression: Includes squared terms to model curvature in the relationship.

Interpretation of Coefficients: Each coefficient represents the expected change in the response variable for a one-unit change in the predictor, holding other variables constant.

Example: Predicting house prices using multiple features such as size, number of bedrooms, and age of the house.

Additional info: These notes summarize key concepts and formulas for regression analysis, including both simple and multiple regression, as well as inference procedures. The content is suitable for exam preparation in a college-level statistics course.