Multiple Regression

Definition and Key Concepts

Multiple regression is a statistical technique used to model the linear relationship between a single response variable (y) and two or more predictor variables (x1, x2, ..., xk). This method allows for more complex modeling than simple linear regression, which only involves one predictor variable.

• Multiple Regression Equation: The general form for sample data is: For population data:

• Predicted Value: is the predicted value of the response variable, computed using the regression equation.

• Predictor Variables: The number of predictors is denoted by k. These are also called independent variables or x variables.

• Sample Size: Denoted by n.

Requirements: For any set of predictor values, the regression equation is associated with a random error (ε), assumed to be normally distributed with mean 0, standard deviation σ, and independent across observations.

Procedure for Finding a Multiple Regression Equation

• Manual calculations are generally impractical for multiple regression; statistical software or technology is required.

Example: Ice Cream Sales

Suppose we want to predict ice cream sales (y) using temperature (x1) and price (x2) as predictors. Data analysis involves:

• Visualizing relationships with scatterplots (e.g., sales vs. temperature, sales vs. price).

• Calculating summary statistics such as multiple R, R Square, Adjusted R Square, and standard error.

• Performing ANOVA to assess the overall significance of the regression model.

• Estimating regression coefficients for each predictor variable.

Example Regression Coefficients:

Example: Predicting Weight

Given a data set of heights, waist circumferences, and weights for 153 males, the multiple regression equation is:

• Where is height and is waist circumference.

Adjusted Coefficient of Determination

Definition

The adjusted coefficient of determination () modifies the multiple coefficient of determination () to account for the number of predictor variables and sample size. It is calculated as:

• Where is the sample size and is the number of predictor variables.

Guidelines for Finding the Best Multiple Regression Equation

• Use common sense and practical considerations to include or exclude variables.

• Consider the P-value for each predictor variable to assess statistical significance.

• Prefer equations with high adjusted values, but try to include only a few predictor variables.

• If adding a predictor does not increase adjusted significantly, it may not be useful.

• For a given number of predictors, select the equation with the largest adjusted .

• Check for multicollinearity (high correlation between predictors); exclude redundant variables.

Example: Predicting Height from Footprint Evidence

Model Selection

Given data on age, foot length, shoe print length, shoe size, and height for 40 subjects, various combinations of predictors can be tested. The best regression equation is selected based on adjusted and P-value.

Interpretation: Although regression methods may suggest using all available predictors, practical considerations (e.g., criminals wear shoes) may favor using a single variable, such as shoe print length. Example equations:

• Height = 64.1 + 4.29 (Foot Length)

• Height = 80.9 + 3.22 (Shoe Print Length)

The P-value of 0.000 indicates a good model fit, but accuracy may be limited by small sample size.

Dummy Variables in Regression

Definition and Use

A dummy variable (or dichotomous variable) is a variable coded as 0 or 1 to represent two categories of a qualitative variable. Dummy variables allow categorical data to be included in regression models.

• Example: Sex coded as 0 = female, 1 = male.

Example: Predicting Child Height Using Dummy Variable

Given heights of father and mother, and sex of child, the regression equation is:

• Height of child = 36.5 - 0.0336 (Height of father) + 0.461 (Height of mother) + 6.14 (Sex)

• Sex = 0 for daughter, 1 for son.

Application:

• For a daughter (Sex = 0): Substitute values to predict height.

• For a son (Sex = 1): Substitute values to predict height.

• The coefficient of 6.14 indicates that, given parental heights, a son is predicted to be 6.14 inches taller than a daughter.

Example Calculation: For a father of 69 in., mother of 63 in.:

• Daughter: Height = 36.5 - 0.0336(69) + 0.461(63) + 6.14(0) = 63.2 in.

• Son: Height = 36.5 - 0.0336(69) + 0.461(63) + 6.14(1) = 69.4 in.

Additional info: Dummy variables are essential for including categorical predictors in regression models, allowing for more flexible and realistic modeling in social and biological sciences.