Correlation and Regression: Building Statistical Models

Introduction

This study guide covers the foundational concepts of correlation and regression analysis, focusing on their use in building statistical models. It includes definitions, hypothesis testing, effect size interpretation, model fitting, residual analysis, and practical applications using R output. These topics are essential for understanding how to analyze and interpret relationships between quantitative variables in statistics.

Correlation vs Regression

Goals and Differences

• Correlation measures the strength and direction of an undirected association between two quantitative variables (e.g., Y1 and Y2).

• Regression models the directed relationship between a predictor (independent variable, X) and a response (dependent variable, Y), often to predict Y from X or to test if X affects Y.

Key Points:

• Correlation: Hypothesis testing for association between variables.

• Regression: Hypothesis testing for effect of X on Y; estimation and prediction.

Example: Testing if two exam scores (TT1 and TT2) are associated (correlation) vs. predicting TT2 from TT1 (regression).

Forms of Association

Types of Relationships

• Linear, positive: Both variables increase together.

• Linear, negative: One variable increases as the other decreases.

• Bent (curvilinear): Relationship is not a straight line.

• Gaussian: Optimum curve, e.g., bell-shaped.

• Sigmoid: S-shaped, threshold response.

• Upper limit: Limiting factor, e.g., saturation.

Example: Scatterplots can reveal the form of association between variables, guiding the choice of statistical model.

Correlation Analysis

Pearson Correlation Coefficient (r)

• Definition: Measures the strength and direction of the linear relationship between two variables.

• Range:

• Interpretation:

  • : Perfect positive linear relationship

  • : Perfect negative linear relationship

  • : No linear relationship

Significance Test for r:

• Null hypothesis: (no linear association)

• Test statistic:

• p-value: Probability of observing the data if is true

Example: Testing if the proportion of black nose pigmentation in lions is correlated with age.

Spearman Rank Correlation (\rho)

• Definition: Non-parametric measure of monotonic association, based on ranks.

• Use: When data are not normally distributed or contain outliers.

• Null hypothesis:

Example: Calculating Spearman's for ranked lion age and black pigmentation data.

Effect Size and Statistical Significance

Direction, Size, and Significance of Effect

• Direction: Is the relationship positive or negative?

• Size: How strong is the association? (e.g., for regression, effect size thresholds)

• Statistical significance: Is the effect unlikely to be due to chance? (p-value < alpha)

Rules of Thumb for Effect Size

Additional info: Table based on Cohen's (1988) guidelines for effect size interpretation.

Regression Analysis

Simple Linear Regression Model

• Equation:

• Interpretation:

  • : Intercept (predicted value when )

  • : Slope (change in for a one-unit increase in )

  • : Error (residual)

• Goal: Find the line that best fits the data (minimizes sum of squared residuals)

Example: Predicting lion age from proportion of black nose pigmentation:

Statistical Inference for Regression

• Hypothesis testing:

  • Test if slope () or intercept () is significantly different from zero (t-test)

  • Test overall model fit (F-test)

• Estimation:

  • Estimate parameters (slope, intercept)

  • Predict mean or individual response for given

Interpreting Regression Output (R Example)

• Coefficients: Estimates for intercept and slope, with standard errors and p-values

• R-squared (): Proportion of variance in explained by

• Significance: p-value for slope tests if is a significant predictor of

Example: In the lion data, means 62% of the variation in age is explained by black nose pigmentation.

Residual Analysis and Model Assumptions

Checking Assumptions

• Linearity: Relationship between and should be linear.

• Normality: Residuals should be approximately normally distributed.

• Homoscedasticity: Variance of residuals should be constant (no thickening).

• Independence: Residuals should not be correlated.

Residual Plots: Used to visually check these assumptions. Look for patterns, non-linearity, or outliers.

Influential Points and Outliers

• High leverage points: Points far from the mean of ; can strongly affect the slope.

• Large residuals: Points far from the regression line.

• Influential points: Points that, if omitted, would substantially change the regression results (e.g., Cook's D > 1).

Should outliers be removed? Only with biological/statistical justification; report results with and without outliers.

Stepwise Guide to Regression Analysis

Applications and Examples

• Predicting Lion Age: Regression of age on black nose pigmentation to inform sustainable trophy hunting.

• Human Brain Example: Regression analysis of glia-neuron ratio vs. brain mass, with interpretation of residual plots and model fit.

Summary Table: Statistical Tests Overview

Key Formulas

• Pearson correlation coefficient:

• Simple linear regression:

• t-test for slope:

• R-squared:

Conclusion

Understanding correlation and regression is fundamental for analyzing relationships between variables, making predictions, and testing hypotheses in statistics. Proper model checking, effect size interpretation, and awareness of assumptions are essential for valid inference and application.