Probability: Fundamental Concepts and Rules

Random Processes and the Law of Large Numbers

Probability is the study of random phenomena and chance behavior. A random process is one where the outcome of any particular trial is unknown, but the proportion of a specific outcome approaches a fixed value as the number of trials increases. The Law of Large Numbers states that as the number of repetitions of a probability experiment increases, the observed proportion of a certain outcome gets closer to the theoretical probability of that outcome.

• Simulation is a technique used to recreate random events and measure how often a goal is observed.

• Probability quantifies the likelihood of an event occurring, ranging from 0 (impossible) to 1 (certain).

• Long-term predictability emerges from random short-term results.

Example: Rolling a die many times and recording the proportion of times a specific number appears demonstrates convergence to the true probability as the number of trials increases.

Example: Tracking the frequency of red lights during a commute over many days illustrates the Law of Large Numbers.

The Law of Large Numbers vs. The Nonexistent Law of Averages

The Law of Large Numbers is often confused with the so-called "Law of Averages," which incorrectly suggests that past outcomes influence future ones. In reality, probability experiments are memoryless: each trial is independent of previous trials.

• For example, the probability of having a girl remains 0.5 regardless of previous children.

Example: Simulating families with multiple children shows that the likelihood of the fifth child being a boy is still 0.5, even if the first four were girls.

Probability Experiments, Sample Space, and Events

A probability experiment is any process that can be repeated with uncertain results. The sample space (S) is the set of all possible outcomes. An event is any collection of outcomes from the experiment, which may consist of one or more outcomes.

• Simple events: events with one outcome.

• Events are denoted by capital letters (E, F, etc.).

Rules of Probability

Basic Probability Rules

Probability models list possible outcomes and their probabilities, and must satisfy:

• Rule 1: for any event E.

• Rule 2: The sum of probabilities for all outcomes in the sample space equals 1: .

Key Concepts:

• Probability 0: impossible event

• Probability 1: certain event

• Closer to 1: more likely; closer to 0: less likely

• Unusual event: probability less than 0.05

Empirical Method for Probability

The empirical method approximates probability by observing the frequency of an event in repeated trials:

Example: Building a probability model from observed frequencies in a game.

Classical Method for Probability

The classical method applies when outcomes are equally likely:

• For sample space S:

Subjective Probability

Subjective probability is based on personal judgment rather than empirical or classical methods.

• Example: An economist's estimate of recession probability.

Addition Rule and Complements

Addition Rule for Disjoint (Mutually Exclusive) Events

Two events are disjoint if they have no outcomes in common. For disjoint events E and F:

This rule extends to more than two disjoint events.

General Addition Rule

For events that are not disjoint, the General Addition Rule avoids double-counting:

Example: Venn diagrams help visualize overlapping and disjoint events.

Complement Rule

The complement of event E, denoted , is all outcomes not in E. The probability of the complement is:

Example: If 31.6% of households own a dog, the probability a household does not own a dog is .

Example: Using travel time data to compute probabilities for commute times.

Independence and the Multiplication Rule

Independent and Dependent Events

Events E and F are independent if the occurrence of E does not affect the probability of F. If the occurrence of E affects F, the events are dependent.

• Disjoint events are not independent.

Multiplication Rule for Independent Events

If E and F are independent:

For n independent events:

At-Least Probabilities

To find the probability that at least one event occurs:

Conditional Probability and the General Multiplication Rule

Conditional Probability

The conditional probability of E given F is:

It represents the probability of E occurring, given that F has occurred.

General Multiplication Rule

The probability that both E and F occur:

Counting Techniques

Multiplication Rule of Counting

If a task consists of a sequence of choices, the total number of ways to make the selections is:

Permutations

A permutation is an ordered arrangement of r objects chosen from n distinct objects:

Combinations

A combination is a selection of r objects from n distinct objects, order not important:

Permutations with Nondistinct Items

If some items are not distinct:

• , where are counts of each type.

Simulation in Probability

Using Simulation to Obtain Probabilities

Simulation is used to model random processes and estimate probabilities by generating outcomes repeatedly. It is also used in data collection to ensure randomness in sampling and assignment.

Choosing Probability and Counting Methods

Probability Rule Selection

Use flowcharts to determine the appropriate probability rule based on the nature of the events (equally likely, empirical, subjective, disjoint, independent, etc.).

Counting Technique Selection

Use flowcharts to select the appropriate counting technique (multiplication rule, permutations, combinations, etc.) based on whether order matters and whether objects are distinct.

Additional info: These notes expand on brief textbook points to provide definitions, examples, formulas, and visual aids for key probability concepts, rules, and methods. All included images directly reinforce the adjacent explanations and are strictly relevant to the statistical content.