Skip to main content

Probability for Machine Learning

· 2 min read

What is Probability​

Blog Reading: https://machinelearningmastery.com/what-is-probability/

  • Uncertainty: Imperfect or incomplete information
  • Probability: A measure that quantifies the likelihood that an event will occur.

It can be calculated by dividing the count of all of the occurrences of the event by the total possible occurrences.

p = occurences / (non-occurrences + occurrences)

Probability Theory​

  • Def: Provides a framework for quantifying uncertainty and making predictions about the likelihood of various outcomes.

Basic Concepts​

  1. Experiment: An action that leads to one or more outcomes. Like rolling a die or flipping a coin.
  2. Sample Space (S): The set of all possible outcomes of the experiment. For dice: 6
  3. Event: A subset of the sample space. An event can consist of one outcome or multiple outcomes. For example, rolling an even number: 6
  4. Probability (P): A measure of the likelihood of an event occurring, expressed as a number between 0 and 1.

Key Principles​

  1. Addition Rule: For two mutually exclusive events A and B, the probability of either A or B occurring can be expressed:
P(A∪B)=P(A)+P(B)
  1. Multiplication Rule: For two independent events A and B, the probability of both A and B happening can be expressed:
P(A∩B)=P(A)×P(B)
  1. Conditional Probability: The probability of an event A given that B has occurred:
P(A∣B)=P(B)P(A∩B)​

Frequentist vs Bayesian Interpretation​

  • Frequentist: Probability is defined as the longrun relative frequency of an event occurring in repeated independent trials. For example, if a coin is flipped many times, the probability of getting heads is interpreted as the limit of the proportion of heads observed as the number of flips approach infinity.

  • Bayesian: Probablility is a measure of belief or certainty about an even, given prior knowledge or evidence. It allows for context or new insights taken from new data.