What are the axioms of probability? Full explanation.

Home » Science » Mathematics » Statistic » What are the axioms of probability? Full explanation.

The question "what are the axioms of probability?" seems simple, but the answer leads to a very solid mathematical construction., which began to be rigorously organized in the 20th century with the work of Andrey Kolmogorov. These axioms are the basis of virtually all modern probability theory, from the study of games of chance to complex statistical models used in data science, finance, and engineering.

Before Kolmogorov formalizationBack then, probability was understood in a more intuitive way, linked to the idea of ​​frequency or chance.And different mathematicians used diverse interpretations. Today, when we talk about axioms of probability, we are referring to a minimum set of rules that any probability function must obey so that we can make coherent calculations, avoid contradictions, and construct powerful theorems.

Basic intuition: random experiences and events

To understand the axioms of probability, the first step is to know what a random experiment is and what we call an event.A random experiment is any procedure whose outcome cannot be predicted with certainty, even though we know all the possible results; classic examples are flipping a coin or rolling a die.

We call the sample space, usually denoted by Ω, the set of all possible outcomes of this experiment.If we toss a coin, for example, the sample space can be written as Ω = {H, T}, where H represents "heads" and T represents "tails". Each element of Ω is called an elementary outcome.

An event is any subset of Ω that we are interested in observing.Thus, if the experiment is the toss of a coin, the set {H} is the event "heads comes up", the set {T} is the event "tails comes up", and Ω itself is the event "heads or tails comes up", that is, a certain event.

Some events are especially important: the impossible event, the elementary event, and the certain event.The empty set ∅ represents the impossible event, since it contains no outcome; a set with a single element {ω}, with ω in Ω, represents an elementary event; and Ω itself is the certain event, the one that always occurs when the experiment is performed.

The language of set theory is very helpful in the study of probability.If A and B are events, then A ∩ B represents the simultaneous occurrence of A and B, A ∪ B represents the occurrence of at least one of them, and the complement of A, often written as ̄A or Ω \ A, represents the “non-occurrence of A”. This notation and the properties of sets will be used directly in the formulation of the axioms.

Interpretations of the concept of probability

Although Kolmogorov's axioms provide the mathematical basis for probability, the word "probability" itself can be interpreted in various ways.Historically, different interpretations have emerged regarding what it means to assign a number P(A) to an event A.

In the classical interpretation of Laplace, valid for finite spaces with equiprobable outcomes, the probability of A is the ratio between the number of favorable cases and the number of possible cases.If the sample space has n equally likely outcomes (that is, #Ω = n) and event A contains n_A of these outcomes (#A = n_A), then the probability is given by P(A) = n_A / n. This formula is quite intuitive when all outcomes have the same chance of occurring.

Yes, the frequentist interpretation It links probability to the relative frequency observed in repetitions of an experiment.From this point of view, we repeat the random experiment n times and count how many times event A occurs, calling this number n_A; then we look at the limit, as n grows, of the fraction n_A / n. The probability of A would be P(A) = lim_{n→∞} (n_A / n), provided that this limit exists.

There is also the subjective interpretation, widely used in Bayesian statistics, in which probability is associated with the degree of belief of a rational subject.In this approach, P(A) quantifies how confident someone is in the occurrence of A, taking into account the available knowledge. It is not experience that "carries" the probability, but the subject who assesses uncertainty coherently.

Despite these different interpretations, they can all coexist within the same axiomatic framework of Kolmogorov.In other words, regardless of whether you prefer a classical, frequentist, or subjective viewpoint, ultimately probability will be mathematically modeled by a function P that obeys a small set of axioms about an event space.

Formal construction: probability spaces and σ-algebras

Kolmogorov described probability in terms of a triple (Ω, F, P), called the probability space.In this triple, Ω is the sample space, F is the set of possible events (technically, a σ-algebra of subsets of Ω), and P is the probability function.

A σ-algebra F is a special collection of subsets of Ω that satisfies some properties.In general terms, F needs to contain the empty set, be closed under complement (if A is in F, then its complement is also in F), and be closed under countable unions (if A₁, A₂, … are in F, then the union of all of them is also in F). This structure ensures that we can work with set operations without leaving the universe of events that have a well-defined probability.

Formally, F is a σ-algebra over Ω when: The empty set ∅ belongs to F; if A is in F, then the complement of A in Ω also belongs to F; and if A₁, A₂, … is a (finite or countably infinite) sequence of elements of F, then the union A₁ ∪ A₂ ∪ … is also in F. In many contexts, F is also called the Borel field or σ-field.

The probability function P is defined on F and assigns to each event E in F a non-negative real number.We then say that P(E) is in ℝ and P(E) ≥ 0 for all E in F. In general measure theory, measures can take on infinite values, but in standard probability theory, P(E) is always finite, which brings some differences in relation to more general measures.

This structure (Ω, F, P) with P(Ω) = 1 is what we call a probability space.The condition P(Ω) = 1 is essential because it represents the idea that, when performing the experiment, some result in Ω certainly occurs; there are no "hidden results" outside the sample space.

Kolmogorov's three axioms

Kolmogorov's axiomatic theory is based on three fundamental axioms that any probability function must satisfy.They are simple to state, but extremely powerful, because virtually all the usual properties of probability are derived from them.

First axiom — Non-negativity: For any event A belonging to the σ-algebra F, we have P(A) ≥ 0. That is, probabilities are never negative. In some more exotic theories, there is talk of "negative probabilities," but these ideas deviate from Kolmogorov's classic framework.

Second axiom — Normalization: The probability of a certain event occurring is equal to 1, that is, P(Ω) = 1. This axiom establishes the convention that 1 corresponds to 100% certainty, and 0 corresponds to impossibility. In more elementary versions, this axiom can also be understood as saying that the sum of the probabilities of all elementary outcomes of Ω is equal to 1.

Third axiom — σ-additivity: If A₁, A₂, … is a sequence of pairwise disjoint (also called mutually exclusive) events, then P(∪ᵢ Aᵢ) = Σᵢ P(Aᵢ). This holds true for both a finite and a countably infinite collection of events. This countable additivity property is the major difference compared to mere finite additivity.

In simpler contexts, some authors work only with finite additivity., requiring that P(A ∪ B) = P(A) + P(B) for disjoint events A and B, and that this extends to a finite number of sets. In this case, it suffices to work with a set algebra, not necessarily a σ-algebra, but the standard approach in modern probability is to require σ-additivity.

It is from this third axiom that several important consequences arise, such as equalities, inequalities, and laws of probability.He is also at the heart of the link between probability and measure theory, which studies measures in sets in a fairly general way.

Properties derived from axioms

Based on Kolmogorov's three axioms, we were able to prove several basic and extremely useful properties.These properties are not assumed beforehand: they are logical consequences of the axioms.

One of the first properties is the monotonicity of probability.If A and B are events in F and A is contained in B (A ⊆ B), then P(A) ≤ P(B). The idea is intuitive: if B encompasses everything that can happen in A, and perhaps more, then B cannot have a lower probability than A.

Another fundamental property is that the probability of an impossible event is zero.From a formal point of view, using σ-additivity, we consider a sequence where E₁ = A, E₂ = B \ A and Eᵢ = ∅ for i ≥ 3, in a scenario where A ⊆ B. Since the Eᵢ are disjoint and their union is B, the sum of the probabilities must converge to P(B). If we assume that P(∅) = a > 0, then the sum of P(∅) infinitely many times would explode to infinity, which is incompatible with finite P(B). Hence we conclude that P(∅) = 0.

Therefore, we can state the inequality 0 ≤ P(E) ≤ 1 for any event E in F.We already knew that P(E) ≥ 0 from the first axiom. Knowing that P(Ω) = 1 and using monotonicity with E ⊆ Ω, it follows that P(E) ≤ P(Ω) = 1. Thus, every probability is always between 0 and 1, inclusive.

A commonly used identity is the so-called additive law for any two events.For events A and B in F, it holds that P(A ∪ B) = P(A) + P(B) − P(A ∩ B). This formula corrects the “double counting” of the common event A ∩ B, which is added twice if we sum P(A) and P(B) without adjustment.

Another important consequence is the relationship between an event and its complement.If we denote the complement of A by ̄A, then P(̄A) = 1 − P(A). This equality conveys the idea that "either A happens or A does not happen," and there is no other possibility within Ω.

From this, it also becomes clear that P(A) = 0 does not necessarily imply that A is the impossible event.In mathematical terms, it is possible for an event to have zero probability without being the empty set (this appears, for example, in continuous spaces), but at the most elementary level, P(A) = 0 is usually associated with practically impossible events.

Practical example: flipping a coin

A classic and very didactic example for visualizing Kolmogorov's axioms is the toss of a coin.Let's assume, to begin with, that the coin can only land on "heads" (H) or "tails" (T), and that these are the only possible outcomes.

We then define the sample space as Ω = {H, T}The possible events form a σ-algebra F composed of {∅, {H}, {T}, {H, T}}. In this context, the impossible event is ∅, the elementary events are {H} and {T}, and the certain event is {H, T}.

From Kolmogorov's axioms, we know that P(∅) = 0 and P(Ω) = 1If we assume that the coin is fair, that is, it does not favor either side, then symmetry suggests that P({H}) = P({T}). Since the sum P({H}) + P({T}) must equal 1, we conclude that both are worth 1/2.

Therefore, the probability of getting "heads or tails" is P({H, T}) = 1The probability of getting "heads" is P({H}) = 1/2 and the probability of getting "tails" is P({T}) = 1/2. The sum of the probabilities of the elementary events exhausts the total probability of the space.

This model, although simple, illustrates how axioms behave in practice and how they prevent inconsistencies in probability calculations.If we do not carefully define the sample space, we can make serious mistakes, because any event is always a subset of Ω; if the subset does not fit into Ω, its probability is not even defined.

Probability in finite and countable spaces

When the sample space is finite or countable, probability can be described in a very concrete way.Suppose that Ω = {ω₁, ω₂, …} is a finite or countable set of possible outcomes.

If A is an event that contains some of these results, such as A = {ω₁*, …, ω_{k*}, …}Therefore, the probability of A can be seen as the sum of the probabilities of the corresponding elementary events: P(A) = P(∪ᵢ {ω_{i*}}) = Σᵢ P({ω_{i*}}). This is a direct application of additivity (or σ-additivity) on disjoint sets.

In the particular case where the sample space is finite, with #Ω = n, and all outcomes are equiprobableWe have P({ωᵢ}) = 1/n for each i. If A contains k distinct outcomes in Ω, then P(A) = Σ_{i=1}^k P({ω_{i*}}) = k/n = (#A)/(#Ω). This is exactly the classic Laplace formula reinterpreted within the modern axiomatic framework.

When the sample space is countably infinite, the sum of the probabilities of the elementary events still needs to converge to 1.That is, Σᵢ P({ωᵢ}) = 1. This is where σ-additivity shows its strength, allowing us to deal not only with finite sums, but also with infinite series of events.

Conditional probability and the role of axioms.

A central aspect of the theory is understanding how probability changes when we know that a certain event has already occurred.This is where conditional probability comes in, usually written as P(A | B), which means "the probability of A given that B has occurred".

The basic formula for conditional probability is P(A | B) = P(A ∩ B) / P(B), provided that P(B) > 0This definition is consistent with Kolmogorov's axioms, and in fact, for each B with P(B) > 0, the function A ↦ P(A | B) again satisfies the three axioms when we restrict the event space to B.

This means that P(· | B) is itself a probability function over the “new” sample space B.As a consequence, all the basic properties hold for conditional probabilities: P(̄A | B) = 1 − P(A | B), P(∅ | B) = 0, conditional monotonicity (if A₁ ⊆ A₂, then P(A₁ | B) ≤ P(A₂ | B)) and the formula P(A₁ ∪ A₂ | B) = P(A₁ | B) + P(A₂ | B) − P(A₁ ∩ A₂ | B).

The important relationship P(A ∩ B) = P(A) P(B | A), when P(A) > 0, also derives from the definition of conditional probability.Symmetrically, we can write P(A ∩ B) = P(B) P(A | B), provided that P(B) > 0. These equalities help to decompose joint probabilities and are the basis for several results, such as Bayes' Theorem.

It is interesting to note that "unconditional" probability can be seen as a particular case of conditional probability.Indeed, we can write P(A) = P(A ∩ Ω) / P(Ω) = P(A | Ω), since P(Ω) = 1. This reinforces the idea that, conceptually, all probability is conditional on some background information, even if it is only the knowledge that we are working within Ω.

Independence of events

Another key concept that relies on axioms is that of the independence between events.Two events A and B are independent if the occurrence of one does not alter the probability of the other.

In formal language, A and B are independent when P(A ∩ B) = P(A) P(B)In terms of conditional probability, this implies that if P(B) > 0, then P(A | B) = P(A), and if P(A) > 0, then P(B | A) = P(B). That is, knowing that B occurred does not change the probability of A, and vice versa.

Every event is independent of the impossible event ∅ and the certain event Ω.For the empty set, P(A ∩ ∅) = 0 and P(∅) = 0, so the relation trivially holds. For the certain event, P(A ∩ Ω) = P(A) and P(Ω) = 1, therefore P(A ∩ Ω) = P(A) P(Ω) = P(A).

A common question is whether two disjoint events can be independent.In general, if A and B are disjoint and both have positive probability, then P(A ∩ B) = 0, but P(A) P(B) > 0, which violates the definition of independence. Thus, in many cases, two disjoint events with non-zero probability are not independent, since the occurrence of one excludes the possibility of the other.

When dealing with more than two events, several different notions of independence arise.We can have pairwise independence, joint independence, and other types. In all these cases, however, the starting point remains the relation P(A ∩ B) = P(A) P(B), based on Kolmogorov's axioms and the definition of conditional probability.

Practical rules and classical laws of probability

Beyond their formal properties, axioms allow for the formulation of more operational laws, useful in the daily work of those who perform probability calculations.One of them is the so-called law of addition, already mentioned in the form P(A ∪ B) = P(A) + P(B) − P(A ∩ B), which can be extended to a larger number of events through the inclusion-exclusion principle.

Another commonly used rule is the relationship between an event and its "outside" part of another event.For A and B in F, the following holds: P(A ∩ ̄B) = P(A) − P(A ∩ B). This is simply decomposing A into two parts: the part that occurs together with B (A ∩ B) and the part that occurs without B (A ∩ ̄B). These two parts are disjoint, and their union is A, which leads to the previous equality by additivity.

The law of total probability and Bayes' Theorem, although not fully detailed here, also rely directly on the axioms.The law of total probability combines conditional probabilities into a partition of the sample space, while Bayes' Theorem "inverts" conditionals, allowing probabilities to be updated based on new evidence.

In more didactic versions, some easy-to-memorize "practical axioms" are also listed.For example: the maximum probability is 1 (100%); the sum of the probabilities of all elements in the sample space is equal to 1; and the probability of an event X added to the probability of "not X" is always 1. These statements are direct reflections of the formal axioms.

With this set of laws, it becomes possible to solve problems ranging from simple games of chance to sophisticated models with many variables.The great advantage is that, behind all the formulas and calculation tricks, the logical support remains the same axiomatic tripod.

Kolmogorov's axioms of probability provide a rigorous yet flexible basis for dealing with uncertainty.Based on three simple principles—non-negativity, normalization, and σ-additivity—an entire rich theory has been constructed, capable of incorporating classical, frequentist, and subjective interpretations, handling finite or infinite spaces, describing conditional probabilities and independence, and supporting applications in virtually all scientific and technological fields.