Probability space Information & Probability space Links at HealthHaven.com

In probability theory, the probability space, or probability triple, is a concept which serves as a rigorous mathematical ground for the conventional idea of randomness. It is a mathematical model of a real-world situation (or “experiment”) where we recognize that certain things occur “at random”.

The model works as following: first, at the outset of the experiment we attempt to envision all possible outcomes which might possibly happen, the set of all such outcomes is called the sample space Ω. Second, we recognize that the elementary outcomes could be too little of practical use, and that the more complicated events, consisting possibly of many different elementary outcomes, are of more interest. The collection of all such events is called the σ-algebra $\scriptstyle \mathcal{F}$ . Third, we have to specify how likely one or another event is going to happen, which is done using the probability measure function P. These three components $\scriptstyle (\Omega,\; \mathcal{F},\; P)$ together constitute the probability space — hence the name “triple”.

Once the probability space is established, it is assumed that the “nature” makes its move and selects a single outcome ω from the sample space Ω. Then we say that all events from $\scriptstyle \mathcal{F}$ which contained the selected outcome ω (recall that each event is just a subset of Ω) “have had occurred”. The selection performed by the nature is done in such a way that if we were to repeat the experiment infinite number of times, the relative frequencies of occurrence of each of the events would have coincided with the probabilities prescribed by the function P.

The notion of probability space together with other axioms of probability was introduced by the prominent Soviet mathematician Andrey Kolmogorov in the 1930s. Nowadays alternative approaches for axiomatization of probability theory exist, see for example “Algebra of random variables”.

This article is concerned with the mathematics of manipulating probabilities. There are several alternative views of what "probability" means and how it should be interpreted that are outlined in the article probability interpretations. In addition, there have been attempts to construct theories for quantities which are notionally similar to probabilities but do not obey all their rules: see for example Free probability, Possibility theory, Negative probability and Quantum probability.

[edit] Introduction

A probability space presents a model for a given class of real-world situations, and therefore like with other models, the choice of the constituting elements Ω, $\scriptstyle \mathcal{F}$ , and P is ultimately up to the author of the model.

The sample space Ω is a set of elementary events — random outcomes which may potentially happen: states of nature, possibilities, outcomes of an experiment etc. Every instance of the real-world situation or run of the experiment must produce exactly one elementary event. If outcomes of different runs of an experiment are different in any way that matters, they represent different elementary events. What differences matter depends, of course, on the kind of analysis we want to do. This leads to different choices of sample space.

The σ-algebra $\scriptstyle \mathcal{F}$ . is a collection of all events (not necessarily elementary) we would like to consider. Here, an "event" is a set of zero or more elementary events, i.e., a subset of the sample space. An event is considered to have "happened" when the outcome of a trial is a member of the event. Since the same elementary event may be a member of many events, it is possible for many events to have happened in a single outcome of a trial. For example, when the trial consists of throwing two dice, outcomes with a sum of 7 pips may constitute an event. Outcomes with an odd number of pips may constitute another event. If the outcome of a trial is the elementary event of two pips on the first die and five on the second, then both of the events of "7 pips" and "odd number of pips" have happened.

The probability measure P is a function which tells us the probabilities of each event. Each probability is a number between zero (the event will not happen in any trial) and one (the event will surely happen in every trial). Thus P is a function $\scriptstyle P:\ \mathcal{F} \rightarrow [0,1]$ . The probability measure function must satisfy a simple requirement: the probability of a union of two disjoint events must be equal to the sum of probabilities of each of these events. For example, if two events are Heads and Tails, then the probability of Heads-or-Tails must be equal to the sum of probabilities for Heads and Tails).

Not every subset of the sample space Ω must necessarily be considered an event: Some of the subsets are simply uninteresting, others cannot be “measured”. This is not so obvious in a case like a coin toss. In a different example, one could consider javelin throw lengths, where the events typically are intervals like "between 60 and 65 meters" and unions of such intervals, but not "irrational numbers between 60 and 65 meters"

[edit] Definition

In short, a probability space is a measure space such that the measure of the whole space is equal to one.

The expanded definition is following: a probability space is a triple $\scriptstyle (\Omega,\; \mathcal{F},\; P)$ consisting of:

the sample space Ω — an arbitrary non-empty set,
the σ-algebra ⊆ 2^Ω (also called σ-field) — a set of subsets of Ω, called events, such that:
- $\scriptstyle \mathcal{F}$ contains an empty set: ∅∈ $\scriptstyle \mathcal{F}$ ,
- $\scriptstyle \mathcal{F}$ is closed under complements: if A∈ $\scriptstyle \mathcal{F}$ , then also (Ω∖A)∈ $\scriptstyle \mathcal{F}$ ,
- is closed under countable unions: if A_i∈ for i=1,2,…, then also (∪_iA_i)∈
  - The corollary from the previous two properties and the De Morgan’s law is that $\scriptstyle \mathcal{F}$ is also closed under countable intersections: if A_i∈ $\scriptstyle \mathcal{F}$ for i=1,2,…, then also (∩_iA_i)∈ $\scriptstyle \mathcal{F}$
the probability measure P: →[0,1] — a function on such that:
- P is countably additive: if {A_i}⊆ $\scriptstyle \mathcal{F}$ is a countable collection of pairwise disjoint sets, then P(⊔A_i) = ∑P(A_i), where “⊔” denotes the disjoint union,
- the measure of entire sample space is equal to one: P(Ω) = 1.

[edit] Discrete case

Discrete probability theory needs only at most countable sample spaces Ω, which makes the foundations much less technical. Probabilities can be ascribed to points of Ω by the probability mass function p: Ω→[0,1] such that ∑_ω∈Ω p(ω) = 1. All subsets of Ω can be treated as events (thus, $\scriptstyle \mathcal{F}$ = 2^Ω is the power set). The probability measure takes the simple form

$(*) \qquad P(A) = \sum_{\omega\in A} p(\omega) \quad \text{for all } A \subseteq \Omega \, .$

The greatest σ-algebra $\scriptstyle \mathcal{F}$ = 2^Ω describes the complete information. In general, a σ-algebra $\scriptstyle \mathcal{F}$ ⊆ 2^Ω corresponds to a finite or countable partition Ω = B₁ ⊔ B₂ ⊔ …, the general form of an event A ∈ $\scriptstyle \mathcal{F}$ being A = B_k₁ ⊔ B_k₂ ⊔ … (here ⊔ means the disjoint union.) See also the examples.

The case p(ω) = 0 is permitted by the definition, but rarely used, since such ω can safely be excluded from the sample space.

[edit] General case

If Ω is uncountable, still, it may happen that p(ω) ≠ 0 for some ω; such ω are called atoms. They are an at most countable (maybe, empty) set, whose probability is the sum of probabilities of all atoms. If this sum is equal to 1 then all other points can safely be excluded from the sample space, returning us to the discrete case. Otherwise, if the sum of probabilities of all atoms is less than 1 (maybe 0), then the probability space decomposes into a discrete (atomic) part (maybe empty) and a non-atomic part.

[edit] Non-atomic case

If p(ω) = 0 for all ω∈Ω then equation (∗) fails: the probability of a set is not the sum over its elements, which makes the theory much more technical. Initially the probabilities are ascribed to some “generator” sets (see the examples). Then a limiting procedure allows to ascribe probabilities to sets that are limits of sequences of generator sets, or limits of limits, and so on. All these sets are the σ-algebra $\scriptstyle \mathcal{F}$ . For technical details see Caratheodory’s extension theorem. Sets belonging to $\scriptstyle \mathcal{F}$ are called measurable. In general they are much more complicated than generator sets, but much better than non-measurable sets.

[edit] Examples

[edit] Discrete examples

[edit] Example 1

If the experiment consists of just one flip of a perfect coin, then the outcomes are either heads or tails: Ω = {H, T}. The σ-algebra $\scriptstyle \mathcal{F}$ = 2^Ω contains 2² = 4 events, namely: {H} – “heads”, {T} – “tails”, {} – “neither heads nor tails”, and {H,T} – “either heads or tails”. So, $\scriptstyle \mathcal{F}$ = {{}, {H}, {T}, {H,T}}. There is a fifty percent chance of tossing heads, and fifty percent for tails. Thus the probability measure in this example is P({}) = 0, P({H}) = 0.5, P({T}) = 0.5, P({H,T}) = 1.

[edit] Example 2

The fair coin is tossed three times. There are 8 possible outcomes: Ω = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT} (here “HTH” for example means that first time the coin landed heads, the second time tails, and the last time heads again). The complete information is described by the σ-algebra $\scriptstyle \mathcal{F}$ = 2^Ω of 2⁸ = 256 events, where each of the events is a subset of Ω.

Alice knows the outcome of the second toss only. Thus her incomplete information is described by the partition Ω = A₁ ⊔ A₂ = {HHH, HHT, THH, THT} ⊔ {HTH, HTT, TTH, TTT}, and the corresponding σ-algebra $\scriptstyle \mathcal{F}$ _Alice = {{}, A₁, A₂, Ω}. Bob knows only the total number of tails. His partition contains four parts: Ω = B₀ ⊔ B₁ ⊔ B₂ ⊔ B₃ = {HHH} ⊔ {HHT, HTH, THH} ⊔ {TTH, THT, HTT} ⊔ {TTT}; accordingly, his σ-algebra $\scriptstyle \mathcal{F}$ _Bob contains 2⁴ = 16 events.

The two σ-algebras are incomparable: neither $\scriptstyle \mathcal{F}$ _Alice ⊆ $\scriptstyle \mathcal{F}$ _Bob nor $\scriptstyle \mathcal{F}$ _Bob ⊆ $\scriptstyle \mathcal{F}$ _Alice; both are sub-σ-algebras of 2^Ω.

[edit] Example 3

If 100 voters are to be drawn randomly from among all voters in California and asked whom they will vote for governor, then the set of all sequences of 100 Californian voters would be the sample space Ω. We assume that sampling without replacement is used: only sequences of 100 different voters are allowed. For simplicity an ordered sample is considered, that is a sequence {Alice, Bob} is different from {Bob, Alice}. We also take for granted that each potential voter knows exactly his future choice, that is he/she doesn’t choose randomly.

Alice knows only whether or not Arnold Schwarzenegger has received at least 60 votes. Her incomplete information is described by the σ-algebra $\scriptstyle \mathcal{F}$ _Alice that contains: (1) the set of all sequences in Ω where at least 60 people vote for Schwarzenegger; (2) the set of all sequences where fewer than 60 vote for Schwarzenegger; (3) the whole sample space Ω; and (4) the empty set ∅.

Bob knows the exact number of voters who are going to vote for Schwarzenegger. His incomplete information is described by the corresponding partition Ω = B₀ ⊔ B₁ … ⊔ B₁₀₀ (though some of these sets may be empty, depending on the Californian voters…) and the σ-algebra $\scriptstyle \mathcal{F}$ _Bob consists of 2¹⁰¹ events.

In this case Alice’s σ-algebra is a subset of Bob’s: $\scriptstyle \mathcal{F}$ _Alice ⊂ $\scriptstyle \mathcal{F}$ _Bob. The Bob’s σ-algebra is in turn the subset of the much larger “complete information” σ-algebra 2^Ω consisting of 2^{n(n−1)…(n−99)} events, where n is the number of all potential voters in California.

[edit] Non-atomic examples

[edit] Example 4

A number between 0 and 1 is chosen at random, uniformly. Here Ω = [0,1], $\scriptstyle \mathcal{F}$ is the σ-algebra of Borel sets on Ω, and P is the Lebesgue measure on [0,1].

In this case the open intervals of the form (a,b), where 0<a<b<1, could be taken as the generator sets. Each such set can be ascribed the probability of P((a,b)) = (b−a), which generates the Lebesgue measure on [0,1], and the Borel σ-algebra on Ω.

[edit] Example 5

A fair coin is tossed endlessly. Here one can take Ω = {0,1}^∞, the set of all infinite sequences of numbers 0 and 1. Cylinder sets {(x₁,x₂,…)∈Ω: x₁=a₁, …, x_n=a_n} may be used as the generator sets. Each such set describes an event in which the first n tosses have resulted in a fixed sequence (a₁, …, a_n), and the rest of the sequence may be arbitrary. Each such event can be naturally given the probability of 2⁻ⁿ.

These two non-atomic examples are closely related: a sequence (x₁,x₂,…) ∈ {0,1}^∞ leads to the number 2⁻¹x₁ + 2⁻²x₂ + … ∈ [0,1]. This is not a one-to-one correspondence between {0,1}^∞ and [0,1] however: it is an isomorphism modulo zero, which allows for treating the two probability spaces as two forms of the same probability space. In fact, all non-pathologic non-atomic probability spaces are the same in this sense.

[edit] Related concepts

[edit] Probability distribution

Any probability distribution defines a probability measure.

[edit] Random variables

A random variable X is a measurable function X: Ω→S from the sample space Ω to another measurable space S called the state space.

The notation Pr(X∈A) is a commonly used shorthand for P({ω∈Ω: X(ω)∈A}).

[edit] Defining the events in terms of the sample space

If Ω is countable we almost always define $\scriptstyle \mathcal{F}$ as the power set of Ω, i.e $\scriptstyle \mathcal{F}$ = 2^Ω which is trivially a σ-algebra and the biggest one we can create using Ω. We can therefore omit ℱ and just write (Ω,P) to define the probability space.

On the other hand, if Ω is uncountable and we use $\scriptstyle \mathcal{F}$ = 2^Ω we get into trouble defining our probability measure P because $\scriptstyle \mathcal{F}$ is too “large”, i.e. there will often be sets to which it will be impossible to assign a unique measure, giving rise to problems like the Banach–Tarski paradox. In this case, we have to use a smaller σ-algebra $\scriptstyle \mathcal{F}$ , for example the Borel algebra of Ω, which is the smallest σ-algebra that makes all open sets measurable.

[edit] Conditional probability

Kolmogorov’s definition of probability spaces gives rise to the natural concept of conditional probability. Every set A with non-zero probability (that is, P(A) > 0) defines another probability measure

$P(B | A) = {P(B \cap A) \over P(A)}$

on the space. This is usually pronounced as the “probability of B given A”.

For any event B such that P(B) > 0 the function Q defined by Q(A) = P(A|B) for all events A is itself a probability measure.

[edit] Independence

Two events, A and B are said to be independent if P(A∩B)=P(A)P(B).

Two random variables, X and Y, are said to be independent if any event defined in terms of X is independent of any event defined in terms of Y. Formally, they generate independent σ-algebras, where two σ-algebras G and H, which are subsets of F are said to be independent if any element of G is independent of any element of H.

[edit] Mutual exclusivity

Two events, A and B are said to be mutually exclusive or disjoint if P(A∩B) = 0. (This is weaker than A∩B = ∅, which is the definition of disjoint for sets).

If A and B are disjoint events, then P(A∪B) = P(A) + P(B). This extends to a (finite or countably infinite) sequence of events. However, the probability of the union of an uncountable set of events is not the sum of their probabilities. For example, if Z is a normally distributed random variable, then P(Z=x) is 0 for any x, but P(Z∈R) = 1.

The event A∩B is referred to as “A and B”, and the event A∪B as “A or B”.

[edit] See also

[edit] Bibliography

Pierre Simon de Laplace (1812) Analytical Theory of Probability

The first major treatise blending calculus with probability theory, originally in French: Théorie Analytique des Probabilités.

Andrei Nikolajevich Kolmogorov (1950) Foundations of the Theory of Probability

The modern measure-theoretic foundation of probability theory; the original German version (Grundbegriffe der Wahrscheinlichkeitrechnung) appeared in 1933.

Harold Jeffreys (1939) The Theory of Probability

An empiricist, Bayesian approach to the foundations of probability theory.

Edward Nelson (1987) Radically Elementary Probability Theory

Discrete foundations of probability theory, based on nonstandard analysis and internal set theory. downloadable. http://www.math.princeton.edu/~nelson/books.html

Patrick Billingsley: Probability and Measure, John Wiley and Sons, New York, Toronto, London, 1979.
Henk Tijms (2004) Understanding Probability

A lively introduction to probability theory for the beginner, Cambridge Univ. Press.

David Williams (1991) Probability with martingales

An undergraduate introduction to measure-theoretic probability, Cambridge Univ. Press.

Gut, Allan (2005). Probability: A Graduate Course. Springer. ISBN 0387228330.
Sazonov, V.V. (2001), "Probability space", in Hazewinkel, Michiel, Encyclopaedia of Mathematics, Kluwer Academic Publishers, ISBN 978-1556080104, http://eom.springer.de/P/p074960.htm
Weisstein, Eric W., "Probability space" from MathWorld.

[edit] External links

Virtual Laboratories in Probability and Statistics (principal author Kyle Siegrist), especially, Probability Spaces

Citizendium

Contents