5 Definitions of Probability
First, we need to define the concept of probability. This is not a trivial task because there are different approaches to define the concept of probability. Several definitions of probability may be found in the literature ( Zellner (1983)):
Classical definition (De Moivre, Laplace): If there are n equally likely possible alternatives, and an event E can happen in m ways out of n, then the probability of occurrence of E (i.e. success) is P(E) = m/n. This definition is circular since it defines probability in terms of itself (i.e. “equally likely” could be interpreted as “equally probable”).
Venn Limit definition (Richard von Mises): If an event occurs a large number of times, then the probability of p is the limit of the ratio of the number of times when p will be true to the whole number of trials when the number of trials tends to infinity. The problem in this definition is the fact that an actual limiting number may not exist.
Hypothetical Infinite Population definition (R. A. Fisher): An actually infinite number of possible trials are assumed. Then the probability of p is defined as the ratio of the number of cases where p is true to the whole number.
Jeffreys’ definition: Probability is the degree of confidence that we may reasonably have in a proposition. This definition is quite vague but is used in Jeffreys’ axiom system. This enables us to derive an axiomatic probability theory.
Value of Expectation definition: If for an individual the utility of the uncertain outcome of getting a sum of s dollars or zero dollars is the same as getting a sure payment of one dollar, the probability of the uncertain outcome of getting s dollars is defined to be u(1)/u(s), where u(.) is a utility function. If u(.) can be taken proportional to returns, the probability of receiving s is 1/s.
Probability theory is commonly formalized with a probability space \((\Omega, \mathcal{F}, \text{P})\), where \(\Omega\) is the sample space (all possible outcomes), \(\mathcal{F}\) is the set of events (subsets of \(\Omega\)), and \(\text{P}\) is the probability measure. In this notation, the probability of the sample space itself equals one, i.e. \(\text{P}(\Omega)=1\).
On the other hand, the chance of one specific event occurring will always lie between 0 and 1. We denote this as
\[ 0 \leq \text{P}(Event_1) \leq 1 \] (where P is used as a symbol for Probability, and Event_1 is a specific event in our space). Also, the probability of non-occurrence (i.e. failure) is denoted by
\[ \text{P}(\text{not} Event_1) = 1 - \text{P}(Event_1) \]
Thus, the sum of success and failure of a specific event equals one:
\[ \text{P}(Event_1) + \text{P}(\text{not} Event_1) = 1 \]
Probability theory is often formalized with set notation where events are represented as sets. The following definitions are commonly used and facilitate nomenclature:
5.1 Negation
\[ \neg A = \text{not} A \]
5.2 Intersection or Conjunction
\[ A \cap B \]
which means A and B
5.3 Union or Disjunction
\[ A \cup B \]
which means A or B
5.4 Multiple Intersection or Conjunction
\[ A \cap B \cap C \cap D … \]
means A and B and C and D etc…
5.5 Multiple Union or Disjunction
\[ A \cup B \cup C \cup D … \]
means A or B or C or D etc…
5.6 Exclusiveness
Propositions \(E_i\) (for \(i=1,2,3,…,n\)) are exclusive on data \(D\) if only one of these propositions can be true given data \(D\), or if none of them is true given \(D\). If exactly one proposition \(E_i\) (for \(i=1,2,3,…,n\)) is true, given \(D\), these propositions are said to be exclusive and exhaustive.
Furthermore, it is very important to keep in mind that the union of several mutually exclusive events of the space equals the sum of the individual probabilities of each event. This can symbolically be written as:
\[ \begin{gather*} \text{P} \left( Event_1 \cup Event_2 \cup Event_3 \cup … \cup Event_n \right) \\ = \\ \text{P}(Event_1) + \text{P}(Event_2) + \text{P}(Event_3) + … + \text{P}(Event_n) \\ \Updownarrow \\ \text{all events are mutually exclusive} \end{gather*} \]
As in the description of exclusiveness and exhaustiveness, probabilities may exist in a conditional form. Let \(C\) and \(X\) denote two events then
\(\text{P}(C | X) =\) the probability of \(C\) given that \(X\) is true
\(\text{P}(C) =\) the probability of \(C\), regardless of whether \(X\) is true or not
\(\text{P}(C | \neg X) =\) the probability of \(C\) given that X is not true
The conditional probability of \(C\) given \(X\) is defined (whenever \(\text{P}(X) \neq 0\)) as
\[ \begin{equation} \text{P}(C | X) = \frac{\text{P}(CX)}{\text{P}(X)} \text{, if } \text{P}(X) \neq 0 \end{equation} \tag{5.1}\]
where \(\text{P}(C X)\) is the probability of \(C\) and \(X\) occurring at the same time.
If the validity of \(X\) does not have any influence on \(C\), this implies that \(C\) is independent of \(X\), and then
\[ \text{P}(C | X) = \text{P}(C). \]
In this special case it follows that \(\text{P}(CX) = \text{P}(C) \text{P}(X)\) (because \(C\) and \(X\) are independent events).
Hence, the generalized form of Equation 5.1 becomes
\[ \text{P}(E_1 E_2 E_3 … E_n) = \text{P}(E_1) \text{P}(E_2 | E_1) \text{P}(E_3 | E_1 E_2) … \text{P}(E_n | E_1 E_2 E_3 … E_{n-1}) \]
and if all events are independent
\[ \text{P}(E_1 E_2 E_3 … E_n) = \text{P}(E_1) \text{P}(E_2) \text{P}(E_3) … \text{P}(E_n) \]