A priori probability

An a priori probability is a probability that is derived purely by deductive reasoning.[1] One way of deriving a priori probabilities is the principle of indifference, which has the character of saying that, if there are N mutually exclusive and collectively exhaustive events and if they are equally likely, then the probability of a given event occurring is 1/N. Similarly the probability of one of a given collection of K events is K / N.

Not to be confused with prior probability.

One disadvantage of defining probabilities in the above way is that it applies only to finite collections of events.

In Bayesian inference, "uninformative priors" or "objective priors" are particular choices of a priori probabilities.[2] Note that "prior probability" is a broader concept.

Similar to the distinction in philosophy between a priori and a posteriori, in Bayesian inference a priori denotes general knowledge about the data distribution before making an inference, while a posteriori denotes knowledge that incorporates the results of making an inference.[3]

A priori probability in statistical mechanics

The a priori probability has an important application in statistical mechanics. The classical version is defined as the ratio of the number of elementary events (e.g. the number of times a die is thrown) to the total number of events - and these considered purely deductively, i.e. without any experimenting. In the case of the die if we look at it on the table without throwing it, each elementary event is reasoned deductively to have the same probability — thus the probability of each outcome of an imaginary throwing of the (perfect) die or simply by counting the number of faces is 1/6. Each face of the die appears with equal probability — probability being a measure defined for each elementary event. The result is different if we throw the die twenty times and ask how many times (out of 20) the number 6 appears on the upper face. In this case time comes into play and we have a different type of probability depending on time or the number of times the die is thrown. On the other hand, the a priori probability is independent of time - you can look at the die on the table as long as you like without touching it and you deduce the probability for the number 6 to appear on the upper face is 1/6.

In statistical mechanics, e.g. that of a gas contained in a finite volume ${\displaystyle V}$, both the spatial coordinates ${\displaystyle q_{i}}$ and the momentum coordinates ${\displaystyle p_{i}}$ of the individual gas elements (atoms or molecules) are finite in the phase space spanned by these coordinates. In analogy to the case of the die, the a priori probability is here (in the case of a continuum) proportional to the phase space volume element ${\displaystyle \Delta q\Delta p}$ divided by ${\displaystyle h}$, and is the number of standing waves (i.e. states) therein, where ${\displaystyle \Delta q}$ is the range of the variable ${\displaystyle q}$ and ${\displaystyle \Delta p}$ is the range of the variable ${\displaystyle p}$ (here for simplicity considered in one dimension). In 1 dimension (length ${\displaystyle L}$) this number or statistical weight or a priori weighting is ${\displaystyle L\Delta p/h}$. In customary 3 dimensions (volume ${\displaystyle V}$) the corresponding number can be calculated to be ${\displaystyle V4\pi p^{2}\Delta p/h^{3}}$.[4] In order to understand this quantity as giving a number of states in quantum (i.e. wave) mechanics, recall that in quantum mechanics every particle is associated with a matter wave which is the solution of a Schrödinger equation. In the case of free particles (of energy ${\displaystyle \epsilon ={\bf {p}}^{2}/2m}$) like those of a gas in a box of volume ${\displaystyle V=L^{3}}$ such a matter wave is explicitly

${\displaystyle \psi \propto \sin(l\pi x/L)\sin(m\pi y/L)\sin(n\pi z/L)}$,

where ${\displaystyle l,m,n}$ are integers. The number of different ${\displaystyle (l,m,n)}$ values and hence states in the region between ${\displaystyle p,p+dp,p^{2}={\bf {p}}^{2},}$ is then found to be the above expression ${\displaystyle V4\pi p^{2}dp/h^{3}}$ by considering the area covered by these points. Moreover, in view of the uncertainty relation, which in 1 spatial dimension is

${\displaystyle \Delta q\Delta p\geq h}$,

these states are indistinguishable (i.e. these states do not carry labels). An important consequence is a result known as Liouville's theorem, i.e. the time independence of this phase space volume element and thus of the a priori probability. A time dependence of this quantity would imply known information about the dynamics of the system, and hence would not be an a priori probability.[5] Thus the region

${\displaystyle \Omega$ :={\frac {\Delta q\Delta p}{\int \Delta q\Delta p}},\;\;\;\int \Delta q\Delta p=const.,}

when differentiated with respect to time ${\displaystyle t}$ yields zero (with the help of Hamilton's equations): The volume at time ${\displaystyle t}$ is the same as at time zero. One describes this also as conservation of information.

In the full quantum theory one has an analogous conservation law. In this case, the phase space region is replaced by a subspace of the space of states expressed in terms of a projection operator ${\displaystyle P}$, and instead of the probability in phase space, one has the probability density

${\displaystyle \Sigma$ :={\frac {P}{TrP}},\;\;\;N=TrP=const.,}

where ${\displaystyle N}$ is the dimensionality of the subspace. The conservation law in this case is expressed by the unitarity of the S-matrix. In either case, the considerations assume a closed isolated system. This closed isolated system is a system with (1) a fixed energy ${\displaystyle E}$ and (2) a fixed number of particles ${\displaystyle N}$ in (c) a state of equilibrium. If one considers a huge number of replicas of this system, one obtains what is called a ``microcanonical ensemble´´. It is for this system that one postulates in quantum statistics the ``fundamental postulate of equal a priori probabilities of an isolated system´´. This says that the isolated system in equilibrium occupies each of its accessible states with the same probability. This fundamental postulate therefore allows us to equate the a priori probability to the degeneracy of a system, i.e. to the number of different states with the same energy.

Example

The following example illustrates the a priori probability (or a priori weighting) in (a) classical and (b) quantal contexts.

(a) Classical a priori probability

Consider the rotational energy E of a diatomic molecule with moment of inertia I in spherical polar coordinates ${\displaystyle \theta ,\phi }$ (this means ${\displaystyle q}$ above is here ${\displaystyle \theta ,\phi }$), i.e.

${\displaystyle E={\frac {1}{2I}}\left(p_{\theta }^{2}+{\frac {p_{\phi }^{2}}{\sin ^{2}\theta }}\right).}$

The ${\displaystyle (p_{\theta },p_{\phi })}$-curve for constant E and ${\displaystyle \theta }$ is an ellipse of area

${\displaystyle \oint dp_{\theta }dp_{\phi }=\pi {\sqrt {2IE}}{\sqrt {2IE}}\sin \theta =2\pi IE\sin \theta }$.

By integrating over ${\displaystyle \theta }$ and ${\displaystyle \phi }$ the total volume of phase space covered for constant energy E is

${\displaystyle \int _{0}^{\phi =2\pi }\int _{0}^{\theta =\pi }2I\pi E\sin \theta d\theta d\phi =8\pi ^{2}IE=\oint dp_{\theta }dp_{\phi }d\theta d\phi }$,

and hence the classical a priori weighting in the energy range ${\displaystyle dE}$ is

${\displaystyle \Omega \propto }$ (phase space volume at ${\displaystyle E+dE}$) minus (phase space volume at ${\displaystyle E}$) is given by ${\displaystyle 8{\pi }^{2}IdE.}$

(b) Quantum a priori probability

Assuming that the number of quantum states in a range ${\displaystyle \Delta q\Delta p}$ for each direction of motion is given, per element, by a factor ${\displaystyle \Delta q\Delta p/h}$, the number of states in the energy range dE is, as seen under (a) ${\displaystyle 8\pi ^{2}IdE/h^{2}}$ for the rotating diatomic molecule. From wave mechanics it is known that the energy levels of a rotating diatomic molecule are given by

${\displaystyle E_{n}={\frac {n(n+1)h^{2}}{8\pi ^{2}I}},}$

each such level being (2n+1)-fold degenerate. By evaluating ${\displaystyle dn/dE_{n}=1/(dE_{n}/dn)}$ one obtains

${\displaystyle {\frac {dn}{dE_{n}}}={\frac {8\pi ^{2}I}{(2n+1)h^{2}}},\;\;\;(2n+1)dn={\frac {8\pi ^{2}I}{h^{2}}}dE_{n}.}$

Thus by comparison with ${\displaystyle \Omega }$ above, one finds that the approximate number of states in the range dE is given by the degeneracy, i.e.

${\displaystyle \Sigma \propto (2n+1)dn.}$

Thus the a priori weighting in the classical context (a) corresponds to the a priori weighting here in the quantal context (b). In the case of the one-dimensional simple harmonic oscillator of natural frequency ${\displaystyle \nu }$ one finds correspondingly: (a) ${\displaystyle \Omega \propto dE/\nu }$, and (b) ${\displaystyle \Sigma \propto dn}$ (no degeneracy). Thus in quantum mechanics the a priori probability is effectively a measure of the degeneracy, i.e. the number of states having the same energy.

In the case of the hydrogen atom or Coulomb potential (where the evaluation of the phase space volume for constant energy is more complicated) one knows that the quantum mechanical degeneracy is ${\displaystyle n^{2}}$ with ${\displaystyle E\propto 1/n^{2}}$. Thus in this case ${\displaystyle \Sigma \propto n^{2}dn}$.

A priori probability and distribution functions

In statistical mechanics (see any book) one derives the so-called distribution functions ${\displaystyle f}$ for various statistics. In the case of Fermi–Dirac statistics and Bose–Einstein statistics these functions are respectively

${\displaystyle f_{i}^{FD}={\frac {1}{e^{(\epsilon _{i}-\epsilon _{0})/kT}+1}},\quad f_{i}^{BE}={\frac {1}{e^{(\epsilon _{i}-\epsilon _{0})/kT}-1}}.}$

These functions are derived for (1) a system in dynamic equilibrium (i.e. under steady, uniform conditions) with (2) total (and huge) number of particles ${\displaystyle N=\Sigma _{i}n_{i}}$ (this condition determines the constant ${\displaystyle \epsilon _{0}}$), and (3) total energy ${\displaystyle E=\Sigma _{i}n_{i}\epsilon _{i}}$, i.e. with each of the ${\displaystyle n_{i}}$ particles having the energy ${\displaystyle \epsilon _{i}}$. An important aspect in the derivation is the taking into account of the indistinguishability of particles and states in quantum statistics, i.e. there particles and states do not have labels. In the case of fermions, like electrons, obeying the Pauli principle (only one particle per state or none allowed), one has therefore

${\displaystyle 0\leq f_{i}^{FD}\leq 1,\quad whereas\quad 0\leq f_{i}^{BE}\leq \infty .}$

Thus ${\displaystyle f_{i}^{FD}}$ is a measure of the fraction of states actually occupied by electrons at energy ${\displaystyle \epsilon _{i}}$ and temperature ${\displaystyle T}$. On the other hand, the a priori probability ${\displaystyle g_{i}}$ is a measure of the number of wave mechanical states available. Hence

${\displaystyle n_{i}=f_{i}g_{i}.}$

Since ${\displaystyle n_{i}}$ is constant under uniform conditions (as many particles as flow out of a volume element also flow in steadily, so that the situation in the element appears static), i.e. independent of time ${\displaystyle t}$, and ${\displaystyle g_{i}}$ is also independent of time ${\displaystyle t}$ as shown earlier, we obtain

${\displaystyle {\frac {df_{i}}{dt}}=0,\quad f_{i}=f_{i}(t,{\bf {v}}_{i},{\bf {r}}_{i}).}$

Expressing this equation in terms of its partial derivatives, one obtains the Boltzmann transport equation. How do coordinates ${\displaystyle {\bf {r}}}$ etc. appear here suddenly? Above no mention was made of electric or other fields. Thus with no such fields present we have the Fermi-Dirac distribution as above. But with such fields present we have this additional dependence of ${\displaystyle f}$.

References

1. Mood A.M., Graybill F.A., Boes D.C. (1974) Introduction to the Theory of Statistics (3rd Edition). McGraw-Hill. Section 2.2 (available online Archived 2012-05-15 at the Wayback Machine)
2. E.g. Harold J. Price and Allison R. Manson, "Uninformative priors for Bayes’ theorem" Archived 2013-08-08 at Archive.today, AIP Conf. Proc. 617, 2001
3. Eidenberger, Horst (2014), Categorization and Machine Learning: The Modeling of Human Understanding in Computers, Vienna University of Technology, p. 109, ISBN 9783735761903.
4. H.J.W. Müller-Kirsten, Basics of Statistical Physics, 2nd. ed. World Scientific (Singapore, 2013), Chapter 6
5. A. Ben-Naim, Entropy Demystified, World Scientific (Singapore, 2007)