Lecture Notes on Statistical Mechanics and Thermodynamics

Universität Leipzig

Instructor: Prof. Dr. S. Hollands

1 Introduction and Historical Overview ..... 3
2 Basic Statistical Notions ..... 7
2.1 Probability Theory and Random Variables ..... 7
2.2 Entropy ..... 11
2.3 Examples of probability distributions ..... 12
2.3.1 Gaussian distribution ..... 13
2.3.2 Binomial and Poisson distribution ..... 13
2.3.3 Ising model ..... 14
2.3.4 Site percolation ..... 15
2.3.5 Random walk on a lattice ..... 17
2.4 Ensembles in Classical Mechanics ..... 18
2.5 Ensembles in Quantum Mechanics (Statistical Operators / Density Matrices) ..... 24
3 Time-evolving ensembles ..... 29
3.1 Boltzmann Equation in Classical Mechanics ..... 29
3.2 Boltzmann Equation, Approach to Equilibrium in Quantum Mechanics ..... 36
4 Equilibrium Ensembles ..... 39
4.1 Generalities ..... 39
4.2 Micro-Canonical Ensemble ..... 39
4.2.1 Micro-Canonical Ensemble in Classical Mechanics ..... 39
4.2.2 Microcanonical Ensemble in Quantum Mechanics ..... 46
4.2.3 Mixing entropy of the ideal gas ..... 49
4.3 Canonical Ensemble ..... 51
4.3.1 Canonical Ensemble in Quantum Mechanics ..... 51
4.3.2 Canonical Ensemble in Classical Mechanics ..... 54
4.3.3 Equidistribution Law and Virial Theorem in the Canonical Ensemble ..... 57
4.4 Grand Canonical Ensemble ..... 61
4.5 Summary of different equilibrium ensembles ..... 64
4.6 Approximation methods ..... 65
4.6.1 The cluster expansion ..... 65
4.6.2 Peierls contours ..... 67
5 The Ideal Quantum Gas ..... 71
5.1 Hilbert Spaces, Canonical and Grand Canonical Formulations ..... 71
5.2 Degeneracy pressure for free fermions ..... 78
5.3 Spin Degeneracy ..... 81
5.4 Black Body Radiation ..... 83
5.5 Degenerate Bose Gas ..... 87
6 The Laws of Thermodynamics ..... 91
6.1 The Zeroth Law ..... 92
6.2 The First Law ..... 94
6.3 The Second Law ..... 99
6.4 Cyclic processes ..... 102
6.4.1 The Carnot Engine ..... 102
6.4.2 General Cyclic Processes ..... 107
6.4.3 The Diesel Engine ..... 109
6.5 Thermodynamic potentials ..... 110
6.6 Chemical Equilibrium ..... 116
6.7 Phase Co-Existence and Clausius-Clapeyron Relation ..... 118
6.8 Osmotic Pressure ..... 123
A Dynamical Systems and Approach to Equilibrium ..... 125
A. 1 The Master Equation ..... 125
A. 2 Properties of the Master Equation ..... 127
A. 3 Relaxation time vs. ergodic time ..... 129
A. 4 Monte Carlo methods and Metropolis algorithm ..... 132
A. 5 Eigenstate thermalization ..... 134
B Exercises ..... 137
B. 1 Exercises for chapter 2 ..... 137
B. 2 Exercises for chapter 3 ..... 141
B. 3 Exercises for chapter 4 ..... 143
B. 4 Exercises for chapter 5 ..... 149
B. 5 Exercises for chapter 6 ..... 152

List of Figures

1.1 Boltzmann's tomb with his famous entropy formula engraved at the top ..... 4
2.1 Graphical expression for the first four moments. ..... 10
2.2 Sketch of a well-potential

W

. ..... 19
2.3 Evolution of a phase space volume under the flow map

Φ_{t}

..... 20
2.4 Sketch of the situation described in the proof of Poincaré recurrence. ..... 23
3.1 Classical scattering of particles in the "fixed target frame". ..... 31
3.2 Pressure on the walls due to the impact of particles. ..... 33
3.3 Sketch of the air-flow across a wing. ..... 34
4.1 Gas in a piston maintained at pressure

P

. ..... 43
4.2 The joint number of states for two systems in thermal contact. ..... 45
4.3 Number of states with energies lying between

E - Δ E

and

E

. ..... 48
4.4 Two gases separated by a removable wall. ..... 49
4.5 A small system in contact with a large heat reservoir. ..... 51
4.6 Distribution and velocity of stars in a galaxy. ..... 59
4.7 Sketch of a potential

V

of a lattice with a minimum at

Q_{0}

. ..... 59
4.8 A small system coupled to a large heat and particle reservoir. ..... 61
4.9 A Peierls contour ..... 68
5.1 The potential

V (\vec{r})

ocurring in (5.46). ..... 81
5.2 Lowest-order Feynman diagram for photon-photon scattering in Quantum Electrodynamics. ..... 84
5.3 Sketch of the Planck distribution for different temperatures. ..... 86
6.1 The triple point of ice water and vapor in the

(P, T)

phase diagram ..... 93
6.2 A large system divided into subsystems I and II by an imaginary wall. ..... 94
6.3 Change of system from initial state

i

to final state

f

along two different paths. ..... 94
6.4 A curve

γ : [0, 1] \to R^{2}

. ..... 95
6.5 Sketch of the submanifolds

A

. ..... 99
6.6 Adiabatics of the ideal gas ..... 101
6.7 Carnot cycle for an ideal gas. The solid lines indicate isotherms and the dashed lines indicate adiabatics. ..... 104
6.8 The Carnot cycle in the

(T, S)

-diagram. ..... 106
6.9 A generic cyclic process in the (

T, S

)-diagram. ..... 107
6.10 A generic cyclic process divided into two parts by an isotherm at temper- ature

T_{I}

. ..... 108
6.11 The process describing the Diesel engine in the

(P, V)

-diagram. ..... 109
6.12 The phase boundary between solution and a solute. ..... 119
6.13 Imaginary phase diagram for the case of 6 different phases. At each point on a phase boundary which is not an intersection point,

φ = 2

phases are supposed to coexist. At each intersection point

φ = 4

phases are supposed to coexist. ..... 120
6.14 Phase boundary of a vapor-solid system in the

(P, T)

-diagram ..... 122

Chapter 1 Introduction and Historical Overview

As the name suggests, thermodynamics historically developed as an attempt to understand phenomena involving heat. This notion is intimately related to irreversible processes involving typically many, essentially randomly excited, degrees of freedom. The proper understanding of this notion as well as the 'laws' that govern it took the better part of the

19^{th}

century. The basic rules that were, essentially empirically, observed were clarified and laid out in the so-called "laws of thermodynamics". These laws are still useful today, and will, most likely, survive most microscopic models of physical systems that we use.

Before the laws of thermodynamics were identified, other theories of heat were also considered. A curious example from the

17^{th}

century is a theory of heat proposed by J. Becher. He put forward the idea that heat was carried by special particles he called

scientists such as A.L. de Lavoisier

^{2}

, who showed that the existence of such a particle did not explain, and was in fact inconsistent with, the phenomenon of burning, which he instead correctly associated also with chemical processes involving oxygen. Heat had already previously been associated with friction, especially through the work of B. Thompson, who showed that in this process work (mechanical energy) is converted to heat. That heat transfer can generate mechanical energy was in turn exemplified through the steam engine as developed by inventors such as J. Watt, J. Trevithick, and

T. Newcomen - the key technical invention of the

18^{th}

and

19^{th}

century. A broader theoretical description of processes involving heat transfer was put forward in 1824 by N.L.S. Carnot, who emphasized in particular the importance of the notion of equilibrium. The quantitative understanding of the relationship between heat and energy was found by J.P. Joule and R. Mayer, who were the first to state clearly that heat is a form of energy. This finally lead to the principle of conservation of energy put forward by H . von Helmholtz in 1847.

Parallel to this largely phenomenological view of heat, there were also early attempts to understand this phenomenon from a microscopic angle. This viewpoint seems to have been first stated in a transparent fashion by D. Bernoulli in 1738 in his work on hydrodynamics, in which he proposed that heat is transferred from regions with energetic molecules (high internal energy) to regions with less energetic molecules (low energy). The microscopic viewpoint ultimately lead to the modern 'bottom up' view of heat by J.C. Maxwell, J. Stefan and especially L. Boltzmann. According to Boltzmann, heat is associated with a quantity called "entropy" which increases in irreversible processes. In the context of equilibrium states, entropy can be understood as a measure of the number of accessible states at a defined energy according to his famous formula

S = k_{B} \log W (E),

which Planck had later engraved in Boltzmann's tomb on Wiener Zentralfriedhof:

Figure 1.1: Boltzmann's tomb with his famous entropy formula engraved at the top.

The formula thereby connects a macroscopic, phenomenological quantity

S

to the microscopic states of the system (counted by

W (E) =

number of accessible states of energy

E

). His proposal to relate entropy to counting problems for microscopic configurations and thereby to ideas from probability theory was entirely new and ranks as one of the major intellectual accomplishments in Physics. The systematic understanding of the relationship between the distributions of microscopic states of a system and
macroscopic quantities such as

S

is the subject of statistical mechanics. That subject nowadays goes well beyond the original goal of understanding the phenomenon of heat but is more broadly aimed at the analysis of systems with a large number of, typically interacting, degrees of freedom and their description in an "averaged", or "statistical", or "coarse grained" manner. As such, statistical mechanics has found an ever growing number of applications to many diverse areas of science, such as

Neural networks and other networks
Financial markets
Data analysis and mining
Astronomy
Black hole physics
and many more. Here is an, obviously incomplete, list of some key innovations in the subject:

Timeline

$17^{th}$ century:

Ferdinand II, Grand Duke of Tuscany: Quantitative measurement of temperature

18^{th}

century:
A.Celsius, C. von Linné: Celsius temperature scale
A.L. de Lavoisier: basic calometry
D. Bernoulli: basics of kinetic gas theory
B. Thompson (Count Rumford): mechanical energy can be converted to heat

19^{th}

century:
1802 J. L. Gay-Lussac: heat expansion of gases
1824 N.L.S.Carnot: thermodynamic cycles and heat engines

1847 H. von Helmholtz: energy conservation (1

^{st}

law of thermodynamics)
1848 W. Thomson (Lord Kelvin): definition of absolute thermodynamic temperature scale based on Carnot processes

1850 W. Thomson and H. von Helmholtz: impossibility of a perpetuum mobile (2

2^{nd}

law)
1857 R. Clausius: equation of state for ideal gases

1860 J.C. Maxwell: distribution of the velocities of particles in a gas
1865 R.Clausius: new formulation of

2^{nd}

law of thermodynamics, notion of entropy
1877 L. Boltzmann:

S = k_{B} \log W

1876 (as well as 1896 and 1909) controversy concerning entropy, Poincaré recurrence is not compatible with macroscopic behavior

1894 W. Wien: black body radiation

20^{th}

century:
1900 M. Planck: radiation law

\to

Quantum Mechanics
1911 P. Ehrenfest: foundations of Statistical Mechanics
1924 Bose-Einstein statistics
1925 Fermi-Pauli statistics
1931 L. Onsager: theory of irreversible processes
1937 L. Landau: phase transitions, later extended to superconductivity by Ginzburg
1930's W. Heisenberg, E. Ising, R. Peierls,... : spin models for magnetism
1943 S. Chandrasekhar, R.H. Fowler: applications of statistical mechanics in astrophysics

1956 J. Bardeen, L.N. Cooper, J.R. Schrieffer: explanation of superconductivity
1956-58 L. Landau: theory of Fermi liquids
1960's T. Matsubara, E. Nelson, K. Symanzik,... : application of Quantum Field Theory methods to Statistical Mechanics

1970's L. Kadanoff, K.G. Wilson, W. Zimmermann, F. Wegner,...: renormalization group methods in Statistical Mechanics

1973 J. Bardeen, B. Carter, S. Hawking, J. Bekenstein, R.M. Wald, W.G. Unruh,....: laws of black hole mechanics, Bekenstein-Hawking entropy

1975 - Neural networks

1985 - Statistical physics in economy

Chapter 2 Basic Statistical Notions

2.1 Probability Theory and Random Variables

Statistical mechanics is an intrinsically probabilistic description of a system, so we do not ask questions like "What is the velocity of the

N^{th}

particle?" but rather questions of the sort "What is the probability for the

N^{th}

particle having velocity between

v

and

v + Δ v

?" in an ensemble of particles. Thus, basic notions and manipulations from probability theory can be useful, and we now introduce some of these, without any attention paid to mathematical rigor.

A random variable $x$ can have different outcomes forming a set $Ω = {x_{1}, x_{2}, \dots}$ , e.g. for tossing a coin $Ω_{coin} = {$ head,tail $}$ or for a dice $Ω_{dice} = {1, 2, 3, 4, 5, 6}$ , or for the velocity of a particle $Ω_{velocity} = {\vec{v} = (v_{x}, v_{y}, v_{z}) \in R^{3}}$ , etc.
An event is a subset $E \subset Ω$ (not all subsets need to be events).
A probability measure is a map that assigns a number $P (E)$ to each event, subject to the following general rules:
(i) $P (E) ⩾ 0$ .
(ii) $P (Ω) = 1$ .
(iii) If $E \cap E^{'} = \emptyset \Rightarrow P (E \cup E^{'}) = P (E) + P (E^{'})$ .

In mathematics, the data

(Ω, P, {E})

is called a probability space and the above axioms basically correspond to the axioms for such spaces. For instance, for a fair dice the probabilities would be

P_{dice} ({1}) = \dots = P_{dice} ({6}) = \frac{1}{6}

and

E

would be any subset of

{1, 2, 3, 4, 5, 6}

. In practice, probabilities are determined by repeating the experiment (independently) many times, e.g. throwing the dice very often. Thus, the "empirical definition" of the probability of an event

E

\begin{matrix} (2.1) & P (E) = lim_{N \to \infty} \frac{N_{E}}{N}, \end{matrix}

where

N_{E} =

number of times

E

occurred, and

N =

total number of experiments. For one real variable

x \in Ω \subset R

, it is common to write the probability of an event

E \subset R

formally as

\begin{matrix} (2.2) & P (E) = \int_{E} p (x) d x \end{matrix}

Here,

p (x)

is the probability density "function", defined formally by:

" p (x) d x = P ((x, x + d x)) "

The axioms for

p

formally imply that we should have

\int_{- \infty}^{\infty} p (x) d x = 1, 0 ⩽ p (x) ⩽ \infty

A mathematically more precise way to think about the quantity

p (x) d x

is provided by measure theory, i.e. we should really think of

p (x) d x = d μ (x)

as defining a measure and of

{E}

as the corresponding collection of measurable subsets. A typical case is that

p

is a smooth (or even just integrable) function on

R

and that

d x

is the Lebesgue measure, with

E

from the set of all Lebesgue measurable subsets of

R

. However, we can also consider more pathological cases, e.g. by allowing

p

to have certain singularities. It is possible to define "singular" measures

d μ

relative to the Lebesgue measure

d x

which are not writable as

p (x) d x

and

p

an integrable function which is non-negative almost everywhere. An example of this is the Dirac measure, which is formally written as

\begin{matrix} (2.3) & p (x) = \sum_{i = 1}^{N} p_{i} δ (x - y_{i}), \end{matrix}

where

p_{i} ⩾ 0

and

\sum_{i} p_{i} = 1

. Nevertheless, we will, by abuse of notation, stick with the informal notation

p (x) d x

. We can also consider several random variables, such as

x = (x_{1}, \dots, x_{N}) \in Ω = R^{N}

. The probability density function would now be - again formally - a function

p (x) ⩾ 0

R^{N}

with total integral of 1 .

Of course, as the example of the coin shows, one can and should also consider discrete probability spaces such as

Ω = {1, \dots, N}

, with the events

E

being all possible subsets. For the elementary event

{n}

the probability

p_{n} = P ({n})

is then a non-negative number and

\sum_{i} p_{i} = 1

. The collection of

{p_{1}, \dots, p_{N}}

completely characterizes the probability distribution.

Let us collect some standard notions and terminology associated with probability spaces:

The expectation value $⟨ F (x) ⟩$ of a function $R^{N} \supset Ω ∋ x \mapsto F (x) \in R$ ("observable") of a random variable is

\begin{matrix} (2.4) & ⟨ F (x) ⟩ := \int_{Ω} F (x) p (x) d^{N} x . \end{matrix}

Here, the function

F (x)

should be such that this expression is actually well-defined, i.e.

F

should be integrable with respect to the probability measure

d μ = p (x) d^{N} x

The moments $m_{n}$ of a probability density function $p$ of one real variable $x \in Ω = R$ are defined by

\begin{matrix} (2.5) & m_{n} := ⟨ x^{n} ⟩ = \int_{- \infty}^{\infty} x^{n} p (x) d x \end{matrix}

Note that it is not automatically guaranteed that the moments are well-defined, and the same remark applies to the expressions given below. The probability distribution

p

can be reconstructed from the moments under certain conditions. This is known as the "Hamburger moment problem".

The characteristic function $\tilde{p}$ of a probability density function of one real variable is its Fourier transform, defined as

\begin{matrix} (2.6) & \tilde{p} (k) = \int_{- \infty}^{\infty} d x e^{- i k x} p (x) = ⟨ e^{- i k x} ⟩ = \sum_{n = 0}^{\infty} \frac{(- i k)^{n}}{n!} ⟨ x^{n} ⟩ \end{matrix}

The Fourier inversion formula gives

\begin{matrix} (2.7) & p (x) = \frac{1}{2 π} \int_{- \infty}^{\infty} d k e^{i k x} \tilde{p} (k) \end{matrix}

The cumulants ${⟨ x^{n} ⟩}_{c}$ are defined via

\begin{matrix} (2.8) & \log \tilde{p} (k) = \sum_{n = 1}^{\infty} \frac{(- i k)^{n}}{n!} {⟨ x^{n} ⟩}_{c} . \end{matrix}

The first four are given in terms of the moments by

\begin{aligned} ⟨ x ⟩_{c} & = ⟨ x ⟩ \\ {⟨ x^{2} ⟩}_{c} & = ⟨ x^{2} ⟩ - ⟨ x ⟩^{2} = ⟨ (x - ⟨ x ⟩)^{2} ⟩ \\ {⟨ x^{3} ⟩}_{c} & = ⟨ x^{3} ⟩ - 3 ⟨ x^{2} ⟩ ⟨ x ⟩ + 2 ⟨ x ⟩^{3} \\ {⟨ x^{4} ⟩}_{c} & = ⟨ x^{4} ⟩ - 4 ⟨ x^{3} ⟩ ⟨ x ⟩ - 3 {⟨ x^{2} ⟩}^{2} + 12 ⟨ x^{2} ⟩ ⟨ x ⟩^{2} - 6 ⟨ x ⟩^{4} . \end{aligned}

There is an important combinatorial scheme relating moments to cumulants. The result expressed by this combinatorial scheme is called the linked cluster theorem, and a variant of it will appear when we discuss the cluster expansion. In order to state and illustrate the content of the linked cluster theorem, we represent the first four moments graphically as follows:

\begin{aligned} ⟨ x ⟩ = . \\ ⟨ x^{2} ⟩ = ⊙ + ⊙ ⊙ \\ ⟨ x^{3} ⟩ =∵ + 3 ≑ + Θ \\ ⟨ x^{4} ⟩ =∵ + 4 ∵ ⊙ + 3 ≑ \end{aligned}

Figure 2.1: Graphical expression for the first four moments.

A blob indicates a connected moment, also called 'cluster'. The linked cluster theorem states that the numerical coefficients in front of the various terms can be obtained by finding the number of ways to break points into clusters of this type. A proof of the linked cluster theorem can be obtained as follows: we write

\begin{matrix} (2.9) & \sum_{0}^{\infty} \frac{(- i k)^{m}}{m!} ⟨ x^{m} ⟩ = e^{\sum_{n = 1}^{\infty} \frac{(- i k)}{n!} {⟨ x^{n} ⟩}_{c}} = \prod_{n = 1}^{\infty} \sum_{i_{n}}^{'} [\frac{(- i k)^{n i_{n}}}{i_{n}!} {(\frac{{⟨ x^{n} ⟩}_{c}}{n!})}^{i_{n}}] \end{matrix}

from which we conclude that

\begin{matrix} (2.10) & ⟨ x^{m} ⟩ = \sum_{{i_{n}}} m! \prod_{n} \frac{{⟨ x^{n} ⟩}_{c}^{i_{n}}}{i_{n}! (n!)^{i_{n}}}, \end{matrix}

where

\sum^{'}

is restricted to

\sum n i_{n} = m

. The claimed graphical expansion follows because

\frac{m!}{\prod_{n} i_{n}! (n!)^{i_{n}}}

is the number of ways to break

m

points into clusters characterized by the numbers

{i_{n}}

, where

i_{n}

is the number of clusters with

n

points.

Let

p

be a probability density on the space

Ω = Ω_{A} \times Ω_{B} = {x = (x_{A}, x_{B}) : x_{A} \in

Ω_{A}, x_{B} \in Ω_{B}}

. If the density is

p

Ω

factorized, as in

\begin{matrix} (2.11) & p (x_{A}, x_{B}) = p_{A} (x_{A}) p_{B} (x_{B}), \end{matrix}

then we say that the variables

x_{A}

and

x_{B}

are independent. If

F_{A}

is an observable for

x_{A}

and

F_{B}

an observable for

x_{B}

, then for independent random variables

x_{A}, x_{B}

, one has

\begin{matrix} (2.12) & ⟨ F_{A} F_{B} ⟩ = ⟨ F_{A} ⟩ ⟨ F_{B} ⟩ \end{matrix}

and one says that

A

and

B

are uncorrelated.
The notion of independence can be generalized immediately to any "Cartesian product"

Ω = Ω_{1} \times \dots \times Ω_{N}

of probability spaces. In the case of independent identically distributed real random variables

x_{i}, i = 1, \dots, N

, there is an important theorem characterizing the limit as

N \to \infty

, which is treated in more detail in problem B.1. Basically it says that (under certain assumptions about

p

) the random variable

y = \frac{\sum (x_{i} - μ)}{\sqrt{N}}

has Gaussian distribution for large

N

with mean 0 and spread

σ / \sqrt{N}

. Thus, in this sense, a sum of a large number of arbitrary random variables is approximately distributed as a Gaussian random variable. This so called "Central Limit Theorem" explains, in some sense, the empirical evidence that the random variables appearing in various applications are distributed as Gaussians.

2.2 Entropy

An important quantity associated with a probability distribution is its "information entropy". Let

{p_{i}}

be a probability distribution for a random variable taking values in

Ω = {x_{1}, \dots, x_{N}}

. If the probability

p_{i}

for finding

x_{i}

is very small, then we should be surprised that

x_{i}

occurred. A measure of surprise for the event

x_{i}

\begin{matrix} (2.13) & surprise at seeing event x_{i} = \log \frac{1}{p_{i}} \end{matrix}

because (i) the surprise is larger the smaller

p_{i}

and (ii) the surprise of independent events (see above) should be additive, thus we should take a log. The average surprise is

\begin{matrix} (2.14) & average surprise = ⟨ \log \frac{1}{p_{i}} ⟩ = - \sum_{i} p_{i} \log p_{i} \end{matrix}

This average surprise is defined to be the "information entropy":

Definition: Let

Ω = {x_{1}, \dots, x_{N}}

and let

{p_{i}}

be a probability distribution. The quan-
tity

\begin{matrix} (2.15) & S_{\inf} ({p_{i}}) := - k_{B} \sum_{i} p_{i} \log p_{i} \end{matrix}

is called information entropy. (Our convention is that

0 \log 0 = 0

and

\log

is the natural logarithm).

The factor

k_{B}

is merely inserted here to be consistent with the conventions in statistical physics. In the context of computer science, it is dropped, and the natural log is replaced by the logarithm with base 2 , which is natural to use if we think of information encoded in bits. The information entropy is also defined with the opposite sign sometimes, since more entropy means less information. It can be shown that the information entropy (in computer science normalization) is roughly equal to the average (with respect to the given probability distribution) number of yes/no questions necessary to determine whether a given event has occurred (cf. problem B.4).

Maximum entropy principle: A practical application of information entropy is as follows: suppose one has an ensemble whose probability distribution

{p_{i} : i = 1, \dots, n}

is not completely known. One would like to make a good guess about

{p_{i}}

based on some partial information such as a finite number of moments, or other observables. Thus, suppose that

F_{A} (x), A = 1, 2, \dots, m

are

m < n

observables for which

⟨ F_{A} (x) ⟩ = f_{A}

are known. Then a good guess, representing in some sense a minimal bias about

{p_{i}}

, is to maximize

S_{\inf}

, subject to the constraints

⟨ F_{A} (x) ⟩ = f_{A}

. In the case when the observables are the mean value

μ

and variance

σ

, the distribution obtained in this way is the Gaussian. So the Gaussian is, in this sense, our best guess if we only know

μ

and

σ

(cf. problem B.3).

The maximum entropy principle is typically analyzed with the help of Lagrange multipliers. For the constraint

\sum_{i} p_{i} = 1

we take a multiplier

μ

and for the other constraints

\sum_{i} p_{i} F_{A} (x_{i}) = f_{A}

we take multipliers

λ_{A}

. Then we should first look at the unconstrained maximization problem

ϕ (p_{1}, \dots, p_{n}) = S (p_{1}, \dots, p_{n}) + μ (\sum_{i} p_{i} - 1) + \sum_{A} λ_{A} (\sum_{i} p_{i} F_{A} (x_{i}) - f_{A}) \to

maximum
The constraints are linear in the

p_{i}

and

S ({p_{i}})

is a concave function. Thus a stationary point that is a solution to the constraints is automatically a maximum.

2.3 Examples of probability distributions

We next give some important examples of probability distributions and related models in statistical mechanics:

2.3.1 Gaussian distribution

The Gaussian distribution for one real random variable

x \in Ω = R

is defined by the following probability density:

\begin{matrix} (2.17) & p (x) = \frac{1}{\sqrt{2 π} σ} e^{- \frac{(x - μ)^{2}}{2 σ^{2}}} \end{matrix}

We find

μ = ⟨ x ⟩

and

σ^{2} = ⟨ x^{2} ⟩ - ⟨ x ⟩^{2} = ⟨ x ⟩_{c}^{2}

. The higher moments are all expressible in terms of

μ

and

σ

in a systematic fashion. For example:

\begin{aligned} ⟨ x^{2} ⟩ = σ^{2} + μ^{2} \\ ⟨ x^{3} ⟩ = 3 σ^{2} μ + μ^{3} \\ ⟨ x^{4} ⟩ = 3 σ^{4} + 6 σ^{2} μ^{2} + μ^{4} \end{aligned}

The generating functional for the moments is

⟨ e^{- i k x} ⟩ = e^{- i k μ} e^{- σ^{2} k^{2} / 2}

. The

N

-dimensional generalization of the Gaussian distribution

(Ω = R^{N})

is expressed in terms of a "covariance matrix",

C

, which is symmetric, real, with positive eigenvalues and a vector

\vec{μ}

. It is

\begin{matrix} (2.18) & p (\vec{x}) = \frac{1}{(2 π)^{N / 2} (\det C)^{1 / 2}} e^{- \frac{1}{2} (\vec{x} - \vec{μ}) \cdot C^{- 1} (\vec{x} - \vec{μ})} . \end{matrix}

The first two moments are

⟨ x_{i} ⟩ = μ_{i}, ⟨ x_{i} x_{j} ⟩ = C_{i j} + μ_{i} μ_{j}

, so

C_{i j}

is the generalization of the variance and

μ_{i}

that of the mean value.

2.3.2 Binomial and Poisson distribution

The binomial distribution occurs naturally when we perform a yes/no probability experiment independently a number of times. Fix

N

and let

Ω = {1, \dots, N}

. Then the events are subsets of

Ω

, such as

{n}

. We think of

n

as the number of times an outcome

A

occurs in

N

trials, where

0 ⩽ q ⩽ 1

is the probability for the event

A

\begin{aligned} (2.19) & P_{N} ({n}) = (\binom{N}{n}) q^{n} (1 - q)^{N - n} \\ (2.20) & \Rightarrow {\tilde{p}}_{N} (k) = ⟨ e^{- i k n} ⟩ = {(q e^{- i k} + (1 - q))}^{N} \end{aligned}

The Poisson distribution is the limit of the binomial distribution for

N \to \infty

when

n

is fixed and

q = \frac{α}{N}

, with

α

fixed (rare events). It is given by (

n \in {0, 1, 2, \dots} = Ω

\begin{matrix} (2.21) & p (n) = \frac{α^{n}}{n!} e^{- α}, \end{matrix}

To see this, we write the binomial distribution as

\begin{aligned} p_{N} (x) = \frac{N (N - 1) \dots (N - n + 1)}{n!} \frac{α^{n}}{N^{n}} {(1 - \frac{α}{N})}^{N - n} \\ (2.22) & = \underset{\to 1}{\underset{⏟}{\frac{N (N - 1) \dots (N - n + 1)}{N^{n}}}} \frac{α^{n}}{n!} \underset{\to e^{- α}}{\underset{⏟}{{(1 - \frac{α}{N})}^{N}}} \underset{\to 1}{\underset{⏟}{{(1 - \frac{α}{N})}^{- n}}} \end{aligned}

A standard application of the Poisson distribution is radioactive decay: let

q = λ Δ t

the decay probability in a time interval

Δ t = \frac{T}{N}

. If

n

denotes the number of decays, then the probability is obtained as:

\begin{matrix} (2.23) & p (n) = \frac{(λ T)^{n}}{n!} e^{- λ T} \end{matrix}

2.3.3 Ising model

The Ising model is basically a probability distribution for spins on a lattice. For each lattice site

i

(atom), there is a spin taking values

σ_{i} \in {\pm 1}

. So, an individual spin configuration

C

is a collection

C = {σ_{i}}

of spin values, one for each site. In

d

dimensions, the lattice is usually taken to be a volume

V = [0, L]^{d} \subset Z^{d}

. The number of lattice sites is then

| V | = {L ⌉^{d}

, and the set of possible configurations

C = {σ_{i}}

Ω = {C} = {- 1, 1}^{| V |}

since each spin can take precisely two values. In the Ising model, one assigns to each configuration an energy

\begin{matrix} (2.24) & H (C) = - J \sum_{i k} σ_{i} σ_{k} - b \sum_{i} σ_{i}, \end{matrix}

where

J, b

are parameters, and where the first sum is over all lattice bonds

i k

in the volume

V

. The second sum is over all lattice sites in

V

. The probability of a configuration is then given by the Boltzmann weight

\begin{matrix} (2.25) & p (C) = \frac{1}{Z} \exp [- β H (C)] \end{matrix}

A large coupling constant

J ≫ 1

energetically favors adjacent spins to be parallel and a large

b ≫ 1

favors spins to be preferentially up ( +1 ). The coupling

b

can thus be thought of as an external magnetic field.

Z = Z (V, J, b)

is a normalization constant ensuring that all the probabilities add up to unity. For the computation of

Z

in dimension

d = 1

and for

b = 0

, see problem B.5. Of particular interest in the Ising model are the mean magnetization

m = | V |^{- 1} \sum ⟨ σ_{i} ⟩

and the free energy density

f =^{- 1} | V |^{- 1} \log Z

, see problem B.16. Another quantity of interest is the two-point function

⟨ σ_{i} σ_{j} ⟩

in the limit of large

V \to Z^{d}

(called "thermodynamic limit") and a large separation between

i

and

j

2.3.4 Site percolation

Consider a finite lattice

V = [0, L]^{d} \subset Z^{d}

. Now we occupy each lattice site

i

randomly with a spin

σ_{i} = + 1

with probability

q

and with spin

σ_{i} = - 1

with probability

1 - q

. If we assume that these choices are independent, then the probability of a configuration

C = {σ_{i}}

of spin values is

\begin{matrix} (2.26) & p (C) = q^{N_{+}} (1 - q)^{N_{-}} \end{matrix}

with

N_{\pm}

the number of + or - spins. The probability space is, as before,

Ω = {C}

. The physical interpretation of such a model can be for example that sites occupied by + spins are conducting, whereas those occupied by - spins are not. A cluster is a set of sites containing only + spins such that between any two sites of the cluster there is a path containing only + spins. If we have a long cluster spanning from one side of the lattice to the opposite side, then the lattice is a conductor, otherwise it is an insulator.

Interesting observables are, for example:

$n_{s} (C)$ : The number of clusters of size $s$ in $C$ , i.e., the number of such clusters divided by the number of all lattice sites (i.e., $L^{d}$ ).
$χ_{i} (C)$ : This is one if $i$ belongs to a cluster in $C$ and zero otherwise.
$ξ (C)$ : The average over all sizes of clusters in $C$ , etc.

We can then form expectation values as usual, for instance

\begin{matrix} (2.27) & ⟨ n_{s} ⟩ = \sum_{C} n_{s} (C) p (C) = average normalized number of clusters of size s, \end{matrix}

s ⟨ n_{s} ⟩

is the probability that an arbitrarily chosen site belongs to a cluster of size

s

. We can also consider

\begin{matrix} (2.28) & N = ⟨ \sum_{s = 1}^{\infty} n_{s} ⟩ = average normalized number of clusters. \end{matrix}

The probability that a site occupied by

a + spin

belongs to a cluster of size

s

\begin{matrix} (2.29) & \frac{s ⟨ n_{s} ⟩}{\sum_{s^{'} = 1}^{\infty} s^{'} ⟨ n_{s^{'}} ⟩} . \end{matrix}

Consequently, the mean size of the clusters is

\begin{matrix} (2.30) & S = \sum_{s = 1}^{\infty} s \frac{s ⟨ n_{s} ⟩}{\sum_{s^{'} = 1}^{\infty} s^{'} ⟨ n_{s^{'}} ⟩} \end{matrix}

Physically, one is interested in the question how, for example,

S

depends on

q

. Can the clusters become macroscopically large for sufficiently large

q

? If this is the case,
then the material is conducting, otherwise it is an insulator and the transition from one behavior to another is thought of as a "phase transition". To study this question more precisely, one likes to take

L \to \infty

and asks whether

S

can diverge as

q

goes to some critical value

q_{*}

, i.e. when

q \to q_{*}

. Near such a critical value, one expects for instance

\begin{matrix} (2.31) & S \propto {(q - q_{*})}^{- γ} \end{matrix}

and calls

γ

a "critical exponent". Other examples of critical exponents can also be defined. For instance, let

P

be the probability that a given " + " site belongs to an infinite cluster. This number can be computed as

\begin{matrix} (2.32) & P = 1 - \frac{1}{q} \sum_{s = 1}^{\infty} s ⟨ n_{s} ⟩ \end{matrix}

where the limit

L \to \infty

is again understood in the end. To see this, consider an arbitrary lattice site. It either has spin - or it has spin + and therefore belongs to some finite cluster of size

s

or an infinite cluster. This means

1 = 1 - q + \sum_{s = 1}^{\infty} s ⟨ n_{s} ⟩ + q P

, which gives the statement. Near the critical value

q_{*}

we then likewise expect

\begin{matrix} (2.33) & P \propto {\begin{cases} 0 & for q < q_{*} \\ {(q - q_{*})}^{β} & for q ⩾ q_{*} \end{cases} \end{matrix}

The numbers

γ, β

and similar other quantities are called "critical exponents". Analogous quantities can be defined for many systems, and their determination is one of the standard problems in statistical mechanics, since, for example in this model, the appearance of a macroscopic cluster is the sign of change of a macroscopic property of the system (conductance vs. insulator in this model). The importance of the values of the exponents is that we may expect them to be independent of the precise nature of the model at the microscopic level. For instance, we would expect to get the same values if we replace a cubic lattice by some other lattice which is not too different.

Unfortunately, the determination of critical points and critical exponents is a very complicated business, and we will not have the time to introduce the methods for doing this in this lecture course. By way of an example, let us treat the trivial case

d = 1

(chain) of the percolation model, which can be treated by elementary means. In this example, we can immediately calculate the normed cluster number

⟨ n_{s} ⟩

: On the one hand, as we have already noted, the probability that an arbitrarily chosen site

i

belongs to a cluster of size

s

is given by

s ⟨ n_{s} ⟩

. On the other hand, it equals

s q^{s} (1 - q)^{2}

because

s

consecutive sites have to carry spin + (factor

q^{s})

, the left and right boundaries of the cluster have to be occupied by

- (

factor

(1 - q)^{2})

, and there are

s

clusters of size

s

containing the chosen site

i

(factor

s

). Thus, we conclude that

\begin{matrix} (2.34) & ⟨ n_{s} ⟩ = q^{s} (1 - q)^{2} \end{matrix}

As long as there is no infinite cluster, we also have

\sum_{s = 1}^{\infty} s ⟨ n_{s} ⟩ = q

, because the probabilities that an arbitrarily chosen site belongs to a cluster of size

s

add up to the probability that this site is occupied by a + spin. As a consequence, our formula for

S

gives

\begin{aligned} S & = \frac{1}{q} \sum_{s = 1}^{\infty} s^{2} ⟨ n_{s} ⟩ \\ = \frac{1}{q} \sum_{s = 1}^{\infty} s^{2} q^{s} (1 - q)^{2} \\ (2.35) & = \frac{(1 - q)^{2}}{q} {(q \frac{d}{d q})}^{2} \sum_{s = 1}^{\infty} q^{s} \\ = \frac{(1 - q)^{2}}{q} {(q \frac{d}{d q})}^{2} \frac{q}{1 - q} \\ = \frac{1 + q}{1 - q} . \end{aligned}

So we read off that

q_{*} = 1

and that

γ = 1

for

d = 1

2.3.5 Random walk on a lattice

A walk

ω

in a volume

V

of a lattice as in the Ising model can be characterized by the sequence of sites

ω = (x, i_{1}, i_{2}, \dots, i_{N - 1}, y)

encountered by the walker, where

x

is the fixed beginning and

y

the fixed endpoint. The number of sites in the walk is denoted

l (ω) (= N + 1

in the example), and the number of self-intersections is denoted by

n (ω)

. The set of walks from

x

y

is our probability space

Ω_{x, y}

, and a natural probability distribution is

\begin{matrix} (2.36) & P (ω) = \frac{1}{Z} e^{- μ l (ω) - g n (ω)} . \end{matrix}

Here,

μ, g

are positive constants. For large

μ ≫ 1

, short walks between

x

and

y

are favored, and for large

g ≫ 1

, self-avoiding walks are favored.

Z = Z_{x, y} (V, μ, g)

is a normalization constant ensuring that the probabilities add up to unity. Of interest are e.g. the "free energy density"

f = | V |^{- 1} \log Z

, or the average number of steps the walk spends in a given subset

S \subset V

, given by

⟨ # {S \cap ω} ⟩

In general, such observables are very difficult to calculate, but for

g = 0

(unconstrained walks) there is a nice connection between

Z

and the Gaussian distribution, which is the starting point to obtain many further results. Let

\partial_{α} f (i) = f (i + {\vec{e}}_{α}) - f (i)

be the "lattice partial derivative" of a function

f (i)

defined on the lattice sites

i \in V

, in the direction of the

α

-th unit vector,

{\vec{e}}_{α}, α = 1, \dots, d

. Let

\sum \partial_{α}^{2} = Δ

be the "lattice

Laplacian". The lattice Laplacian can be identified with a matrix

Δ_{i j}

of size

| V | \times | V |

defined by

Δ f (i) = \sum_{j} Δ_{i j} f (j)

. Define the covariance matrix as

C = {(- Δ + m^{2})}^{- 1}

and consider the corresponding Gaussian measure for the variables

{ϕ_{i}} \in R^{| V |}

(one real variable per lattice site in

V

). One shows that

\begin{matrix} (2.37) & Z_{x, y} = ⟨ ϕ_{x} ϕ_{y} ⟩ \equiv \frac{1}{(2 π)^{| V | / 2} (\det C)^{1 / 2}} \int ϕ_{x} ϕ_{y} e^{- \frac{1}{2} \sum ϕ_{i} {(- Δ + m^{2})}_{i j} ϕ_{j}} d^{| V |} ϕ_{ϕ} \end{matrix}

for

g = 0, μ = \log (2 d + m^{2})

(exercise).

2.4 Ensembles in Classical Mechanics

The basic ideas of probability theory outlined in the previous sections can be used for the statistical description of systems obeying the laws of classical mechanics. Consider a classical system of

N

particles, described by

6 N

phase space coordinates

^{1}

which we abbreviate as

\begin{matrix} (2.38) & (P, Q) = ({\vec{p}}_{1}, \dots, {\vec{p}}_{N}; {\vec{x}}_{1}, \dots, {\vec{x}}_{N}) \in R^{(3 + 3) N} = Ω \end{matrix}

A classical ensemble is simply a probability density function

ρ (P, Q)

, i.e.

\begin{matrix} (2.39) & \int_{Ω} ρ (P, Q) d^{3 N} P d^{3 N} Q = 1, 0 ⩽ ρ (P, Q) ⩽ \infty . \end{matrix}

According to the basic concepts of probability theory, the ensemble average of an observable

F (P, Q)

is then simply

\begin{matrix} (2.40) & ⟨ F (P, Q) ⟩ = \int_{Ω} F (P, Q) ρ (P, Q) d^{3 N} Q d^{3 N} P \end{matrix}

The probability distribution

ρ (P, Q)

represents our limited knowledge about the system which, in reality, is of course supposed to be described by a single trajectory

(P (t), Q (t))

in phase space. In practice, we cannot know what this trajectory is precisely other than for a very small number of particles

N

and, in some sense, we do not really want to know the precise trajectory at all. The idea behind ensembles is rather that the time evolution (=phase space trajectory

(Q (t), P (t)))

typically scans the entire accessible phase space (or sufficiently large parts of it) such that the time average of

F

equals the ensemble average of

F

, i.e. in many cases we expect to have:

\begin{matrix} (2.41) & lim_{T \to \infty} \frac{1}{T} \int_{0}^{T} F (P (t), Q (t)) d t = ⟨ F (P, Q) ⟩ \end{matrix}

for a suitable (stationary) probability density function. This is closely related to the "ergodic theorem" which in turn is related to the fact that the equations of motion are derivable from a (time independent) Hamiltonian. Hamilton's equations are
$$

\begin{matrix} (2.42) & {\dot{x}}_{i α} = \frac{\partial H}{\partial p_{i α}} {\dot{p}}_{i α} = - \frac{\partial H}{\partial x_{i α}}, \end{matrix}

w h e r e $ i = 1, \dots, N $ a n d $ α = 1, 2, 3 $ . T h e H a m i l t o n i a n $ H $ i s t y p i c a l l y o f t h e f o r m

\begin{matrix} (2.43) & H = \underset{kinetic energy}{\underset{⏟}{\sum_{i} \frac{{\vec{p}}_{i}^{2}}{2 m}}} + \underset{interaction}{\underset{⏟}{\sum_{i < j} V ({\vec{x}}_{i} - {\vec{x}}_{j})}} + \underset{external potential}{\underset{⏟}{\sum_{j} W ({\vec{x}}_{j})}} \end{matrix}

$$
if there are no internal degrees of freedom. It is a standard theorem in classical mechanics that

E = H (P, Q)

is conserved under time evolution. Let us imagine a well-potential

W (\vec{x})

as in the following picture:

Figure 2.2: Sketch of a well-potential

W

Then

Ω_{E} = {(P, Q) ∣ H (P, Q) = E}

is compact for sufficiently large

W_{0}

. We call

Ω_{E}

the energy surface. The "hyper"-area of this surface is denoted by

| Ω_{E} |

, so

\begin{matrix} (2.44) & | Ω_{E} | \equiv \int_{Ω_{E}} d S \equiv \int_{Ω} δ (H (P, Q) - E) d^{3 N} P d^{3 N} Q \end{matrix}

Particle trajectories do not leave this surface by energy conservation. If Hamilton's equations admit other constants of motion, then it is natural to define a corresponding surface with respect to all constants of motion.

An important feature of the dynamics given by Hamilton's equations is

Liouville's Theorem: The flow map

Φ_{t} : (P, Q) \mapsto (P (t), Q (t))

is area-preserving.

Figure 2.3: Evolution of a phase space volume under the flow map

Φ_{t}

Proof of the theorem: Let

(P^{'}, Q^{'}) = (P (t), Q (t))

, such that

(P (0) = P, Q (0) = Q)

. Then we have

\begin{matrix} (2.45) & d^{3 N} P^{'} d^{3 N} Q^{'} = \frac{\partial (P^{'}, Q^{'})}{\partial (P, Q)} d^{3 N} P d^{3 N} Q \end{matrix}

and we would like to show that

J_{P, Q} (t) = 1

for all

t

. Let us write the Jacobian as

J_{P, Q} (t) = \frac{\partial (P^{'}, Q^{'})}{\partial (P, Q)}

. Since the flow evidently satisfies

Φ_{t + t^{'}} (P, Q) = Φ_{t^{'}} (Φ_{t} (P, Q))

, the chain rule and the properties of the Jacobian imply

J_{P, Q} (t + t^{'}) = J_{P, Q} (t) J_{P^{'}, Q^{'}} (t^{'})

. We now show that

\partial J_{P, Q} / \partial t (0) = 0

. For small

t

, we can expand as follows:

\begin{aligned} P^{'} = P + t \dot{P} + O (t^{2}) = P - t \frac{\partial H}{\partial Q} + O (t^{2}) \\ Q^{'} = Q + t \dot{Q} + O (t^{2}) = Q + t \frac{\partial H}{\partial P} + O (t^{2}) \end{aligned}

It follows that

\begin{aligned} J_{P, Q} (t) & = \frac{\partial (P^{'}, Q^{'})}{\partial (P, Q)} \\ = \det [(\begin{array}{cc} 1_{3 N \times 3 N} & 0 \\ 0 & 1_{3 N \times 3 N} \end{array}) + t (\begin{array}{cc} - \partial_{P} \partial_{Q} H & - \partial_{Q}^{2} H \\ \partial_{P}^{2} H & \partial_{Q} \partial_{P} H \end{array}) + O (t^{2})] \\ = 1 + ttr (\begin{array}{cc} - \partial_{P} \partial_{Q} H & - \partial_{Q}^{2} H \\ \partial_{P}^{2} H & \partial_{Q} \partial_{P} H \end{array}) + O (t^{2}) \\ = 1 + t (\underset{= 0}{\underset{⏟}{- \sum_{α, i} \frac{\partial^{2} H}{\partial x_{i α} \partial p_{i α}} + \sum_{α, i} \frac{\partial^{2} H}{\partial p_{i α} \partial x_{i α}}}}) + O (t^{2}) \\ = 1 + O (t^{2}) . \end{aligned}

(Here we used

\det (I + t A) = 1 + t tr A + O (t^{2})

for any matrix A.) This implies

\begin{matrix} (2.46) & \partial J_{P, Q} / \partial t (0) = lim_{t \to 0} \frac{1}{t} (J_{P, Q} (t) - J_{P, Q} (0)) = lim_{t \to 0} \frac{1}{t} O (t^{2}) = 0 \end{matrix}

using

J_{P, Q} (0) = 0

. The functional equation for the Jacobian then implies that the time derivative vanishes for arbitrary

t

\begin{matrix} (2.47) & \frac{\partial}{\partial t} J_{P, Q} (t) = {\frac{\partial}{\partial t^{'}} J_{P, Q} (t + t^{'}) |}_{t^{'} = 0} = {J_{P, Q} (t) \frac{\partial}{\partial t^{'}} J_{P^{'}, Q^{'}} (t^{'}) |}_{t^{'} = 0} = 0 \end{matrix}

Together with

J_{P, Q} (0) = 1

, this gives the result

J_{P, Q} (t) = 1

for all

t

, i.e. the flow is area-preserving.

If we start with a classical ensemble, described by a probability distribution

ρ (P, Q)

on phase space, then its time evolution is defined as

\begin{matrix} (2.48) & ρ (P, Q; t) \equiv ρ (P (- t), Q (- t)) \equiv ρ_{t} (P, Q) \end{matrix}

where

(P (t), Q (t))

are the phase space trajectories. Using our notation for the flow map, we could also write this as

\begin{matrix} (2.49) & ρ_{t} (P, Q) = ρ (Φ_{- t} (P, Q)) \end{matrix}

The reason for the "-" is as follows: The probability to be at

(P, Q)

at time

t

should be given by the probability for having been at

Φ_{- t} (P, Q)

at time 0 . Note that the time evolution of observables (corresponding to the Heisenberg picture in Quantum Mechanics) is defined oppositely:

\begin{matrix} (2.50) & F_{t} (P, Q) = F (P (t), Q (t)) \end{matrix}

Using Liouville's theorem, one then easily checks that

⟨ F ⟩_{ρ_{t}} = {⟨ F_{t} ⟩}_{ρ}

.
Differentiating (2.49) with respect to

t

gives

\begin{matrix} (2.51) & \frac{\partial}{\partial t} ρ_{t} (P, Q) = - \frac{\partial ρ_{t} (P, Q)}{\partial P} \underset{= - \frac{\partial H}{\partial Q}}{\underset{⏟}{\frac{\partial P}{\partial t}}} - \frac{\partial ρ_{t} (P, Q)}{\partial Q} \underset{= \frac{\partial H}{\partial P}}{\underset{⏟}{\frac{\partial Q}{\partial t}}} = {H, ρ_{t}} (P, Q), \end{matrix}

where

{\cdot, \cdot}

denotes the Poisson bracket. An ensemble is called stationary if

ρ_{t}

remains constant, i.e.

\partial ρ_{t} / \partial t = 0

. It follows that an ensemble is stationary if and only if

ρ

Poisson-commutes with

H

, that is

{ρ, H} = 0

. Examples of stationary ensembles are thus the functions of

H

, i.e.

\begin{matrix} (2.52) & ρ (P, Q) = f (H (P, Q)) \end{matrix}

where

f

is some function. A particular example of a stationary ensemble is

\begin{matrix} (2.53) & ρ (P, Q) = \frac{1}{Z_{β}} e^{- β H (P, Q)} \end{matrix}

where

β > 0

is a parameter and the normalization factor

Z_{β}

ensures that

ρ

is properly normalized, so

Z_{β} = \int e^{- β H} d^{3 N} P d^{3 N} Q

. We will come back to this ensemble below.

The flow

Φ_{t}

is not only area preserving on the entire phase-space, but also on the energy surface

Ω_{E}

(with the natural integration element understood). Such area-preserving flows under certain conditions imply that the phase space average equals the time average, cf. (2.41). This is expressed by the ergodic theorem:

Theorem: Let

(P (t), Q (t))

be dense in

Ω_{E}

and

F

continuous. Then the time average is equal to the ensemble average:

\begin{matrix} (2.54) & lim_{T \to \infty} \frac{1}{T} \int_{0}^{T} F (P (t), Q (t)) d t = \frac{1}{| Ω_{E} |} \int_{Ω_{E}} F (P, Q) d S \end{matrix}

The ergodic theorem may thus be summarized as
time average = ensemble average,
where the "ensemble" is the uniform distribution on

Ω_{E}

given by

ρ (P, Q) = 1 / | Ω_{E} |

(we will come back to this ensemble later in the context of "

S = k_{B} \log W

", where it is also called the "micro-canonical ensemble"). In quantum theory, there is an analogue of this phenomenon going under the name "eigenstate thermalization", which is outlined in the appendix.

The key hypothesis is that the orbit lies dense in

Ω_{E}

and that this surface is compact. The first is clearly not the case if there are further constants of motion, since the orbit must then lie on a submanifold of

Ω_{E}

corresponding to particular values of these constants. The Kolmogorov-Arnold-Moser (KAM) theorem shows that small perturbations of systems with sufficiently many constants of motion again possess such invariant submanifolds, i.e. the ergodic theorem does not hold in such cases. Nevertheless, the ergodic theorem still remains an important motivation for studying ensembles.

One puzzling consequence of Liouville's theorem is that a trajectory starting at

(P_{0}, Q_{0})

comes back arbitrarily closely to that point, a phenomenon called Poincaré recurrence. An intuitive "proof" of this statement can be given as follows:

Figure 2.4: Sketch of the situation described in the proof of Poincaré recurrence.

Let

B_{0}

be an

ϵ

-neighborhood of a point

(P_{0}, Q_{0})

. For

k \in N

define

B_{k} := Φ_{k} (B_{0})

, which are

ϵ

-neighborhoods of

(P_{k}, Q_{k}) = Φ_{k} ((Q_{0}, P_{0}))

. Let us assume that the statement of the theorem is wrong. This yields

B_{0} \cap B_{k} = \emptyset \forall k \in N

Then it follows that

B_{n} \cap B_{k} = \emptyset \forall n, k \in N, n \neq k

Now, by Liouville's theorem we have

| B_{0} | = | B_{1} | = \dots = | B_{k} | = \dots

which immediately yields

| Ω_{E} | ⩾ | B_{0} | + \dots + | B_{k} | + \dots = \infty

This clearly contradicts the assumption that

Ω_{E}

is compact and therefore the statement of the theorem has to be true.

Historically, the recurrence argument played an important role in early discussions of the notion of irreversibility, i.e. the fact that systems generically tend to approach an equilibrium state, whereas they never seem to spontaneously leave an equilibrium state and evolve back to the (non-equilibrium) initial conditions. To explain the origin of resp. the mechanisms behind irreversibility is one of the major challenges of non-equilibrium thermodynamics and we shall briefly come back to this point later. For the moment, we simply note that in practice the recurrence time

τ_{recurrence}

would be extremely large compared to the natural scales of the system such as the equilibration time. We will verify this by investigating the dynamics of a toy model in the appendix. Here we only
give a heuristic explanation. Consider a gas of

N

particles in a volume

V

. The volume is partitioned into sub volumes

V_{1}, V_{2}

of equal size. We start the system in a state where the atoms only occupy

V_{1}

. By the ergodic theorem we estimate that the fraction of time the system spends in such a state is

⟨ χ_{Q \in V_{1}} ⟩ = 2^{- 3 N}

(for an ideal gas), where

χ_{Q \in V_{1}}

gives 1 if all particles are in

V_{1}

, and zero otherwise. For

N = 1 mol

, i.e.

N = O (10^{23})

, this fraction is astronomically small. So there is no real puzzle!

One often has the situation that a system can be divided up into two (or more) parts which can be treated approximately as isolated or which have some simple well-known interaction. One way to model this situation is to suppose that the phase space is a direct product

Ω = Ω_{A} \times Ω_{B}

, where

A

and

B

label the subsystems. For instance, in an

N

-particle system,

Ω_{A}

could comprise the phase space coordinates of one (or more) distinguished particle, and

Ω_{B}

those of all the others. If we have a probability density (ensemble) of the total system, i.e. function

ρ (P, Q)

Ω

, we may decide to probe it with observables of system A only, i.e. with observables

F_{A} = F_{A} (P_{A}, Q_{A})

which are functions of the phase space coordinates of

Ω_{A}

only. It is then clear that for such an

F_{A}

, we can write (with

P = (P_{A}, P_{B}), Q = (Q_{A}, Q_{B})

)

\begin{matrix} (2.56) & ⟨ F_{A} ⟩ = \int_{Ω_{A} \times Ω_{B}} ρ (P_{A}, P_{B}, Q_{A}, Q_{B}) F_{A} (P_{A}, Q_{A}) = \int_{Ω_{A}} ρ_{A} (P_{A}, Q_{A}) F_{A} (P_{A}, Q_{A}), \end{matrix}

so it is natural to make the following definition:
Definition For a phase space

Ω = Ω_{A} \times Ω_{B}

describing two subsystems

A

and

B

of a given system, the reduced probability distribution for

A

is defined by

\begin{matrix} (2.57) & ρ_{A} (P_{A}, Q_{A}) = \int_{Ω_{B}} ρ (P_{A}, P_{B}, Q_{A}, Q_{B}) . \end{matrix}

The reduced probability distribution is that assigned to the system by an observer having only access to observables of system A (and similarly B). Note that

ρ_{A}

satisfies again all the axioms of a probability density (on

Ω_{A}

2.5 Ensembles in Quantum Mechanics (Statistical Operators / Density Matrices)

Quantum mechanical systems are of an intrinsically probabilistic nature, so the language of probability theory is, in this sense, not just optional but actually essential. In fact, to
say that the system is in a state

| Ψ ⟩

really means that, if

A

is a self adjoint operator and

\begin{matrix} (2.58) & A = \sum_{i} a_{i} | i ⟩ ⟨ i | \end{matrix}

its spectral decomposition

^{2}

, the probability for measuring the outcome

a_{i}

is given by

p_{A, Ψ} (a_{i}) = | ⟨ Ψ ∣ i ⟩ |^{2} \equiv p_{i} .

Thus, if we assign the state

| Ψ ⟩

to the system, the set of possible measuring outcomes for

A

is the probability space

Ω = {a_{1}, a_{2}, \dots}

with (discrete) probability distribution given by

{p_{1}, p_{2}, \dots}

In statistical mechanics we have incomplete information about the state of a quantum mechanical system. In particular, we do not want to prejudice ourselves by ascribing a pure state

| Ψ ⟩

to the system. Instead, we describe it by a statistical mixture i.e. an ensemble of pure states. Suppose we believe that the system is in the state

| Ψ_{i} ⟩

with probability

λ_{i}

, where, as usual,

\sum λ_{i} = 1, λ_{i} ⩾ 0

. For example, before preparing the state we perform a classical random experiment to determine which state

| Ψ_{i} ⟩

we prepare. The states

| Ψ_{i} ⟩

should be normalized, i.e.

⟨ Ψ_{i} ∣ Ψ_{i} ⟩ = 1

, but they do not have to be orthogonal or complete. Then the expectation value

⟨ A ⟩

of an operator is defined as

\begin{matrix} (2.59) & ⟨ A ⟩ = \sum_{i} λ_{i} ⟨ Ψ_{i} | A | Ψ_{i} ⟩ . \end{matrix}

Introducing the density matrix

ρ = \sum_{i} λ_{i} | Ψ_{i} ⟩ ⟨ Ψ_{i} |

this may also be written as

\begin{matrix} (2.60) & ⟨ A ⟩ = tr (ρ A) . \end{matrix}

The density matrix has the properties

tr ρ = 1

, as well as

ρ^{†} = ρ

. Furthermore, for any state

| Φ ⟩

we have

⟨ Φ | ρ | Φ ⟩ = \sum_{i} λ_{i} {| ⟨ Ψ_{i} ∣ Φ ⟩ |}^{2} ⩾ 0 .

The density matrix should be thought of as analogous to the classical probability distribution

{p_{i}}

given by the eigenvalues

p_{i}

ρ

(which coincide with the

λ_{i}

if and only if the states

| Ψ_{i} ⟩

are orthogonal).

In the context of quantum mechanical ensembles one can define a quantity that is closely analogous to the information entropy for ordinary probability distributions. This

quantity is defined as
$$

\begin{matrix} (2.61) & S_{v . N .} (ρ) = - k_{B} tr (ρ \log ρ) = - k_{B} \sum_{i} p_{i} \log p_{i} \end{matrix}

a n d i s c a l l e d t h e v o n N e u m a n n e n t r o p y a s s o c i a t e d w i t h $ ρ $ . A c c o r d i n g t o t h e r u l e s o f q u a n t u m m e c h a n i c s, t h e t i m e e v o l u t i o n o f a s t a t e i s d e s c r i b e d b y S c h r ö d i n g e r^{'} s e q u a t i o n

\begin{aligned} i ℏ \frac{d}{d t} | Ψ (t) ⟩ & = H | Ψ (t) ⟩ \\ \Rightarrow i ℏ \frac{d}{d t} ρ (t) & = [H, ρ (t)] \equiv H ρ (t) - ρ (t) H \end{aligned}

Therefore an ensemble is stationary if

[H, ρ] = 0

. In particular,

ρ

is stationary if it is of the form

ρ = f (H) = \sum_{i} f (E_{i}) | Ψ_{i} ⟩ ⟨ Ψ_{i} |,

where

\sum_{i} f (E_{i}) = 1

and

p_{i} = f (E_{i}) > 0

(here,

E_{i}

label the eigenvalues of the Hamiltonian

H

and

| Ψ_{i} ⟩

its eigenstates, i.e.

H | Ψ_{i} ⟩ = E_{i} | Ψ_{i} ⟩)

. The characteristic example is given by

\begin{matrix} (2.62) & f (H) = \frac{1}{Z_{β}} e^{- β H} \end{matrix}

where

Z_{β} = \sum_{i} e^{- β E_{i}}

. More generally, if

{Q_{α}}

are operators commuting with

H

, then another choice is

\begin{matrix} (2.63) & ρ = \frac{1}{Z (β, μ_{α})} e^{- β H - \sum_{α} μ_{α} Q_{α}} \end{matrix}

We will come back to discuss such ensembles below in chapter 4.
One often deals with situations in which a system is comprised of two sub-systems A and B described by Hilbert spaces

H_{A}, H_{B}

. The total Hilbert space is then

H = H_{A} \otimes H_{B}

(

\otimes

is the tensor product). If

{| i ⟩_{A}}

and

{| j ⟩_{B}}

are orthonormal bases of

H_{A}

and

H_{B}

, an orthonormal basis of

H

is given by

{| i, j ⟩ = | i ⟩_{A} \otimes | j ⟩_{B}}

Consider a (pure) state

| Ψ ⟩

H

, i.e. a pure state of the total system. It can be expanded as

| Ψ ⟩ = \sum_{i, j} c_{i, j} | i, j ⟩

We assume that the state is normalized, meaning that

\begin{matrix} (2.64) & \sum_{i, j} {| c_{i, j} |}^{2} = 1 \end{matrix}

Observables describing measurements of subsystem A consist of operators of the form

\tilde{a} = a \otimes 1_{B}

, where

a

is an operator on

H_{A}

and

1_{B}

is the identity operator on

H_{B}

(similarly
an observable describing a measurement of system

B

corresponds to

\tilde{b} = 1_{A} \otimes b

). For such an operator we can write:

\begin{aligned} ⟨ Ψ | \tilde{a} | Ψ ⟩ & = \sum_{i, j, k, l} {\bar{c}}_{i, k} c_{j, l} ⟨ i, k | a \otimes 1_{B} | j, l ⟩ \\ = \sum_{i, j, k, l} {\bar{c}}_{i, k} c_{j, l}_{A} ⟨ i | a | j ⟩_{A} \underset{δ_{k l}}{\underset{⏟}{B ⟨ k ∣ l ⟩_{B}}} \\ = \sum_{i, j} \underset{=: {(ρ_{A})}_{j i}}{\underset{⏟}{(\sum_{k} {\bar{c}}_{i, k} c_{j, k})}}) A ⟨ i | a | j ⟩_{A} \\ = {tr}_{A} (a ρ_{A}) . \end{aligned}

The operator

ρ_{A}

H_{A}

by definition satisfies

ρ_{A}^{†} = ρ_{A}

and by (2.64), it satisfies

tr ρ_{A} = 1

. It is also not hard to see that

⟨ Φ | ρ_{A} | Φ ⟩ ⩾ 0

. Thus,

ρ_{A}

defines a density matrix on the Hilbert space

H_{A}

of system A . One similarly defines

ρ_{B}

H_{B}

Definition: The operator

ρ_{A}

is called reduced density matrix of subsystem A , and

ρ_{B}

that of subsystem B.

The reduced density matrix reflects the limited information of an observer only having access to a subsystem. The quantity

\begin{matrix} (2.65) & S_{ent} := S_{v . N .} (ρ_{A}) = - k_{B} tr (ρ_{A} \log ρ_{A}) \end{matrix}

is called the entanglement entropy of subsystem A. One shows that

S_{v.N.} (ρ_{A}) =

S_{v.N.} (ρ_{B})

, so it does not matter which of the two subsystems we use to define it.

Example: Let

H_{A} = C^{2} = H_{B}

with orthonormal basis

{| ↑ ⟩, | ↓ ⟩}

for either system A or B. The orthonormal basis of

H

is then given by

{| ↑↑ ⟩, | ↑↓ ⟩, | ↓↑ ⟩, | ↓↓ ⟩}

.
(i) Let

| Ψ ⟩ = | ↑↓ ⟩

. Then

\begin{matrix} (2.66) & ⟨ Ψ | \tilde{a} | Ψ ⟩ = ⟨ ↑↓ | a \otimes 1_{B} | ↑↓ ⟩ = ⟨ ↑ | a | ↑ ⟩ . \end{matrix}

from which it follows that the reduced density matrix of subsystem A is given by

\begin{matrix} (2.67) & ρ_{A} = | ↑ ⟩ ⟨ ↑ | \end{matrix}

The entanglement entropy is

\begin{matrix} (2.68) & S_{ent} = - k_{B} tr (ρ_{A} \log ρ_{A}) = - k_{B} (1 \cdot \log 1) = 0 \end{matrix}

(ii) Let

| Ψ ⟩ = \frac{1}{\sqrt{2}} (| ↑↓ ⟩ - | ↓↑ ⟩)

. Then

\begin{aligned} ⟨ Ψ | \tilde{a} | Ψ ⟩ & = \frac{1}{2} (⟨ ↑↓ | - ⟨ ↓↑ |) (a \otimes 1_{B}) (| ↑↓ ⟩ - | ↓↑ ⟩) \\ (2.69) & = \frac{1}{2} (⟨ ↑ | a | ↑ ⟩ + ⟨ ↓ | a | ↓ ⟩), \end{aligned}

from which it follows that the reduced density matrix of subsystem A is given by

\begin{matrix} (2.70) & ρ_{A} = \frac{1}{2} (| ↑ ⟩ ⟨ ↑ | + | ↓ ⟩ ⟨ ↓ |) . \end{matrix}

The entanglement entropy is

\begin{matrix} (2.71) & S_{ent} = - k_{B} tr (ρ_{A} \log ρ_{A}) = - k_{B} (\frac{1}{2} \log \frac{1}{2} + \frac{1}{2} \log \frac{1}{2}) = k_{B} \log 2 \end{matrix}

Chapter 3 Time-evolving ensembles

3.1 Boltzmann Equation in Classical Mechanics

In order to understand the dynamical properties of systems in statistical mechanics one has to study non-stationary (i.e. time-dependent) ensembles. A key question, already brought up earlier, is whether systems initially described by a non-stationary ensemble will eventually approach an equilibrium ensemble. An important quantitative tool to understand the approach to equilibrium (e.g. in the case of thin media or weakly coupled systems) is the Boltzmann equation, which we discuss here in the case of classical mechanics.

Let

ρ

be an ensemble and

ρ_{t} (P, Q) = ρ (P (t), Q (t))

be the time evolving ensemble defined in the previous chapter. We would like to learn something about this function

ρ_{t}

. It is of course in general impossible to determine this exactly, because we cannot find the particle trajectories

(P (t), Q (t))

for a system with a large number

N

of particles. Also, even if we could, the full

ρ_{t} (P, Q)

as a function of

6 N

phase space coordinates is in general way more information than we really would like to have in practice. Instead, we often only want to know the time evolution of a relatively small subset of observables. One such observable is the 1 -particle density

f_{1}

which is defined by

\begin{aligned} f_{1} ({\vec{p}}_{1}, {\vec{x}}_{1}; t) & := ⟨ \sum_{i} δ^{3} ({\vec{p}}_{1} - {\vec{p}}_{i}) δ^{3} ({\vec{x}}_{1} - {\vec{x}}_{i}) ⟩ \\ (3.1) & = N \int ρ_{t} ({\vec{p}}_{1}, {\vec{x}}_{1}, {\vec{p}}_{2}, {\vec{x}}_{2} \dots, {\vec{p}}_{N}, {\vec{x}}_{N}) \prod_{i = 2}^{N} d^{3} x_{i} d^{3} p_{i} . \end{aligned}

Similarly, the two particle density can be computed from

ρ

via

\begin{matrix} (3.2) & f_{2} ({\vec{p}}_{1}, {\vec{x}}_{1}, {\vec{p}}_{2}, {\vec{x}}_{2}; t) = N (N - 1) \int ρ_{t} ({\vec{p}}_{1}, {\vec{x}}_{1}, {\vec{p}}_{2}, {\vec{x}}_{2} \dots, {\vec{p}}_{N}, {\vec{x}}_{N}) \prod_{i = 3}^{N} d^{3} x_{i} d^{3} p_{i} \end{matrix}

Analogously, we define the

s

-particle densities

f_{s}

, for

2 < s ⩽ N

. Note that the

s

-particle densities are, up to the normalization, nothing but the reduced probability distributions obtained from

ρ_{t}

if we divide the total system into a system A consisting of

s

fixed particles and a system B consisting of the remaining ones, i.e. for instance for

s = 1

\begin{matrix} (3.3) & ρ_{t A} ({\vec{p}}_{1}, {\vec{x}}_{1}) = \frac{1}{N} f_{1} ({\vec{p}}_{1}, {\vec{x}}_{1}; t) . \end{matrix}

The Hamiltonian

H_{s}

describing the interaction between

s

particles can be written as

\begin{matrix} (3.4) & H_{s} = \sum_{i = 1}^{s} \frac{{\vec{p}}_{i}^{2}}{2 m} + \sum_{1 ⩽ i < j ⩽ s} V ({\vec{x}}_{i} - {\vec{x}}_{j}) + \sum_{i = 1}^{s} W ({\vec{x}}_{i}) \end{matrix}

so that in particular

H_{N} = H

. One finds the relations

\begin{matrix} (3.5) & \underset{streaming term}{\underset{⏟}{\frac{\partial f_{s}}{\partial t} - {H_{s}, f_{s}}}} = \underset{collision term}{\underset{⏟}{\sum_{i = 1}^{s} \int d^{3} p_{s + 1} d^{3} x_{s + 1} \frac{\partial V ({\vec{x}}_{i} - {\vec{x}}_{s + 1})}{\partial {\vec{x}}_{i}} \cdot \frac{\partial f_{s + 1}}{\partial {\vec{p}}_{i}}}} \end{matrix}

This system of equations is called the BBGKY hierarchy (for Bogoliubov-Born-GreenKirkwood

Y

von hierarchy). The first term

(s = 1)

is given by

\begin{matrix} (3.6) & [\frac{\partial}{\partial t} - \underset{= \vec{F} (ext. force)}{\underset{⏟}{\frac{\partial W}{\partial {\vec{x}}_{1}}}} \cdot \frac{\partial}{\partial {\vec{p}}_{1}} + \underset{= \vec{v} (velocity)}{\underset{⏟}{\frac{{\vec{p}}_{1}}{m}}} \cdot \frac{\partial}{\partial {\vec{x}}_{1}}] f_{1} = \int d^{3} p_{2} d^{3} x_{2} \frac{\partial V ({\vec{x}}_{1} - {\vec{x}}_{2})}{\partial {\vec{x}}_{1}} \cdot \underset{unknown!}{\underset{⏟}{\frac{\partial f_{2}}{\partial {\vec{p}}_{1}}}} \end{matrix}

An obvious feature of the BBGKY hierarchy is that the equation for

f_{1}

involves

f_{2}

, that for

f_{2}

involves

f_{3}

, etc. In this sense the equations for the individual

f_{i}

are not closed. To get a manageable system, some approximations/truncations are necessary.

The simplest truncation, which leads to the Boltzmann equation, is to assume uncorrelated densities, i.e.

\begin{aligned} f_{2} ({\vec{p}}_{1}, {\vec{x}}_{1}, {\vec{p}}_{2}, {\vec{x}}_{2}) & = \frac{N (N - 1)}{N^{2}} f_{1} ({\vec{p}}_{1}, {\vec{x}}_{1}) f_{1} ({\vec{p}}_{2}, {\vec{x}}_{2}) \\ (3.7) & ≃ f_{1} ({\vec{p}}_{1}, {\vec{x}}_{1}) f_{1} ({\vec{p}}_{2}, {\vec{x}}_{2}), \end{aligned}

where in the first line we took the different proportionality factors of reduced density matrices and the

f_{s}

into account. With this assumption, the equation (3.6) closes. In general, this assumption is inconsistent with the dynamical equation for

f_{2}

, i.e., it is not preserved under time evolution. However, in a certain limit in which

N \to \infty

and the interaction range

d \to 0

, one can prove the consistency of this truncation. To discuss conditions under which the assumption is approximately valid, one introduces several time scales which have to be sufficiently separated:
(i) Let

v

be the typical velocity of gas particles (e.g.

v \approx 100 \frac{m}{s}

at room temperature and 1atm) and let

L

be the scale over which

W (\vec{x})

varies, i.e. the box size. Then

τ_{v} := \frac{L}{v}

is the extrinsic scale (e.g.

τ_{v} \approx 10^{- 5}

s for

L \approx 1 mm

).
(ii) If

d

is the range of the interaction

V (\vec{x})

(e.g.

d \approx 10^{- 10} m

), then

τ_{c} := \frac{d}{v}

is the collision time (e.g.

τ_{c} \approx 10^{- 12} s

). We should have

τ_{c} ≪ τ_{v}

.
(iii) We can also define the mean free time

τ_{x} \approx \frac{τ_{c}}{n d^{3}} \approx \frac{1}{n v d^{2}}, n = \frac{N}{V}

, which is the average time between subsequent collisions. We have

τ_{x} \approx 10^{- 8} S ≫ τ_{c}

in our example.

For (a) and (b) to hold, we should have

τ_{v} ≫ τ_{x} ≫ τ_{c}

.
As by the assumptions on the scales, a particle moves freely between two encounters, we can approximate the interaction of two particles as a scattering process, described by the differential cross section

\frac{d σ}{d Σ}

, as indicated in the following sketch:

Figure 3.1: Classical scattering of particles in the "fixed target frame".

Here

d σ = b d b d ϕ

is the infinitesimal area element shaded in grey, through which a flux of particles passes, and

d Ω

is the infinitesimal area element on the unit sphere, indicating into which direction these particles are scattered. In other words,

| \frac{d σ}{d Ω} |

is the Jacobian between

\vec{b}

and

\hat{Ω} = (θ, ϕ)

. Hence, if

F = \frac{# of incoming particles}{area \cdot time}

is the incoming flux of particles, then

| \frac{d σ}{d Ω} (Ω) | F d Ω

is the number rate of particles being scattered into the area element

d Ω

If, instead of the fixed target frame indicated in the figure, one considers scattering in the center of mass frame and denotes

\vec{p} = {\vec{p}}_{1} - {\vec{p}}_{2}

and

{\vec{p}}^{'} = {\vec{p}}_{1}^{'} - {\vec{p}}_{2}^{'}

as the relative momentum before and after the scattering (where we assume elastic collision, i.e.,

| \vec{p} | =

| \vec{p} |)

, one can then arrive at the Boltzmann equation

\begin{aligned} [\frac{\partial}{\partial t} - \vec{F} \frac{\partial}{\partial {\vec{p}}_{1}} + {\vec{v}}_{1} \frac{\partial}{\partial {\vec{x}}_{1}}] f_{1} ({\vec{p}}_{1}, {\vec{x}}_{1}; t) = \\ (3.8) & - \int d^{3} p_{2} d^{2} Ω \underset{cross-section}{\underset{⏟}{| \frac{d σ}{d Ω} |}} \cdot \underset{α fllux}{\underset{⏟}{∣ {\vec{v}}_{1} - {\vec{v}}_{2}}} \cdot [f_{1} ({\vec{p}}_{1}, {\vec{x}}_{1}; t) f_{1} ({\vec{p}}_{2}, {\vec{x}}_{1}; t) - f_{1} ({\vec{p}}_{1}^{'}, {\vec{x}}_{1}; t) f_{1} ({\vec{p}}_{2}^{'}, {\vec{x}}_{1}; t)], \end{aligned}

where

Ω = (θ, ϕ)

is the solid angle between

\vec{p}

and

\vec{p}

, and

d^{2} Ω = \sin θ d θ d ϕ

. The integral expression on the right side of the Boltzmann equation (3.8) is called the collision operator, and is often denoted as

C [f_{1}] (t, {\vec{p}}_{1}, {\vec{x}}_{1})

. It represents the change in the

1 -

particle distribution due to collisions of particles. The two terms in the brackets [...] under the integral in (3.8) can be viewed as taking into account that new particles with momentum

{\vec{p}}_{1}

can be created or be lost, respectively, when momentum is transferred from other particles in a collision process.

It is important to know whether

f_{1} (\vec{p}, \vec{x}; t)

is stationary, i.e. time-independent. Intuitively, this should be the case when the collision term

C [f_{1}]

vanishes. This in turn should happen if

\begin{matrix} (3.9) & f_{1} ({\vec{p}}_{1}, \vec{x}; t) f_{1} ({\vec{p}}_{2}, \vec{x}; t) = f_{1} ({\vec{p}}_{1}^{'}, \vec{x}; t) f_{1} ({\vec{p}}_{2}^{'}, \vec{x}; t) \end{matrix}

As we will now see, one can derive the functional form of the 1-particle density from this condition. Taking the logarithm on both sides of (3.9) gives, with

F_{1} = \log f_{1}

etc.,

\begin{matrix} (3.10) & F_{1} + F_{2} = F_{1^{'}} + F_{2^{'}} \end{matrix}

whence

F

must be a conserved quantity, i.e. either we have

F = β \frac{{\vec{p}}^{2}}{2 m}

F = \vec{α} \cdot \vec{p}

F = γ

. It follows, after renaming constants, that

\begin{matrix} (3.11) & f_{1} = c \cdot e^{- β \frac{{(\vec{p} - {\vec{p}}_{0})}^{2}}{2 m}} \end{matrix}

In principle

c, β, {\vec{p}}_{0}

could be functions of

\vec{x}

and

t

at this stage, but then the left hand side of the Boltzmann equation does not vanish in general. So (3.11) represents the general stationary homogeneous solution to the Boltzmann equation. It is known as the Maxwell-Boltzmann distribution. The proper normalization is, from

\int f_{1} d^{3} p d^{3} x =

\begin{matrix} (3.12) & c = \frac{N}{V} {(\frac{β}{2 π m})}^{\frac{3}{2}}, {\vec{p}}_{0} = ⟨ \vec{p} ⟩ . \end{matrix}

The mean kinetic energy is found to be

⟨ \frac{{\vec{p}}^{2}}{2 m} ⟩ = \frac{3}{2 β}

, so

β = \frac{1}{K_{B} T}

is identified with the inverse temperature of the gas.

This interpretation of

β

is reinforced by considering a gas of

N

particles confined to a
box of volume

V

. The pressure of the gas results from a force

K

acting on a wall element of area

A

, as depicted in the figure below. The force is equal to:

\begin{aligned} K & = \frac{1}{Δ t} \int^{3} p \cdot # \overset{(f_{1} (\vec{p}) d^{3} p) \cdot (A v_{x} Δ t)}{\overset{⏞}{(\binom{particles impacting A}{during Δ t with momenta between \vec{p} and \vec{p} + d \vec{p}})}} \times \overset{2 p_{x}}{\overset{⏞}{(\binom{momentum transfer}{in x - direction})}} \\ = \frac{1}{Δ t} \int_{0}^{\infty} d p_{x} \int_{- \infty}^{\infty} d p_{y} \int_{- \infty}^{\infty} d p_{z} f_{1} (\vec{p}) (A v_{x} Δ t) \cdot (2 p_{x}) . \end{aligned}

Note, that the first integral is just over half of the range of

p_{x}

, which is due to the fact that only particles moving in the direction of the wall will hit it.
Together with (3.11) it follows that the pressure

P

is given by

\begin{matrix} (3.13) & P = \frac{K}{A} = \int d^{3} p f_{1} (\vec{p}) \frac{p_{x}^{2}}{m} = \frac{n}{β} . \end{matrix}

Comparing with the equation of state for an ideal gas,

P V = N k_{B} T

, we get

β = \frac{1}{k_{B} T}

Figure 3.2: Pressure on the walls due to the impact of particles.

It is noteworthy that, in the presence of external forces, other solutions representing equilibrium (but with a non-vanishing collision term) should also be possible. One only has to think of the following situation, representing a stationary air flow across a wing:

Figure 3.3: Sketch of the air-flow across a wing.

In this case we have to deal with a much more complicated

f_{1}

, not equal to the Maxwell-Boltzmann distribution. As the example of an air-flow suggests, the Boltzmann equation is also closely related to other equations for fluids such as the Euler- or NavierStokes equation, which can be seen to arise as approximations of the Boltzmann equation.

The Boltzmann equation can easily be generalized to a gas consisting of several species

α, β, \dots

which are interacting via the 2 -body potentials

V_{α, β} (\vec{x} (α) - \vec{x} (β))

. As before, we can define the 1-particle density

f_{1}^{(α)} (\vec{p}, \vec{x}, t)

for each species. The same derivation leading to the Boltzmann equation now gives the system of equations

\begin{matrix} (3.14) & [\frac{\partial}{\partial t} - \vec{F} \frac{\partial}{\partial \vec{p}} + \vec{v} \frac{\partial}{\partial \vec{x}}] f_{1}^{(α)} = \sum_{β} C^{(α, β)} \end{matrix}

where the collision term

C^{(α, β)}

is given by

\begin{aligned} (3.15) & C^{(α, β)} = & - \int d^{3} p_{2} d^{2} Ω | \frac{d σ_{α, β}}{d Ω} | | {\vec{v}}_{1} - {\vec{v}}_{2} | \times \\ \times [f_{1}^{(α)} ({\vec{p}}_{1}, {\vec{x}}_{1}; t) f_{1}^{(β)} ({\vec{p}}_{2}, {\vec{x}}_{1}; t) - f_{1}^{(α)} ({\vec{p}}_{1}^{'}, {\vec{x}}_{1}; t) f_{1}^{(β)} ({\vec{p}}_{2}^{'}, {\vec{x}}_{1}; t)] \end{aligned}

This system of equations has great importance in practice e.g. for the evolution of the abundances of different particle species in the early universe. In this case

\begin{matrix} (3.16) & f_{1}^{(α)} (\vec{p}, \vec{x}; t) \approx f_{1}^{(α)} (\vec{p}, t) \end{matrix}

are homogeneous distributions and the external force

\vec{F}

on the left hand side of equations (3.14) is related to the expansion of the universe.

Demanding equilibrium now amounts to

\begin{matrix} (3.17) & f_{1}^{(α)} ({\vec{p}}_{1}; t) f_{1}^{(β)} ({\vec{p}}_{2}; t) = f_{1}^{(α)} ({\vec{p}}_{1}^{'}; t) f_{1}^{(β)} ({\vec{p}}_{2}^{'}; t) \end{matrix}

and similar arguments as above lead to

\begin{matrix} (3.18) & f_{1}^{(α)} \propto e^{- β \frac{{(\vec{p} - m_{α} {\vec{v}}_{0})}^{2}}{2 m_{α}}}, \end{matrix}

i.e. we have the same temperature

T

and average velocity

{\vec{v}}_{0}

for all

α

. In the context of the early universe it is essential to study deviations from equilibrium in order to explain the observed abundances.

By contrast to the original system of equations (Hamilton's equations or the BBGKY hierarchy), the Boltzmann equation is irreversible. This can be seen for example by introducing the function

\begin{matrix} (3.19) & h (t) = - k_{B} \int d^{3} x d^{3} p f_{1} (\vec{p}, \vec{x}; t) \log f_{1} (\vec{p}, \vec{x}; t) = S_{\inf} (f_{1} (t)), \end{matrix}

which is called Boltzmann H-function. It can be shown (cf. problem B.11) that

\dot{h} (t) ⩾ 0

, with equality if

f_{1} ({\vec{p}}_{1}, \vec{x}; t) f_{1} ({\vec{p}}_{2}, \vec{x}; t) = f_{1} ({\vec{p}}_{1}^{'}, \vec{x}; t) f_{1} ({\vec{p}}_{2}^{'}, \vec{x}; t)

a result which is known as the

H

-theorem. We just showed this equality holds if and only if

f_{1}

is given by the Maxwell-Boltzmann distribution. Thus, we conclude that

h (t)

is an increasing function, as long as

f_{1}

is not equal to the Maxwell-Boltzmann distribution. In particular, the evolution of

f_{1}

, as described by the Boltzmann equation, is irreversible. Since the Boltzmann equation is only an approximation to the full BBGKY hierarchy, which is reversible, there is no mathematical inconsistency. However, it is not clear, a priori, at which stage of the derivation the irreversibility has been allowed to enter. Looking at the approximations (a) and (b) made above, it is clear that the assumption that the 2 -particle correlations

f_{2}

are factorized, as in (b), cannot be exactly true, since the outgoing momenta of the particles are correlated. Although this correlation is extremely small after several collisions, it is not exactly zero. Our decision to neglect it can be viewed as one reason for the emergence of irreversibility on a macroscopic scale.

The close analogy between the definition of the Boltzmann

H

-function and the information entropy

S_{inf}

, as defined in (2.15), together with the monotonicity of

h (t)

suggest that

h

should represent some sort of entropy of the system. The

H

-theorem is then viewed as a "derivation" of the

2^{nd}

law of thermodynamics (see Chapter 6). However, this point of view is not entirely correct, since

h (t)

only depends on the 1-particle density

f_{1}

and not on the higher particle densities

f_{s}

, which in general should also contribute to the entropy. It is not clear how an entropy with sensible properties has to be defined in a completely general situation, in particular when the above approximations (a) and (b) are not justified.

3.2 Boltzmann Equation, Approach to Equilibrium in Quantum Mechanics

A version of the Boltzmann equation and the

H

-theorem can also be derived in the quantum mechanical context. The main difference to the classical case is a somewhat modified collision term: the classical differential cross section is replaced by the quantum mechanical differential cross section (in the Born approximation) and the combination

f_{1} ({\vec{p}}_{1}, \vec{x}; t) f_{1} ({\vec{p}}_{2}, \vec{x}; t) - f_{1} ({\vec{p}}_{1}^{'}, \vec{x}; t) f_{1} ({\vec{p}}_{2}^{'}, \vec{x}; t)

is somewhat changed in order to accommodate Bose-Einstein resp. Fermi-Dirac statistics (see section 5.1 for an explanation of these terms). This then leads to the corresponding equilibrium distributions in the stationary case. Starting from the quantum Boltzmann equation, one can again derive a corresponding

H

-theorem. Rather than explaining the details, we give a simplified "derivation" of the

H

-theorem, which also will allow us to introduce a simple minded but very useful approximation of the dynamics of probabilities, discussed in more detail in the Appendix.

The basic idea is to ascribe the approach to equilibrium to an incomplete knowledge of the true dynamics due to perturbations. The true Hamiltonian is written as

\begin{matrix} (3.20) & H = H_{0} + H_{1}, \end{matrix}

where

H_{1}

is a tiny perturbation over which we do not have control. For simplicity, we assume that the spectrum of the unperturbed Hamiltonian

H_{0}

is discrete and we write

H_{0} | n ⟩ = E_{n} | n ⟩

. For a typical eigenstate

| n ⟩

we then have

\begin{matrix} (3.21) & \frac{⟨ n | H_{1} | n ⟩}{E_{n}} ≪ 1 \end{matrix}

Let

p_{n}

be the probability that the system is in the state

| n ⟩

, i.e. we ascribe to the system the density matrix

ρ = \sum_{n} p_{n} | n ⟩ ⟨ n |

. For generic perturbations

H_{1}

, this ensemble is not stationary with respect to the true dynamics because

[ρ, H] \neq 0

. Consequently, the von Neumann entropy

S_{v . N . of} ρ (t) = e^{i t H} ρ e^{- i t H}

depends upon time. We define this to be the

H

-function

\begin{matrix} (3.22) & h (t) := S_{v \cdot N \cdot} (ρ (t)) = - k_{B} \sum_{n} p_{n} \log p_{n} \end{matrix}

Next, we approximate the dynamics by imagining that our perturbation

H_{1}

will cause jumps from state

| n ⟩

to state

| m ⟩

leading to time-dependent probabilities as described
by the master equation

^{1}

\begin{matrix} (3.23) & {\dot{p}}_{n} (t) = \sum_{m : m \neq n} (T_{n m} p_{m} (t) - T_{m n} p_{n} (t)), \end{matrix}

where

T_{n m}

is the transition rate of going from state

| n ⟩

to state

| m ⟩

. Thus, the approximated, time-dependent density matrix is

ρ (t) = \sum_{n} p_{n} (t) | n ⟩ ⟨ n |

, with

p_{n} (t)

obeying the master equation. By the latter,

\begin{aligned} \dot{h} (t) & = - \sum_{n} ({\dot{p}}_{n} \log p_{n} + {\dot{p}}_{n}) \\ = - \frac{1}{2} (\sum_{n} {\dot{p}}_{n} \log (e \cdot p_{n}) + \sum_{m} {\dot{p}}_{m} \log (e \cdot p_{m})) \\ = \frac{1}{2} k_{B} \sum_{n, m} T_{n m} [p_{n} (t) - p_{m} (t)] [\log p_{n} (t) - \log p_{m} (t)] ⩾ 0 . \end{aligned}

The latter inequality follows from the fact that both terms in parentheses [...] have the same sign, just as in the proof of the classical

H

-theorem (problem B.11. Note that if we had defined

h (t)

as the von Neumann entropy, using a density matrix

ρ

that is diagonal in an eigenbasis of the full Hamiltonian

H

(rather than the unperturbed Hamiltonian), then we would have obtained

[ρ, H] = 0

and consequently

ρ (t) = ρ

, i.e. a constant

h (t)

. Thus, in this approach, the

H

-theorem is viewed as a consequence of our partial ignorance about the system, which prompts us to ascribe to it a density matrix

ρ (t)

which is diagonal with respect to

H_{0}

. In order to justify working with a density matrix

ρ

that is diagonal with respect to

H_{0}

(and therefore also in order to explain the approach to equilibrium), one may argue very roughly as follows: suppose that we start with a system in a state

| Ψ ⟩ = \sum_{n} γ_{n} | n ⟩

that is not an eigenstate of the true Hamiltonian

H

. Let us write

| Ψ (t) ⟩ = \sum_{n} γ_{n} (t) e^{\frac{- i E_{n} t}{ℏ}} | n ⟩ \equiv e^{- i H t} | Ψ ⟩ .

for the time evolved state. If there is no perturbation, i.e.

H_{1} = 0

, we get

γ_{n} (t) = γ_{n} = const.

but for

H_{1} \neq 0

this is typically not the case. The time average of an operator (observable)

A

is given by

\begin{matrix} (3.24) & lim_{T \to \infty} \frac{1}{T} \int_{0}^{T} ⟨ Ψ (t) | A | Ψ (t) ⟩ d t = lim_{T \to \infty} tr (ρ (T) A), \end{matrix}

with
$$

\begin{matrix} (3.25) & ⟨ n | ρ (T) | m ⟩ = \frac{1}{T} \int_{0}^{T} γ_{n} (t) \overset{―}{γ_{m} (t)} e^{\frac{i t (E_{n} - E_{m})}{ℏ}} d t \end{matrix}

For

T \to \infty

the oscillating phase factor

e^{i t (E_{n} - E_{m})}

is expected to cause the integral to vanish for

E_{n} \neq E_{m}

(destructive interference). Thus, we expect that

⟨ n | ρ (T) | m ⟩ \to_{T \to \infty}^{⟶}

p_{n} δ_{n, m}

. It follows that

\begin{matrix} (3.26) & lim_{T \to \infty} \frac{1}{T} \int_{0}^{T} ⟨ Ψ (t) | A | Ψ (t) ⟩ d t = tr (A ρ) \end{matrix}

where the density matrix

ρ

ρ = \sum_{n} p_{n} | n ⟩ ⟨ n |

. Since

[ρ, H_{0}] = 0

, the ensemble described by

ρ

is stationary with respect to

H_{0}

. The plausibility of this "derivation" rests on the basic assumption that, while

⟨ n | H_{1} | n ⟩

≪ E_{n}

, it can be large compared to

Δ E_{n} = E_{n} -

E_{n + 1} = O (e^{- N})

(where

N

is the particle number) and can therefore induce transitions causing the system to equilibrate.

Chapter 4 Equilibrium Ensembles

4.1 Generalities

In the probabilistic description of a system with a large number of constituents one considers probability distributions (=ensembles)

ρ (P, Q)

on phase space, rather than individual trajectories. In the previous section, we have given various arguments leading to the expectation that the time evolution of an ensemble will generally lead to an equilibrium ensemble. The study of such ensembles is the subject of equilibrium statistical mechanics. Standard equilibrium ensembles are:
(a) Micro-canonical ensemble (section 4.2).
(b) Canonical ensemble (section 4.3).
(c) Grand canonical (Gibbs) ensemble (section 4.4).

4.2 Micro-Canonical Ensemble

4.2.1 Micro-Canonical Ensemble in Classical Mechanics

Recall that in classical mechanics the phase space

Ω

of a system consisting of

N

particles without internal degrees of freedom is given by

\begin{matrix} (4.1) & Ω = R^{6 N} \end{matrix}

As before, we define the energy surface

Ω_{E}

\begin{matrix} (4.2) & Ω_{E} = {(P, Q) \in Ω : H (P, Q) = E}, \end{matrix}

where

H

denotes the Hamiltonian of the system. In the micro-canonical ensemble each point of

Ω_{E}

is considered to be equally likely. In order to write down the corresponding
ensemble, i.e. the density function

ρ (P, Q)

, we define the invariant volume

| Ω_{E} |

Ω_{E}

\begin{matrix} (4.3) & | Ω_{E} | := lim_{Δ E \to 0} \frac{1}{Δ E} \int_{E - Δ E ⩽ H (P, Q) ⩽ E} d^{3 N} P d^{3 N} Q \end{matrix}

which can also be expressed as

\begin{matrix} (4.4) & | Ω_{E} | = \frac{\partial Φ (E)}{\partial E}, with Φ (E) = \int_{H (P, Q) ⩽ E} d^{3 N} P d^{3 N} Q \end{matrix}

Thus, we can write the probability density of the micro-canonical ensemble as

\begin{matrix} (4.5) & ρ (P, Q) = \frac{1}{| Ω_{E} |} δ (H (P, Q) - E) . \end{matrix}

To avoid subtleties coming from the

δ

-function for sharp energy one sometimes replaces this expression by

\begin{matrix} (4.6) & ρ (P, Q) = \frac{1}{| {E - Δ E ⩽ H (P, Q) ⩽ E} |} \cdot {\begin{cases} 1, & if H (P, Q) \in (E - Δ E, E) \\ 0, & if H (P, Q) \notin (E - Δ E, E) \end{cases} \end{matrix}

Strictly speaking, this depends not only on

E

but also on

Δ E

. But in typical cases

| Ω_{E} |

depends exponentially on

E

, so there is practically no difference between these two expressions for

ρ (P, Q)

as long as

Δ E ≲ E

. We may alternatively write the second definition as:

\begin{matrix} (4.7) & ρ = \frac{1}{W (E)} [Θ (H - E + Δ E) - Θ (H - E)] \end{matrix}

Here we have used the Heaviside step function

Θ

, defined by

Θ (E) = {\begin{cases} 1, & for E > 0 \\ 0, & otherwise \end{cases}

We have also defined

\begin{matrix} (4.8) & W (E) = | {E - Δ E ⩽ H (P, Q) ⩽ E} | . \end{matrix}

Following Boltzmann, we give the following

Definition: The entropy of the micro-canonical ensemble is defined by

\begin{matrix} (4.9) & S (E) = k_{B} \log W (E) \end{matrix}

As we have already said, in typical cases, changing

S (E)

in this definition to

k_{B} \log | Ω_{E} |

will not significantly change the result. It is not hard to see that we may equivalently
write in either case

\begin{matrix} (4.10) & S (E) = - k_{B} \int ρ (P, Q) \log ρ (P, Q) d^{3 N} P d^{3 N} Q = S_{\inf} (ρ) \end{matrix}

i.e. Boltzmann's definition of entropy coincides with the definition of the information entropy (2.15) of the microcanonical ensemble

ρ

. As defined,

S

is a function of

E

and implicitly

V, N

, since these enter the definition of the Hamiltonian and phase space. Sometimes one also specifies other constants of motion or parameters of the system other than

E

when defining

S

. Denoting these constants collectively as

{I_{α}}

, one defines

W

accordingly with respect to

E

and

{I_{α}}

by replacing the energy surface with:

\begin{matrix} (4.11) & Ω_{E, {I_{α}}} := {(P, Q) \in Ω : H (P, Q) = E, I_{α} (P, Q) = I_{α}} . \end{matrix}

In this case

S = S (E, {I_{α}}, N, V)

becomes a function of more variables.

Example:

The ideal gas of

N

particles in a box has the Hamiltonian

H = \sum_{i = 1}^{N} (\frac{{\vec{p}}^{2}}{2 m} + W ({\vec{x}}_{i}))

, where the external potential

W

represents the walls of a box of volume

V

. For a box with hard walls we take, for example,

\begin{matrix} (4.12) & W (\vec{x}) = {\begin{cases} 0 & inside V \\ \infty & outside V \end{cases} . \end{matrix}

For the energy surface

Ω_{E}

we then find

\begin{aligned} (4.13) & Ω_{E} = {(P, Q) \in Ω ∣ \underset{\to V^{N}}{\underset{⏟}{{\vec{x}}_{i} inside the box}}, \underset{⏟}{\sum_{i = 1}^{N} {\vec{p}}^{2} = 2 E m}}, \\ = hyper sphere of dimension \\ 3 N - 1 and radius \sqrt{2 E m} \end{aligned}

from which it follows that

\begin{matrix} (4.14) & | Ω_{E} | = V^{N} {\sqrt{2 E m}}^{3 N - 1} \underset{= \frac{23^{3 N / 2}}{Γ (\frac{1}{2} 3 N)}}{\underset{⏟}{area (S^{3 N - 1})}} \end{matrix}

Here,

Γ (x) = (x - 1)

! denotes the

Γ

-function. The entropy

S (E, V, N)

is therefore given by

\begin{matrix} (4.15) & S (E, V, N) \approx k_{B} [N \log V + \frac{3 N}{2} \log (2 π m E) - \frac{3 N}{2} \log \frac{3 N}{2} + \frac{3 N}{2}] \end{matrix}

where we have used Stirling's approximation:

\begin{aligned} \log x! \approx \sum_{i = 1}^{x} \log i \approx \int_{1}^{x} \log y d y = x \log x - x + 1 \\ \Rightarrow x! \approx e^{- x} x^{x} . \end{aligned}

Thus, we obtain for the entropy of the ideal gas:

\begin{matrix} (4.16) & S (E, V, N) \approx N k_{B} \log [V {(\frac{4 π e m E}{3 N})}^{3 / 2}] \end{matrix}

Given the function

S (E, V, N)

for a system, one can define the corresponding temperature, pressure and chemical potential as follows:

Definition: The empirical temperature

T

, pressure

P

and chemical potential

μ

of the microcanonical ensemble are defined as:

\begin{matrix} (4.17) & \frac{1}{T} := {\frac{\partial S}{\partial E} |}_{V, N}, P := {T \frac{\partial S}{\partial V} |}_{E, N}, μ := - {T \frac{\partial S}{\partial N} |}_{E, V} \end{matrix}

For the ideal classical gas this definition, together with (4.16), yields for instance

\begin{matrix} (4.18) & \frac{1}{T} = \frac{\partial S}{\partial E} = \frac{3}{2} \frac{N k_{B}}{E} \end{matrix}

which we can rewrite in the more familiar form

\begin{matrix} (4.19) & E = \frac{3}{2} N k_{B} T . \end{matrix}

This formula states that for the ideal gas we have the equidistribution law

\begin{matrix} (4.20) & \frac{average energy}{degree of freedom} = \frac{1}{2} k_{B} T . \end{matrix}

One can similarly verify that the abstract definition of

P

in (4.17) above gives

\begin{matrix} (4.21) & P V = k_{B} N T, \end{matrix}

which is the familiar "equation of state" for an ideal gas.
In order to further motivate the second relation in (4.17), we consider a system comprised of a piston applied to an enclosed gas chamber:

Figure 4.1: Gas in a piston maintained at pressure

P

Here, we obviously have

P V = m g z

. The total energy is obtained as

\begin{matrix} (4.22) & H_{total} = H_{gas} (P, Q) + H_{piston} (p, z) = H_{gas} (P, Q) + \underset{=kin. energy of piston}{\underset{⏟}{p^{2} / 2 m}} + \overset{=pot. energy of piston}{\overset{⏞}{m g z}}, \end{matrix}

where

m

is the mass of the piston (in a moment, we will let

m \to \infty

and at the same time

g \to 0)

. Next, we calculate the reduced probability distribution of the piston, assuming that the total system (piston-gas system) is in the micro canonical ensemble:

\begin{aligned} ρ_{piston} (p, z) & = \frac{1}{W_{total} (E_{total}, N)} \int_{E_{total} - Δ E ⩽ H_{gas} + p^{2} / 2 m + m g z ⩽ E_{total}} d^{3 N} P d^{3 N} Q \\ = \frac{W_{gas} (E_{total} - P V - p^{2} / 2 m, V, N)}{W_{total} (E_{total}, N)} \end{aligned}

using that the force

F = m g

is also equal to

F = P / A

, where

A

is the area of the piston (so that

V = A z

is the volume occupied by the gas, hence

m g z = P V

). Now we let

m \to \infty

keeping the force

F = m g

acting on the piston constant. Then the dependence on

p

clearly drops out. Let us calculate for which

z

the probability

ρ_{piston} (z) \equiv ρ_{piston} (z, p)

of finding the piston at position

z

is maximized. This happens when

\begin{aligned} 0 & = \frac{d}{d z} W_{gas} (E_{total} - P V, V, N) = \frac{d}{d V} W_{gas} (E_{total} - P V, V, N) \\ = \frac{\partial W_{gas}}{\partial E} \cdot (- P) + \frac{\partial W_{gas}}{\partial V} = (- \frac{P}{k_{B}} \frac{\partial S_{gas}}{\partial E} + \frac{1}{k_{B}} \frac{\partial S_{gas}}{\partial V}) e^{\frac{S_{gas}}{k_{B}}} . \end{aligned}

Using

S_{gas} = k_{B} \log W_{gas}

, it follows that

\begin{matrix} (4.23) & {\frac{\partial S_{gas}}{\partial V} |}_{E, N} = P \frac{\partial S_{gas}}{\partial E} = \frac{P}{T} \end{matrix}

which gives the desired relation

\begin{matrix} (4.24) & P = {T \frac{\partial S}{\partial V} |}_{E, N} \end{matrix}

The quantity

E_{total} = E_{gas} + P V

is also called the enthalpy.
It is instructive to compare the definition of the temperature in (4.17) with the parameter

β

that arose in the Boltzmann-Maxwell distribution (3.11), which we also interpreted as temperature there. We first ask the following question: What is the probability for finding particle number 1 having momentum lying between

{\vec{p}}_{1}

and

{\vec{p}}_{1} +

d {\vec{p}}_{1}

? The answer is:

W ({\vec{p}}_{1}) d^{3} p_{1}

, where

W ({\vec{p}}_{1})

is given by

\begin{matrix} (4.25) & W ({\vec{p}}_{1}) = \int ρ (P, Q) d^{3} p_{2} \dots d^{3} p_{N} d^{3} x_{1} \dots d^{3} x_{N} \end{matrix}

This is of course nothing but the reduced probability distribution where system A consists of the the phase space coordinates

{\vec{p}}_{1}

, and system B of all other coordinates,

({\vec{x}}_{1}, {\vec{p}}_{2}, {\vec{x}}_{2}, {\vec{p}}_{3}, {\vec{x}}_{3}, \dots)

. We wish to calculate this for the ideal gas. To this end we introduce the Hamiltonian

H^{'}

and the kinetic energy

E^{'}

for the remaining atoms:

\begin{aligned} (4.26) & H^{'} = \sum_{i = 2}^{N} (\frac{{\vec{p}}_{i}^{2}}{2 m} + W ({\vec{x}}_{i})), \\ (4.27) & E^{'} = E - \frac{{\vec{p}}_{1}^{2}}{2 m}, E - H = E^{'} - H^{'} . \end{aligned}

From this we get, together with (4.25) and (4.5):

\begin{aligned} W ({\vec{p}}_{1}) & = \frac{V}{| Ω_{E} |} \int δ (E^{'} - H^{'}) \prod_{i = 2}^{N} d^{3} p_{i} d^{3} x_{i} = \frac{V | Ω_{E^{'}, N - 1} |}{| Ω_{E, N} |} \\ (4.28) & = \frac{(\frac{3}{2} N - 1)!}{π^{\frac{3}{2}} (\frac{3}{2} N - \frac{5}{2})! (2 m E)^{3 / 2}} {(\frac{E^{'}}{E})}^{\frac{3 N}{2} - \frac{5}{2}} \end{aligned}

Using now the relations

\frac{(\frac{3 N}{2} + a)!}{(\frac{3 N}{2} + b)!} \approx {(\frac{3 N}{2})}^{a - b}, for a, b ≪ \frac{3 N}{2}

we see that for a sufficiently large number of particles (e.g.

N = O (10^{23})

)

\begin{matrix} (4.29) & W ({\vec{p}}_{1}) \approx {(\frac{3 N}{4 π m E})}^{\frac{3}{2}} {(1 - \frac{{\vec{p}}_{1}^{2}}{2 m E})}^{\frac{3 N}{2} - \frac{5}{2}} \end{matrix}

Using

{(1 - \frac{a}{N})}^{b N} \underset{N \to \infty}{⟶} e^{- a b}

and

β = \frac{3 N}{2 E} (\Leftrightarrow E = \frac{3}{2} k_{B} N T)

, we find that

\begin{matrix} (4.30) & {(1 - \frac{{\vec{p}}_{1}^{2}}{2 m E})}^{\frac{3 N}{2} - \frac{5}{2}} \to_{N \to \infty}^{} e^{- \frac{3 N}{2} \frac{{\vec{p}}_{1}^{2}}{2 m E}} \end{matrix}

Consequently, we get exactly the Maxwell-Boltzmann distribution

\begin{matrix} (4.31) & W ({\vec{p}}_{1}) = {(\frac{β}{2 π m})}^{\frac{3}{2}} e^{- β \frac{{\vec{p}}_{1}^{2}}{2 m}} \end{matrix}

which confirms our interpretation of

β

β = \frac{1}{k_{B} T}

.
We can also confirm the interpretation of

β

by the following consideration: consider two initially isolated systems and put them in thermal contact. The resulting joint probability distribution is given by

\begin{matrix} (4.32) & ρ (P, Q) = \frac{1}{| Ω_{E} |} δ (\underset{system 1}{\underset{⏟}{H_{1} (P_{1}, Q_{1})}} + \underset{system 2}{\underset{⏟}{H_{2} (P_{2}, Q_{2})}} - E) . \end{matrix}

Since only the overall energy is fixed, we may write for the total allowed phase space volume (exercise):

\begin{aligned} | Ω_{E} | & = \int d E_{1} d E_{2} \underset{system 1}{\underset{⏟}{| Ω_{E_{1}} |}} \cdot \underset{system 2}{\underset{⏟}{| Ω_{E_{2}} |}} \cdot δ (E - E_{1} - E_{2}) \\ (4.33) & = \int d E_{1} e^{\frac{S_{1} (E_{1}) + S_{2} (E - E_{1})}{k_{B}}} \end{aligned}

For typical systems, the integrand is very sharply peaked at the maximum

(E_{1}^{*}, E_{2}^{*})

, as depicted in the following figure:

Figure 4.2: The joint number of states for two systems in thermal contact.

At the maximum we have

\frac{\partial S_{1}}{\partial E} (E_{1}^{*}) = \frac{\partial S_{2}}{\partial E} (E_{2}^{*})

from which we get the relation:

\begin{matrix} (4.34) & \frac{1}{T_{1}} = \frac{1}{T_{2}} = \frac{1}{T} (uniformity of temperature). \end{matrix}

Since one expects the function to be very sharply peaked at

(E_{1}^{*}, E_{2}^{*})

, the integral in (4.33) can be approximated by

S (E) \approx S_{1} (E_{1}^{*}) + S_{2} (E_{2}^{*})

which means that the entropy is (approximately) additive. Note that from the condition of

(E_{1}^{*}, E_{2}^{*})

being a genuine maximum (not just a stationary point), one gets the important stability condition

\begin{matrix} (4.35) & \frac{\partial^{2} S_{1}}{\partial E_{1}^{2}} + \frac{\partial^{2} S_{2}}{\partial E_{2}^{2}} ⩽ 0 \end{matrix}

implying

\frac{\partial^{2} S}{\partial E^{2}} ⩽ 0

if applied to two copies of the same system. We can apply the same considerations if

S

depends on additional parameters, such as other constants of motion. Denoting the parameters collectively as

X = (X_{1}, \dots, X_{n})

, the stability condition becomes

\begin{matrix} (4.36) & \sum_{i, j} \frac{\partial^{2} S}{\partial X_{i} \partial X_{j}} v_{i} v_{j} ⩽ 0 \end{matrix}

for any choice of displacements

v_{i}

(negativity of the Hessian matrix). Thus, in this case,

S

is a concave function of its arguments. Otherwise, if the Hessian matrix has a positive eigenvalue e.g. in the

i

-th coordinate direction, then the corresponding displacement

v_{i}

will drive the system to an inhomogeneous state, i.e. one where the quantity

X_{i}

takes different values in different parts of the system (different phases).

4.2.2 Microcanonical Ensemble in Quantum Mechanics

Let

H

be the Hamiltonian of a system with eigenstates

| n ⟩

and eigenvalues

E_{n}

, i.e.

H | n ⟩ = E_{n} | n ⟩

, and consider the density matrix

\begin{matrix} (4.37) & ρ = \frac{1}{W} \sum_{n : E - Δ E ⩽ E_{n} ⩽ E} | n ⟩ ⟨ n |, \end{matrix}

where the normalization constant

W

is chosen such that

tr ρ = 1

. The density matrix

ρ

is analogous to the distribution function

ρ (P, Q)

in the classical microcanonical ensemble, eq. (4.6), since it effectively amounts to giving equal probability to all eigenstates with energies lying between

E

and

E - Δ E

. By analogy with the classical case we get

\begin{matrix} (4.38) & W = number of states between E - Δ E and E, \end{matrix}

and we define the corresponding entropy

S (E)

again by

\begin{matrix} (4.39) & S (E) = k_{B} \log W (E) . \end{matrix}

Since

W (E)

is equal to the number of states with energies lying between

E - Δ E

and

E

, it also depends, strictly speaking, on

Δ E

. But for

Δ E ≲ E

and large

N

, this dependency can be neglected. Note that

\begin{aligned} S_{v . N .} (ρ) & = - k_{B} tr (ρ \log ρ) = - k_{B} \cdot \sum_{n : E - Δ E ⩽ E_{n} ⩽ E} \frac{1}{W} \log \frac{1}{W} \\ = k_{B} \log W \cdot \frac{1}{W} \sum_{n : E - Δ E ⩽ E_{n} ⩽ E} \\ = k_{B} \log W \end{aligned}

S = k_{B} \log W

is equal to the von Neumann entropy for the statistical operator

ρ

, defined in (4.37) above.

Example: Free atom in a cube We consider a free particle

(N = 1)

in a cube of side lengths

(L_{x}, L_{y}, L_{z})

. The Hamiltonian is given by

H = \frac{1}{2 m} (p_{x}^{2} + p_{y}^{2} + p_{z}^{2})

. We impose boundary conditions such that the normalized wave function

Ψ

vanishes at the boundary of the cube. This yields the eigenstates

\begin{matrix} (4.40) & Ψ (x, y, z) = \sqrt{\frac{8}{V}} \sin (k_{x} \cdot x) \sin (k_{y} \cdot y) \sin (k_{z} \cdot z) \end{matrix}

where

k_{x} = \frac{π n_{x}}{L_{x}}, \dots

, with

n_{x} = 1, 2, 3, \dots

The corresponding energy eigenvalues are given by

\begin{matrix} (4.41) & E_{n} = \frac{ℏ^{2}}{2 m} (k_{x}^{2} + k_{y}^{2} + k_{z}^{2}) \end{matrix}

since

p_{x} = \frac{ℏ}{i} \frac{\partial}{\partial x}

, etc. Recall that

W

was defined by

W = number of states | n_{x}, n_{y}, n_{z} ⟩ with E - Δ E ⩽ E_{n} ⩽ E

The following figure gives a sketch of this situation (with

k_{z} = 0

Figure 4.3: Number of states with energies lying between

E - Δ E

and

E

In the continum approximation we have (recalling that

ℏ = \frac{h}{2 π}

\begin{aligned} W = \sum_{E - Δ E ⩽ E_{n} ⩽ E} 1 \approx \int_{{E - Δ E ⩽ E_{n} ⩽ E}} d^{3} n \\ = \int_{{E - Δ E ⩽ \frac{ℏ^{2}}{2 m} (k_{x}^{2} + k_{y}^{2} + k_{z}^{2}) ⩽ E}} \frac{L_{x} L_{y} L_{z}}{π^{3}} d^{3} k \\ = {(\frac{2 m}{ℏ^{2}})}^{\frac{3}{2}} \frac{V}{π^{3}} \int_{{E - Δ E ⩽ E^{'} ⩽ E}} \frac{1}{2} E^{' \frac{1}{2}} d E^{'} \int_{1 / 8 of S^{2}} d^{2} Ω \\ (4.42) & = {\frac{4 π}{3} \frac{V}{(2 π)^{3}} {(\frac{2 m E}{ℏ^{2}})}^{\frac{3}{2}} |}_{E - Δ E}^{E} \end{aligned}

If we compute

W

according to the definition in classical mechanics, we would get

\begin{aligned} W & = \int_{{E - Δ E ⩽ H ⩽ E}} d^{3} p d^{3} x = V \int_{{E - Δ E ⩽ \frac{{\vec{p}}^{2}}{2 m} ⩽ E}} d^{3} p \\ = V (2 m)^{\frac{3}{2}} \int_{{E - Δ E ⩽ E^{'} ⩽ E}} \frac{1}{2} E^{' \frac{1}{2}} d E^{'} \int_{S^{2}} d^{2} Ω \\ = {\frac{4 π}{3} V (2 m E)^{\frac{3}{2}} |}_{E - Δ E}^{E} \end{aligned}

This is just

h^{3}

times the quantum mechanical result. For the case of

N

particles, this suggests the following relation

^{1}

\begin{matrix} (4.44) & W_{N}^{qm} \approx \frac{1}{h^{3 N}} W_{N}^{cl} \end{matrix}

This can be understood intuitively by recalling the uncertainty relation

Δ p Δ x ≳ h

, together with

p \sim \frac{π n ℏ}{V^{\frac{1}{3}}}, n \in N

4.2.3 Mixing entropy of the ideal gas

A puzzle concerning the definition of entropy in the micro-canonical ensemble (e.g. for an ideal gas) is revealed if we consider the following situation of two chambers, each of which is filled with an ideal gas:

Figure 4.4: Two gases separated by a removable wall.

The total volume is given by

V = V_{1} + V_{2}

, the total particle number by

N = N_{1} + N_{2}

and the total energy by

E = E_{1} + E_{2}

. Both gases are at the same temperature

T

. Using the expression (4.16) for the classical ideal gas, the entropies

S_{i} (N_{i}, V_{i}, E_{i})

are calculated as

\begin{matrix} (4.45) & S_{i} (N_{i}, V_{i}, E_{i}) = N_{i} k_{B} \log [V_{i} {(\frac{4 π e m_{i} E_{i}}{3 N_{i}})}^{\frac{3}{2}}] \end{matrix}

The wall is now removed and the gases can mix. The temperature of the resulting ideal gas is determined by

\begin{matrix} (4.46) & \frac{3}{2} k_{B} T = \frac{E_{1} + E_{2}}{N_{1} + N_{2}} = \frac{E_{i}}{N_{i}} \end{matrix}

The total entropy S is now found as (we assume

m_{1} = m_{2} \equiv m

for simplicity):

\begin{aligned} S & = N k_{B} \log [V {(2 π e m k_{B} T)}^{\frac{3}{2}}] \\ (4.47) & = \underset{Δ S}{\underset{⏟}{N k_{B} \log V - N_{1} k_{B} \log V_{1} - N_{2} k_{B} \log V_{2}}} + S_{1} + S_{2}, \end{aligned}

From this it follows that the mixing entropy

Δ S

is given by

\begin{aligned} Δ S & = N_{1} k_{B} \log \frac{V}{V_{1}} - N_{2} k_{B} \log \frac{V}{V_{2}} \\ (4.48) & = - N k_{B} \sum_{i} c_{i} \log v_{i} \end{aligned}

with

c_{i} = \frac{N_{i}}{N}

and

v_{i} = \frac{V_{i}}{V}

. This holds also for an arbitrary number of components and raises the following paradox: if both gases are identical with the same density

\frac{N_{1}}{N} = \frac{N_{2}}{N}

, from a macroscopic viewpoint clearly "nothing happens" as the wall is removed. Yet,

Δ S \neq 0

. The resolution of this paradox is that the particles have been treated as distinguishable, i.e. the states

have been counted as microscopically different. However, if both gases are the same, they ought to be treated as indistinguishable. This change results in a different definition of

W

in both cases. Namely, depending on the case considered, the correct definition of

W

should be:

\begin{matrix} (4.49) & W (E, V, {N_{i}}) := {\begin{cases} | Ω (E, V, {N_{i}}) | & if distinguishable \\ \frac{1}{\prod_{i} N_{i}!} | Ω (E, V, {N_{i}}) | & if indistinguishable \end{cases} \end{matrix}

where

N_{i}

is the number of particles of species

i

. Thus, the second definition is the physically correct one in our case. With this change (which in turn results in a different definition of the entropy

S

), the mixing entropy of two identical gases is now

Δ S = 0

. In quantum mechanics the symmetry factor

\frac{1}{N!}

W^{qm}

(for each species of indistinguishable particles) is automatically included due to the Bose/Fermi alternative, which we shall discuss later, leading to an automatic resolution of the paradox.

The non-zero mixing entropy of two identical gases is seen to be unphysical also
at the classical level because the entropy should be an extensive quantity. Indeed, the arguments of the previous subsection suggest that for

V_{1} = V_{2} = \frac{1}{2} V

and

N_{1} = N_{2} = \frac{1}{2} N

we have

\begin{aligned} | Ω (E, V, N) | & = \int d E^{'} | Ω (E - E^{'}, \frac{V}{2}, \frac{N}{2}) | | Ω (E^{'}, \frac{V}{2}, \frac{N}{2}) | \\ \approx {| Ω (\frac{1}{2} E, \frac{1}{2} V, \frac{1}{2} N) |}^{2} \end{aligned}

(the maximum of the integrand above should be sharply peaked at

E^{'} = \frac{E}{2}

). It follows for the entropy that, approximately,

\begin{matrix} (4.50) & S (E, N, V) = 2 S (\frac{E}{2}, \frac{N}{2}, \frac{V}{2}) \end{matrix}

The same consideration can be repeated for

ν

subsystems and yields

\begin{matrix} (4.51) & S (E, N, V) = ν S (\frac{E}{ν}, \frac{N}{ν}, \frac{V}{ν}) \end{matrix}

and thus

\begin{matrix} (4.52) & S (E, N, V) = N \cdot σ (ϵ, n) \end{matrix}

for some function

σ

in two variables, where

ϵ = \frac{E}{N}

is the average energy per particle and

n = \frac{N}{V}

is the particle density. Hence

S

is an extensive quantity, i.e.

S

is proportional to

N

. A non-zero mixing entropy would contradict the extensivity property of

S

4.3 Canonical Ensemble

4.3.1 Canonical Ensemble in Quantum Mechanics

We consider a system (system A) in thermal contact with an (infinitely large) heat reservoir (system B):

Figure 4.5: A small system in contact with a large heat reservoir.

The overall energy

E = E_{A} + E_{B}

of the combined system is fixed, as are the particle numbers

N_{A}, N_{B}

of the subsystems. We think of

N_{B}

as much larger than

N_{A}

; in fact we shall let

N_{B} \to \infty

at the end of our derivation. We accordingly describe the total Hilbert space of the system by a tensor product,

H = H_{A} \otimes H_{B}

. The total Hamiltonian of the combined system is

\begin{matrix} (4.53) & H = \underset{system A}{\underset{⏟}{H_{A}}} + \underset{system B}{\underset{⏟}{H_{B}}} + \underset{interaction (neglected)}{\underset{⏟}{H_{A B}}}, \end{matrix}

where the interaction is needed in order that the subsystems can interact with each other. Its precise form is not needed, as we shall assume that the interaction strength is arbitrarily small. The Hamiltonians

H_{A}

and

H_{B}

of the subsystems

A

and

B

act on the Hilbert spaces

H_{A}

and

H_{B}

, and we choose bases so that:

\begin{aligned} H_{A} | n ⟩_{A} & = E_{n}^{(A)} | n ⟩_{A} \\ H_{B} | m ⟩_{B} & = E_{m}^{(B)} | m ⟩_{B} \\ | n, m ⟩ & = | n ⟩_{A} \otimes | m ⟩_{B} . \end{aligned}

Since

E

is conserved, the quantum mechanical statistical operator of the combined system is given by the micro canonical ensemble with density matrix

\begin{matrix} (4.54) & ρ = \frac{1}{W} \cdot \sum_{\begin{matrix} n, m : \\ E - Δ E ⩽ E_{n}^{(A)} + E_{m}^{(B)} ⩽ E \end{matrix}} | n, m ⟩ ⟨ n, m | . \end{matrix}

The reduced density matrix for sub system A is calculated as

ρ_{A} = \frac{1}{W} \sum_{n} \overset{= W_{B} (E - E_{n}^{(A)})}{\overset{⏞}{(\sum_{m : E - E_{n}^{(A)} - Δ E ⩽ E_{m}^{(A)} ⩽ E - E_{n}^{(A)}} 1)}} | n ⟩_{A A} ⟨ n | .

Now, using the extensiveness of the entropy

S_{B}

of system B we find (with

n_{B} = N_{B} / V_{B}

the particle density and

σ_{B}

the entropy per particle of system B )

\begin{aligned} \log W_{B} (E - E_{n}^{(A)}) & = \frac{1}{k_{B}} S_{B} (E - E_{n}^{(A)}) \\ = \frac{N_{B}}{k_{B}} σ_{B} (\frac{E}{N_{B}}, n_{B}) - \frac{N_{B}}{k_{B}} \frac{E_{n}^{(A)}}{N_{B}} \frac{\partial σ_{B}}{\partial ϵ} (\frac{E}{N_{B}}, n_{B}) \\ + \frac{1}{2} \underset{⏟}{\frac{N_{B}}{k_{B}} {(\frac{E_{n}^{(A)}}{N_{B}})}^{2} \frac{\partial^{2} σ_{B}}{\partial ϵ^{2}} (\frac{E}{N_{B}}, n_{B}) + \dots} . \\ = O (\frac{1}{N_{B}}) \to 0 (as N_{B} \to \infty, i.e., reservoir \infty -large!) \end{aligned}

Thus, using

β = \frac{1}{k_{B} T}

and

\frac{1}{T} = \frac{\partial S}{\partial E}

, we have for an infinite reservoir

\begin{matrix} (4.55) & \log W_{B} (E - E_{n}^{(A)}) = \log W_{B} (E) - E_{n}^{(A)} β \end{matrix}

which means

\begin{matrix} (4.56) & W_{B} (E - E_{n}^{(A)}) = \frac{1}{Z} e^{- β E_{n}^{(A)}} \end{matrix}

Therefore, we find the following expression for the reduced density matrix for system A:

\begin{matrix} (4.57) & ρ_{A} = \frac{1}{Z} \sum_{n} e^{- β E_{n}^{(A)}} | n ⟩_{A} \otimes_{A} ⟨ n |, \end{matrix}

where

Z = Z (β, N_{A}, V_{A})

is called canonical partition function. Explicitly:

\begin{matrix} (4.58) & Z (N, β, V) = tr [e^{- β H (V, N)}] = \sum_{n} e^{- β E_{n}} \end{matrix}

Here we have dropped the subscripts "A" referring to our sub system since we can at this point forget about the role of the reservoir B (so

H = H_{A}, V = V_{A}

etc. in this formula). This finally leads to the statistical operator of the canonical ensemble:

\begin{matrix} (4.59) & ρ = \frac{1}{Z (β, N, V)} e^{- β H (N, V)} . \end{matrix}

Particular, the only quantity characterizing the reservoir entering the formula is the temperature

T

4.3.2 Canonical Ensemble in Classical Mechanics

In the classical case we can make similar considerations as in the quantum mechanical case. Consider the same situation as above. The phase space coordinates of the combined system are divided up as

(P, Q) = (\underset{system A}{\underset{⏟}{P_{A}, Q_{A}}}, \underset{system B}{\underset{⏟}{P_{B}, Q_{B}}}) .

The Hamiltonian of the total system is written as

\begin{matrix} (4.60) & H (P, Q) = H_{A} (P_{A}, Q_{A}) + H_{B} (P_{B}, Q_{B}) + H_{A B} (P, Q) \end{matrix}

H_{A B}

accounts for the interaction between the particles from both systems and is neglected in the following. By analogy with the quantum mechanical case we get a reduced probability distribution

ρ_{A}

for sub system A:

ρ_{A} (P_{A}, Q_{A}) = \int d^{3 N_{B}} P_{B} d^{3 N_{B}} Q_{B} ρ (P_{A}, Q_{A}, P_{B}, Q_{B}),

with

ρ = \frac{1}{W} \cdot {\begin{cases} 1 & if E - Δ E ⩽ H (P, Q) ⩽ E \\ 0 & otherwise. \end{cases}

From this it follows that

\begin{aligned} ρ_{A} (P_{A}, Q_{A}) & = \frac{1}{W} \int_{{E - Δ E ⩽ H_{A} + H_{B} ⩽ E}} d^{3 N_{B}} P_{B} d^{3 N_{B}} Q_{B} \\ = \frac{1}{W} \int_{{E - H_{A} (P_{A}, Q_{A}) - Δ E ⩽ H_{B} (P_{B}, Q_{B}) ⩽ E + H_{A} (P_{A}, Q_{A})}} d^{3 N_{B}} P_{B} d^{3 N_{B}} Q_{B} \\ = \frac{1}{W (E)} W_{2} (E - H_{A} (P_{A}, Q_{A})) \end{aligned}

It is then demonstrated precisely as in the quantum mechanical case that the reduced density matrix

ρ \equiv ρ_{A}

for system A is given by (for an infinitely large system B ):

\begin{matrix} (4.61) & ρ (P, Q) = \frac{1}{Z} e^{- β H (P, Q)}, \end{matrix}

where

P = P_{A}, Q = Q_{A}, H = H_{A}

in this formula. The classical canonical partition function

Z = Z (β, N, V)

for

N

indistinguishable particles is conventionally fixed by

{(h^{3 N} N!)}^{- 1} \int ρ d^{3 N} P d^{3 N} Q = 1

. For an external square well potential

(H (P, Q) =

\sum_{i = 1}^{3 N} \frac{P_{i}^{2}}{2 m} + V_{N} (Q))

confining the system to a box of volume

V

this leads to

\begin{aligned} Z & := (\frac{1}{N! h^{3 N}}) \int d^{3 N} P d^{3 N} Q e^{- β H (P, Q)} \\ (4.62) & = \frac{1}{N! h^{3 N}} {(\frac{2 π m}{β})}^{3 N / 2} \int_{V^{N}} d^{3 N} Q e^{- β V_{N} (Q)} \end{aligned}

The quantity

λ := \frac{h}{\sqrt{2 π m k_{B} T}}

is sometimes called the "thermal deBroglie wavelength". As a rule of thumb, quantum effects start being significant if

λ

exceeds the typical dimensions of the system, such as the mean free path length or system size. Using this definition, we can write

\begin{matrix} (4.63) & Z (β, N, V) = \frac{1}{N! λ^{3 N}} \int_{V^{N}} d^{3 N} Q e^{- β V_{N} (Q)} \end{matrix}

Of course, this form of the partition function applies to classical, not quantum, systems. The unconventional factor of

h^{3 N}

is nevertheless put in by analogy with the quantum mechanical case because one imagines that the "unit" of phase space for

N

particles (i.e. the phase space measure) is given by

d^{3 N} P d^{3 N} Q / (N! h^{3 N})

, inspired by the uncertainty principle

Δ Q Δ P \sim h

, see e.g. our discussion of the atom in a cube for why the normalized classical partition function then approximates the quantum partition function. The motivation of the factor

N

! is due to the fact that we want to treat the particles as indistinguishable. Therefore, a permuted phase space configuration should be viewed as equivalent to the unpermuted one, and since there are

N

! permutations, the factor

1 / N

! effectively compensates a corresponding overcounting (here we implicitly assume that

V_{N}

is symmetric under permutations). For the discussion of the

N

!-factor, see also our discussion on mixing entropy. In practice, these factors often do not play a major role because the quantities most directly related to thermodynamics are derivatives of

\begin{matrix} (4.64) & F := - β^{- 1} \log Z (β, N, V) \end{matrix}

for instance

P = - \partial F / {\partial V |}_{T, N}

, see chapter 6.5 for a detailed discussion of such relations.

F

is also called the free energy.

Example:

One may use the formula (4.61) to obtain the barometric formula for the average particle density at a position

\vec{x}

in a given external potential. In this case the Hamiltonian

H

is given by

H = \sum_{i = 1}^{N} \frac{{\vec{p}}_{i}^{2}}{2 m} + \sum_{i = 1}^{N} \underset{\begin{matrix} external potential, \\ no interaction \\ between the particles \end{matrix}}{\underset{⏟}{W ({\vec{x}}_{i})}},

which yields the probability distribution

\begin{matrix} (4.65) & ρ (P, Q) = \frac{1}{Z} e^{- β H (P, Q)} = \frac{1}{Z} \prod_{i = 1}^{N} e^{- β (\frac{{\vec{p}}_{i}^{2}}{2 m} + W ({\vec{x}}_{i}))} . \end{matrix}

The particle density

n (\vec{x})

is given by

\begin{matrix} (4.66) & n (\vec{x}) = ⟨ \sum_{i = 1}^{N} δ^{3} ({\vec{x}}_{i} - \vec{x}) ⟩ = N \int d^{3} p \frac{1}{Z_{1}} e^{- β (\frac{{\vec{p}}^{2}}{2 m} + W (\vec{x}))} \end{matrix}

where

\begin{matrix} (4.67) & Z_{1} = \int d^{3} p d^{3} x e^{- β (\frac{{\vec{p}}^{2}}{2 m} + W (\vec{x}))} = {(\frac{2 π m}{β})}^{\frac{3}{2}} \int d^{3} x e^{- β (W (\vec{x}))} \end{matrix}

From this we obtain the barometric formula

\begin{matrix} (4.68) & n (\vec{x}) = n_{0} e^{- β W (\vec{x})} \end{matrix}

with

n_{0}

given by

\begin{matrix} (4.69) & n_{0} = \frac{N}{\int d^{3} x e^{- β W (\vec{x})}} \end{matrix}

In particular, for the gravitational potential,

W (x, y, z) = m g z

, we find

\begin{matrix} (4.70) & n (z) = n_{0} e^{- z \frac{m g}{k_{B} T}} \end{matrix}

To double-check with our intuition we provide an alternative derivation of this formula: let

P (\vec{x})

be the pressure at

\vec{x}

and

\vec{F} (\vec{x}) = - \vec{\nabla} W (\vec{x})

the force acting on one particle. For the average force density

\vec{f} (\vec{x})

in equilibrium we thus obtain

\begin{matrix} (4.71) & \vec{f} (\vec{x}) = \vec{n} (\vec{x}) \vec{F} (\vec{x}) = - \vec{n} (\vec{x}) \vec{\nabla} W (\vec{x}) = \vec{\nabla} P (\vec{x}) \end{matrix}

Together with

P (\vec{x}) = n (\vec{x}) k_{B} T

it follows that

\begin{matrix} (4.72) & k_{B} T \vec{\nabla} n (\vec{x}) = - n (\vec{x}) \vec{\nabla} W (\vec{x}) \end{matrix}

and thus

\begin{matrix} (4.73) & k_{B} T \vec{\nabla} \log n (\vec{x}) = - \vec{\nabla} W (\vec{x}) \end{matrix}

which again yields the barometric formula

\begin{matrix} (4.74) & n (\vec{x}) = n_{0} e^{- β W (\vec{x})} \end{matrix}

4.3.3 Equidistribution Law and Virial Theorem in the Canonical Ensemble

We first derive the equidistribution law for classical systems with a Hamiltonian of the form

\begin{matrix} (4.75) & H = \sum_{i = 1}^{N} \frac{{\vec{p}}_{i}^{2}}{2 m_{i}} + V (Q), Q = ({\vec{x}}_{1}, \dots, {\vec{x}}_{N}) \end{matrix}

We take as the probability distribution the canonical ensemble as discussed in the previous subsection, with probability distribution given by

\begin{matrix} (4.76) & ρ (P, Q) = \frac{1}{Z} e^{- β H (P, Q)} \end{matrix}

Then we have for any observable

A (P, Q)

\begin{aligned} 0 & = \int d^{3 N} P d^{3 N} Q \frac{\partial}{\partial p_{i α}} (A (P, Q) ρ (P, Q)) \\ = \int d^{3 N} P d^{3 N} Q (\frac{\partial A}{\partial p_{i α}} - β A \frac{\partial H}{\partial p_{i α}}) ρ (P, Q) \\ (4.77) & = ⟨ \frac{\partial A}{\partial p_{i α}} ⟩ - β ⟨ A \frac{\partial H}{\partial p_{i α}} ⟩, i = 1, \dots, N, α = 1, 2, 3 . \end{aligned}

From this we obtain the relation

\begin{matrix} (4.78) & k_{B} T ⟨ \frac{\partial A}{\partial p_{i α}} ⟩ = ⟨ A \frac{\partial H}{\partial p_{i α}} ⟩ \end{matrix}

and similarly

\begin{matrix} (4.79) & k_{B} T ⟨ \frac{\partial A}{\partial x_{i α}} ⟩ = ⟨ A \frac{\partial H}{\partial x_{i α}} ⟩ \end{matrix}

The function

A

should be chosen such that that the integrand falls of sufficiently rapidly. For

A (P, Q) = p_{i α}

and

A (P, Q) = x_{i α}

, respectively, we find

\begin{aligned} (4.80) & ⟨ p_{i α} \frac{\partial H}{\partial p_{i α}} ⟩ & = ⟨ \frac{p_{i α}^{2}}{m_{i}} ⟩ = k_{B} T \\ (4.81) & ⟨ x_{i α} \frac{\partial H}{\partial x_{i α}} ⟩ & = ⟨ x_{i α} \frac{\partial V}{\partial x_{i α}} ⟩ = k_{B} T \end{aligned}

The first of these equations is called equipartition or equidistribution law.
We split up the potential

V

into a part coming from the interactions of the particles and
a part describing an external potential, i.e.

\begin{aligned} (4.82) & V (Q) = & \underset{interactions}{\underset{⏟}{\sum_{i < j} V ({\vec{x}}_{i} - {\vec{x}}_{j})}} + \underset{external potential}{\underset{⏟}{\sum_{i} W ({\vec{x}}_{i},)}} \\ = & \frac{1}{2} \sum_{i \neq j} V ({\vec{x}}_{i} - {\vec{x}}_{j}) \end{aligned}

Writing

{\vec{x}}_{k l} \equiv {\vec{x}}_{k} - {\vec{x}}_{l}

for the relative distance between the

k

-th and the

l

-th particle, we find by a lengthy calculation:

\begin{aligned} \sum_{i, α} ⟨ x_{i α} \frac{\partial V (Q)}{\partial x_{i α}} ⟩ & = \frac{1}{2} \sum_{i, α} \sum_{k \neq l} ⟨ x_{i α} \frac{\partial}{\partial x_{i α}} V ({\vec{x}}_{k} - {\vec{x}}_{l}) ⟩ + \sum_{i, α} \sum_{k} ⟨ x_{i α} \frac{\partial}{\partial x_{i α}} W ({\vec{x}}_{k}) ⟩ \\ = \frac{1}{2} \sum_{i, α, β, k \neq l} ⟨ x_{i α} (\frac{\partial V}{\partial x_{β}}) ({\vec{x}}_{k} - {\vec{x}}_{l}) ⟩ (δ_{i k} - δ_{i l}) δ_{α β} + \int d^{3} x ⟨ \sum_{k} δ^{3} (\vec{x} - {\vec{x}}_{k}) ⟩ \vec{x} \cdot \vec{\nabla} W (\vec{x}) \\ = \frac{1}{2} \sum_{k \neq l} ⟨ ({\vec{x}}_{k} - {\vec{x}}_{l}) \vec{\nabla} V ({\vec{x}}_{k} - {\vec{x}}_{l}) ⟩ - \int d^{3} x \underset{\vec{\nabla} P}{\underset{⏟}{n (\vec{x}) \vec{F} (\vec{x})}} \cdot \vec{x} \\ = \frac{1}{2} \sum_{k \neq l} ⟨ {\vec{x}}_{k l} \frac{\partial V}{\partial {\vec{x}}_{k l}} ⟩ + \int d^{3} x \underset{= 3}{\underset{⏟}{\vec{\nabla} \cdot \vec{x}}} P \\ = \frac{1}{2} \sum_{k \neq l} ⟨ {\vec{x}}_{k l} \frac{\partial V}{\partial {\vec{x}}_{k l}} ⟩ + 3 P V, \end{aligned}

where we assumed that the potential

W

is constant within the volume, so that the pressure is also constant. According to the equipartition law we have

\sum_{i α} ⟨ x_{i α} \frac{\partial V}{\partial x_{i α}} ⟩ =

3 N k_{B} T

and therefore obtain the virial law for classical systems

\begin{matrix} (4.83) & P V = N k_{B} T - \underset{= 0 for ideal gas}{\underset{⏟}{\frac{1}{6} \sum_{k \neq l} ⟨ {\vec{x}}_{k l} \frac{\partial V}{\partial {\vec{x}}_{k l}} ⟩}} . \end{matrix}

Thus, interactions tend to increase

P

when they are repulsive, and tend to decrease

P

when they are attractive. This is of course consistent with our intuition.

A well-known application of the virial law is the following example:

Example: estimation of the mass of distant galaxies:

Figure 4.6: Distribution and velocity of stars in a galaxy.

We use the relations (4.80) we found above,

⟨ \frac{{\vec{p}}_{1}^{2}}{m_{1}} ⟩ = ⟨ \frac{\partial V}{\partial {\vec{x}}_{1}} {\vec{x}}_{1} ⟩ = 3 k_{B} T

assuming that the stars in the outer region have reached thermal equilibrium, so that they can be described by the canonical ensemble. We put

\vec{v} = {\vec{p}}_{1} / m_{1}, v = | \vec{v} |

and

R = | \vec{x_{1}} |

, and assume that

⟨ {\vec{v}}^{2} ⟩ \approx ⟨ v ⟩^{2}

as well as

\begin{matrix} (4.84) & ⟨ \frac{\partial V}{\partial {\vec{x}}_{1}} {\vec{x}}_{1} ⟩ = m_{1} G \sum_{j \neq 1} ⟨ \frac{m_{j}}{| {\vec{x}}_{1} - {\vec{x}}_{j} |} ⟩ \approx m_{1} M G ⟨ \frac{1}{R} ⟩ \approx m_{1} M G \frac{1}{⟨ R ⟩} \end{matrix}

supposing that the potential felt by star 1 is dominated by the Newton potential created by the core of the galaxy containing most of the mass

M \approx \sum_{j} m_{j}

. Under these approximations, we conclude that

\begin{matrix} (4.85) & \frac{M}{⟨ R ⟩} \approx \frac{⟨ v ⟩^{2}}{G} \end{matrix}

This relation is useful for estimating

M

because

⟨ R ⟩

and

⟨ v ⟩

can be measured or estimated. Typically

⟨ v ⟩ = O (10^{2} \frac{km}{s})

Continuing with the general discussion, if the potential attains a minimum at

Q = Q_{0}

we have

\frac{\partial V}{\partial x_{i α}} (Q_{0}) = 0

, as sketched in the following figure:

Figure 4.7: Sketch of a potential

V

of a lattice with a minimum at

Q_{0}

Setting

V (Q_{0}) \equiv V_{0}

, we can Taylor expand around

Q_{0}

\begin{matrix} (4.86) & V (Q) = V_{0} + \frac{1}{2} \sum \underset{= f_{i α j β}}{\underset{⏟}{\frac{\partial^{2} V}{\partial x_{i α} \partial x_{j β}} (Q_{0})}} Δ x_{i α} Δ x_{j β} + \dots \end{matrix}

where

Δ Q = Q - Q_{0}

. In this approximation

| Δ Q | ≪ 1

, i.e. for small oscillations around the minimum) we have, setting the zero point energy

V_{0} = 0

\begin{aligned} (4.87) & \sum_{i, α} ⟨ x_{i α} \frac{\partial V}{\partial x_{i α}} ⟩ & \approx 2 ⟨ V ⟩ = \sum_{i, α} k_{B} T = 3 N k_{B} T \\ (4.88) & \sum_{i, α} ⟨ \frac{p_{i α}^{2}}{m_{i}} ⟩ & = 2 ⟨ \sum_{i} \frac{{\vec{p}}_{i}^{2}}{2 m_{i}} ⟩ = 3 N k_{B} T \end{aligned}

It follows that the mean energy

⟨ H ⟩

of the system is given by

\begin{matrix} (4.89) & ⟨ H ⟩ = 3 N k_{B} T . \end{matrix}

This relation is called the Dulong-Petit law. For real lattice systems there are deviations from this law at low temperature

T

through quantum effects and at high temperature

T

through non-linear effects, which are not captured by the approximation (4.86).

Our discussion for classical systems can be adapted to the quantum mechanical context, but there are some changes. Consider the canonical ensemble with statistical operator

ρ = \frac{1}{Z} e^{- β H}

. From this it immediately follows that

\begin{matrix} (4.90) & [ρ, H] = 0 \end{matrix}

which in turn implies that for any observable

A

we have

\begin{matrix} (4.91) & ⟨ [H, A] ⟩ = tr (ρ [A, H]) = - tr ([ρ, H] A) = 0 . \end{matrix}

Now let

A = \sum_{i} {\vec{x}}_{i} \cdot {\vec{p}}_{i}

and assume, as before, that

\begin{matrix} (4.92) & H = \sum_{i} \frac{{\vec{p}}_{i}^{2}}{2 m_{i}} + V_{N} (Q) \end{matrix}

By using

[a, b c] = [a, b] c + b [a, c]

and

{\vec{p}}_{j} = \frac{ℏ}{i} \frac{\partial}{\partial {\vec{x}}_{j}}

we obtain

\begin{aligned} [H, A] & = \sum_{i, j} [\frac{{\vec{p}}_{i}^{2}}{2 m_{i}}, {\vec{x}}_{j}] \cdot {\vec{p}}_{j} + \sum_{j} {\vec{x}}_{j} [V (Q), {\vec{p}}_{j}] \\ = \frac{ℏ}{i} \sum_{j} \frac{{\vec{p}}_{j}^{2}}{m_{j}} + i ℏ \sum_{j} {\vec{x}}_{j} \partial_{{\vec{x}}_{j}} V (Q) \end{aligned}

which gives

\begin{matrix} (4.93) & \sum_{j} ⟨ {\vec{x}}_{j} \frac{\partial V}{\partial {\vec{x}}_{j}} ⟩ = 2 ⟨ H_{kin} ⟩ (ℏ cancels out) . \end{matrix}

Applying now the same arguments as in the classical case to evaluate the left hand side leads to

\begin{matrix} (4.94) & P V = \underset{\begin{matrix} \neq N k_{B} T for ideal gas \\ \Rightarrow quantum effects! \end{matrix}}{\underset{⏟}{\frac{2}{3} ⟨ H_{kin} ⟩}} - \frac{1}{6} \sum_{k \neq l} ⟨ {\vec{x}}_{k l} \frac{\partial V}{\partial {\vec{x}}_{k l}} ⟩ . \end{matrix}

For an ideal gas the contribution from the potential is by definition absent, but the contribution from the kinetic piece does not give the same formula as in the classical case, as we will discuss in more detail below in chapter 5. Thus, even for an ideal quantum gas

(V = 0)

, the classical formula

P V = N k_{B} T

receives corrections!

4.4 Grand Canonical Ensemble

This ensemble describes the following physical situation: a small system (system A) is coupled to a large reservoir (system B). Energy and particle exchange between A and B are possible.

Figure 4.8: A small system coupled to a large heat and particle reservoir.

The treatment of this ensemble is similar to that of the canonical ensemble. For definiteness, we consider the quantum mechanical case. We have

E = E_{A} + E_{B}

for the total energy, and

N = N_{A} + N_{B}

for the total particle number. The total system

A + B

is described by the microcanonical ensemble, since

E

and

N

are conserved. The Hilbert space for the total system is again a tensor product, and the statistical operator

ρ

of the total system is accordingly given by

\begin{matrix} (4.95) & ρ = \frac{1}{W} \cdot \sum_{\begin{matrix} E - Δ E ⩽ E_{n}^{(A)} + E_{m}^{(B)} ⩽ E \\ N_{n}^{(A)} + N_{m}^{(B)} = N \end{matrix}} | n, m ⟩ ⟨ n, m |, \end{matrix}

where the total Hamiltonian of the combined system is

\begin{matrix} (4.96) & H = \underset{system A}{\underset{⏟}{H_{A}}} + \underset{system B}{\underset{⏟}{H_{B}}} + \underset{interaction (neglected)}{\underset{⏟}{H_{A B}}}, \end{matrix}

We are using notations similar to the canonical ensemble such as

| n, m ⟩ = | n ⟩_{A} | m ⟩_{B}

and

\begin{aligned} (4.97) & H_{A / B} | n ⟩_{A / B} & = E_{n}^{(A / B)} | n ⟩_{A / B}, \\ (4.98) & {\hat{N}}_{A / B} | n ⟩_{A / B} & = N_{n}^{(A / B)} | n ⟩_{A / B} . \end{aligned}

Note that the particle numbers of the individual subsystems fluctuate, so we describe them by number operators

{\hat{N}}_{A}, {\hat{N}}_{B}

acting on

H_{A}, H_{B}

The statistical operator for system A is described by the reduced density matrix

ρ_{A}

for this system, namely by

\begin{matrix} (4.99) & ρ_{A} = \frac{1}{W} \sum_{n} W_{B} (E - E_{n}^{(A)}, N - N_{n}^{(A)}, V_{B}) | n ⟩_{A A} ⟨ n | \end{matrix}

As before, in the canonical ensemble, we use that the entropy is an extensive quantity to write

\begin{aligned} \log W_{B} (E_{B}, N_{B}, V_{B}) & = \frac{1}{k_{B}} S_{B} (E_{B}, N_{B}, V_{B}) \\ = \frac{1}{k_{B}} V_{B} σ_{B} (\frac{E_{B}}{V_{B}}, \frac{N_{B}}{V_{B}}), \end{aligned}

for some function

σ

in two variables. Now we let

V_{B} \to \infty

, but keeping

\frac{E}{V_{B}}

and

\frac{N}{V_{B}}

constant. Arguing precisely as in the case of the canonical ensemble, and using now also the definition of the chemical potential in (4.17), we find

\begin{matrix} (4.100) & \log W_{B} (E - E_{n}^{(A)}, N - N_{n}^{(A)}, V_{B}) = \log W_{B} (E, N, V_{B}) - β E_{n}^{(A)} + β μ N_{n}^{(A)} \end{matrix}

for

N_{B}, V_{B} \to \infty

. By the same arguments as for the temperature in the canonical ensemble the chemical potential

μ

is the same for both systems in equilibrium. We
obtain for the reduced density matrix of system A:

\begin{matrix} (4.101) & ρ_{A} = \frac{1}{Y} \sum_{n} e^{- β (E_{n}^{(A)} - μ N_{n}^{(A)})} | n ⟩_{A} \otimes_{A} ⟨ n | \end{matrix}

Thus, only the quantities

β

and

μ

characterizing the reservoir (system B ) have an influence on system A. Dropping from now on the reference to "A", we can write the statistical operator of the grand canonical ensemble as

\begin{matrix} (4.102) & ρ = \frac{1}{Y} e^{- β (H (V) - μ \hat{N} (V))}, \end{matrix}

where

H

and

\hat{N}

are now operators. The constant

Y = Y (μ, β, V)

is determined by

tr ρ_{1} = 1

and is called the grand canonical partition function. Explicitly:

\begin{matrix} (4.103) & Y (μ, β, V) = tr [e^{- β (H (V) - μ \hat{N})}] = \sum_{n} e^{- β (E_{n} - μ N_{n})} \end{matrix}

The analog of the free energy for the grand canonical ensemble is the Gibbs free energy. It is defined by

\begin{matrix} (4.104) & G := - β^{- 1} \log Y (β, μ, V) . \end{matrix}

The grand canonical partition function can be related to the canonical partition function. The Hilbert space of our system (i.e., system A) can be decomposed

\begin{matrix} (4.105) & H = \underset{vacuum}{\underset{⏟}{C}} \oplus \underset{1 particle}{\underset{⏟}{H_{1}}} \oplus \underset{2 particles}{\underset{⏟}{H_{2}}} \oplus \underset{3 particles}{\underset{⏟}{H_{3}}} \oplus \dots \end{matrix}

with

H_{N}

the Hilbert space for a fixed number

N

of particles

^{2}

, and that the total Hamiltonian is given by

\begin{aligned} H = H_{1} + H_{2} + H_{3} + \dots \\ H_{N} = \sum_{i = 1}^{N} \frac{{\vec{p}}_{i}^{2}}{2 m} + V_{N} ({\vec{x}}_{1}, \dots, {\vec{x}}_{N}) \end{aligned}

Then

[H, \hat{N}] = 0 (\hat{N}

has eigenvalue

N

H_{N})

, and

H

and

\hat{N}

are simultaneously diagonalized, with (assuming a discrete spectrum of

H

)

\begin{matrix} (4.106) & H | α, N ⟩ = E_{α, N} | α, N ⟩ and \hat{N} | α, N ⟩ = N | α, N ⟩ \end{matrix}

From this we get:

\begin{aligned} Y (β, μ, V) & = \sum_{α, N} e^{- β (E_{α, N} - μ N)} = \sum_{N} e^{+ β μ N} \sum_{α} e^{- β E_{α, N}} \\ (4.107) & = \sum_{N} \underset{canonical partition function!}{\underset{⏟}{Z (N, β, V)}} e^{β μ N}, \end{aligned}

which is the desired relation between the canonical and the grand canonical partition function.

We also note that for a potential of the standard form

V_{N} = \sum_{1 ⩽ i < j ⩽ N} V ({\vec{x}}_{i} - {\vec{x}}_{j}) + \sum_{1 ⩽ i ⩽ N} W ({\vec{x}}_{i})

we may think of replacing

H_{N} \to H_{N} - μ N

as being due to

W \to W - μ

. Therefore, for variable particle number

N

, there is no arbitrary additive constant in the 1-particle potential

W

, but it is determined by the chemical potential

μ

. A larger

μ

gives greater statistical weight in

Y

to states with larger

N

, just as larger

T

(smaller

β

) gives greater weight to states with larger

E

4.5 Summary of different equilibrium ensembles

Let us summarize the equilibrium ensembles we have discussed in this chapter in a table:

Ensemble

Defining

property

Partition

function

Statistical operator

ρ

Microcanonical

ensemble

no energy exchange

no particle exchange

W (E, N, V)

\frac{1}{W} [Θ (H - E + Δ E) - Θ (H - E)]

Canonical

ensemble

energy exchange

no particle exchange

Z (β, N, V)

\frac{1}{Z} e^{- β H}

Grand canonical

ensemble

energy exchange

particle exchange

Y (β, μ, V)

\frac{1}{Y} e^{- β (H - μ \hat{N})}

Table 4.1: Properties of the different equilibrium ensembles.

The relationship between the partition functions

W, Z, Y

and the corresponding natural termodynamic "potentials" is summarized in the following table:

Further explanations regarding the various thermodynamic potentials are given below

Ensemble

Name of

potential

Symbol

Relation with

partion function

Microcanonical

ensemble

Entropy

S (E, N, V)

S = k_{B} \log W

Canonical

ensemble

Free energy

F (β, N, V)

F = - β^{- 1} \log Z

Grand canonical

ensemble

Gibbs free

energy

G (β, μ, V)

G = - β^{- 1} \log Y

Table 4.2: Relationship to different thermodynamic potentials.
in section 6.7.

4.6 Approximation methods

For interacting systems, it is normally impossible to calculate thermodynamic quantities exactly. In these cases, approximations, estimations or numerical methods must be used. In the appendix, we present an example of a numerical method, the Monte Carlo algorithm, which can be turned into an efficient method for numerically evaluating quantities like partition functions. In problem B. 16 we discuss the mean field approximation in the example of the Ising model. In the following two subsections we present an example of an expansion technique and an example of a method based on rigorous estimations.

4.6.1 The cluster expansion

For simplicity, we consider a classical system in a box of volume

V

, with

N

-particle Hamiltonian

H_{N}

given by
where

V_{i j} = V ({\vec{x}}_{i} - {\vec{x}}_{j})

is the two-particle interaction between the

i

-th and the

j

-th particle. The partition function for the grand canonical ensemble is (see (4.103)):

\begin{aligned} Y (μ, β, V) & = \sum_{N = 0}^{\infty} e^{β μ N} Z (β, V, N) \\ (4.109) & = \sum_{N = 0}^{\infty} e^{β μ N} \cdot \frac{1}{N! λ^{3 N}} \cdot \int_{V^{N}} d^{3 N} Q e^{- β V_{N} (Q)} . \end{aligned}

Here,

λ = \frac{h}{\sqrt{2 π m k_{B} T}}

is the thermal deBroglie wavelength. To compute the remaining integral over

Q = ({\vec{x}}_{1}, \dots, {\vec{x}}_{N})

is generally impossible, but one can derive an expansion of which the first few terms may often be evaluated exactly. For this we write the integrand as

\begin{matrix} (4.110) & e^{- β V_{N}} = \exp (- β \sum_{i < j} V_{i j}) = \prod_{i < j} e^{- β V_{i j}} \equiv \prod_{i < j} (1 + f_{i j}), \end{matrix}

where we have set

f_{i j} \equiv f ({\vec{x}}_{i} - {\vec{x}}_{j}) = 1 - e^{- β ν_{i j}}

. The idea is that we can think of

| f_{i j} |

as small in some situations of interest, e.g. when the gas is dilute (such that

| V_{i j} | ≪ 1

in "most of phase space"), or when

β

is small (i.e. for large temperature

T

). With this in mind, we expand the above product as

\begin{matrix} (4.111) & e^{- β V_{N}} = 1 + \sum_{i < j} f_{i j} + \sum_{i < j, k < l} f_{i j} f_{k l} + \dots \end{matrix}

and substitute the result into the integral

\int_{V^{N}} d^{3 N} Q e^{- β V_{N} (Q)}

. The general form of the resulting integrals that we need to evaluate is suggested by the following representative example for

N = 6

particles:

\begin{aligned} (4.112) & \int d^{3} x_{1} \dots d^{3} x_{6} f_{12} f_{35} f_{45} f_{36} = \end{aligned}

To keep track of all the integrals that come up, we introduced the following convenient graphical notation. In our example, this graphical notation amounts to the following. Each circle corresponds to an an integration, e.g.

\begin{matrix} (4.114) & (1) \leftrightarrow d^{3} x_{1}, \end{matrix}

and each line corresponds to an

f_{i j}

in the integrand, e.g.

\begin{matrix} (4.115) & (1)-2 ⟷ f_{12} . \end{matrix}

The connected parts of a diagram are called "clusters". Obviously, the integral associated with a graph factorizes into the corresponding integrals for the clusters. Therefore, the "cluster integrals" are the building blocks, and we define

\begin{matrix} (4.116) & b_{l} (V, β) = \frac{1}{l! λ^{3 l - 3} V} \cdot (sum of all l -point cluster integrals) . \end{matrix}

The main result in this context, known as the linked cluster theorem

^{3}

, is that

\begin{matrix} (4.117) & \frac{1}{V} \log Y (μ, V, β) = \frac{1}{λ^{3}} \sum_{l = 1}^{\infty} b_{l} (V, β) z^{l} \end{matrix}

where

z = e^{β μ}

is sometimes called the fugacity. If the

f_{i j}

are sufficiently small, the first few terms

(b_{1}, b_{2}, b_{3}, \dots)

will give a good approximation. Explicitly, one finds (exercise):

\begin{aligned} (4.118) & b_{1} = \frac{1}{1! λ^{0} V} \int_{V} d^{3} x = 1, \\ (4.119) & b_{2} = \frac{1}{2! λ^{3} V} \int_{V^{2}} d^{3} x_{1} d^{3} x_{2} f_{12}, \\ (4.120) & b_{3} = \frac{1}{3! λ^{6} V} \int_{V^{3}} d^{3} x_{1} d^{3} x_{2} d^{3} x_{3} (\underset{\to 3}{\underset{⏟}{(f_{12} f_{23} + f_{13} f_{12} + f_{13} f_{23}}} + f_{12} f_{13} f_{23}), \end{aligned}

since the possible 1-,2- and 3-clusters are given by:

As exemplified by the first 3 terms in

b_{3}

, topologically identical clusters (i.e. ones that differ only by a permutation of the particles) give the same cluster integral. Thus, we only need to evaluate the cluster integrals for topologically distinct clusters.

Given an approximation for

\frac{1}{V} \log Y

, one obtains approximations for the equations of state etc. by the general methods described in more detail in section 6.5.

4.6.2 Peierls contours

Next, we present an example of a rigorous estimation method proving the existence of phase transition in the two-dimensional Ising model. In this model, we have spins

σ_{i} = \pm 1

on a square lattice and the Hamitonian (energy) is

H ({σ}) = - J \sum_{i k} (σ_{i} σ_{k} - 1) - b \sum_{i} σ_{i},

where the first sum is over all lattice bonds

i k

and the second sum is over all lattice sites

i

. Note that we shifted the energies in the first term such that a pair of parallel spins has vanishing contribution to the total energy. We present an argument, due to Peierls, showing that under the boundary condition that the spins on the boundary of the lattice are positive, at sufficiently low temperatures, this model shows an equilibrium magnetization. In the following, we set

b = 0

Each configuration

{σ_{i}} = {σ_{1}, σ_{2}, \dots}

of spins is in one-to-one correspondence with set of (connected) contours separating regions of positive and negative spins, known as Peierls contours. They may be chosen so that they consist of the line segments in the middle between opposite spins, see figure 4.9.

Figure 4.9: A Peierls contour

Due to our boundary condition, these contours are closed. Each pair of antiparallel spins contributes an energy of

+ 2 J

. Since the total number of antiparallel spin pairs in a configuration corresponding to the contours

C_{1}, \dots, C_{r}

is given by the sum of their lengths,

| C_{1} | + \dots + | C_{r} |

, the energy of that configuration is

\begin{matrix} (4.121) & H = 2 J \sum_{i = 1}^{r} | C_{i} | \end{matrix}

Consider the totality of configurations having a given contour

C

as a Peierls contour or domain wall. The sum of their probabilities

P_{C} = \sum_{{σ} \supset C} Z^{- 1} e^{- β H ({σ})}

is the probability that there is a Peierls contour

C

. We can find an estimate for that probability as follows. For each configuration

{σ}

containing

C

, we may define a modified configuration

{σ^{'}}

obtained by flipping all spins inside domain defined by

C

. Then

{σ^{'}}

no longer has
the Peierls contour

C

. The subset of all distinct configurations

{σ}

containing

C

leads in this way to a set of distinct configurations

{σ^{'}}

without

C

, and this subset is itself a subset of all configurations without

C

. Since, by (4.121), the ratio of the probabilities of the original and the modified configuration is

e^{- 2 β J | C |}

, and since the sum of the probabilities of configurations without

C

is at most 1 , the probability for

C

to be a Peierls contour is at most

e^{- 2 β J | C |}

Now, if the spin

σ_{x}

at some place

x

is negative, there must be a Peierls contour surrounding

x

. Therefore, the probability

P_{x}^{-}

that the spin at

x

is negative can be estimated against the sum of the probabilities

P_{C} ⩽ e^{- 2 β J | C |}

for there to be a contour

C

surrounding

x

. Thus,

P_{x}^{-} ⩽ \sum_{C} e^{- 2 β J | C |},

where the sum runs over all contours

C

surrounding

x

. Sorting them by length

l

and extending the sum to infinite lengths, we obtain

P_{x}^{-} ⩽ \sum_{l = 4}^{\infty} N (l) e^{- 2 β J l}

where

N (l)

is the number of contours of length

l

surrounding

x

. The sum starts at length 4, because the length of a contour is at least 4 times the lattice spacing, which is assumed to be 1 .

To get an estimate for

N (l)

, we observe the following. First, we consider the set of all possible shapes of contours of the given length

l

, where two contours are considered to have the same shape if they are congruent after some translation. We follow a given contour at some starting point, from which we have 2 possibilities to proceed (taking into account that the order is irrelevant). At the subsequent sites, we have at most 3 possibilities to continue, as we can not go straight back. At the last site, there is at most one possibility to close the contour, so that there are at most

2 \times 3^{l - 2}

shapes of closed curves of length

l

(Actually, this is a vast over-counting since the curves we include in this counting may neither be closed nor free of self-intersections!) Now we impose that

x

must lie within the contour. Within a contour of length

l

there can be at most

(l / 4)^{2}

points since

l / 4

is the side-length of a square of circumference

l

. If a contour surrounds

x

, then we may shift it in as many ways as there are points inside it while still keeping

x

inside. Therefore,

N (l) ⩽ 2 (l / 4)^{2} 3^{l - 2}

hence

P_{x}^{-} ⩽ \sum_{l = 4}^{\infty} 2 (l / 4)^{2} 3^{l - 2} e^{- 2 β J l} = \frac{1}{72} \sum_{l = 4}^{\infty} l^{2} e^{(- 2 β J + \log 3) l} .

By the integral test, the right hand side converges for

2 β J > \log 3

, and it tends to zero
for

β \to \infty

i.e.

T \to 0

. Hence, for every

m \in (0, 1)

, there exists a temperature

T_{0}

such that

P_{x}^{-} < \frac{1}{2} (1 - m)

for all lower temperatures. Then,

P_{x}^{+} > \frac{1}{2} (1 + m)

and thus

⟨ σ_{x} ⟩ = P_{x}^{+} - P_{x}^{-} > m

for all

T < T_{0}

. Since this holds for all lattice sites

x

, we conclude that below a certain threshold temperature, the system shows a macroscopic magnetization.

Chapter 5 The Ideal Quantum Gas

5.1 Hilbert Spaces, Canonical and Grand Canonical Formulations

When discussing the mixing entropy of classical ideal gases in section 4.2.3, we noted that Gibbs' paradox could resolved by treating the particles of the same gas species as indistinguishable. How to treat indistinguishable particles in quantum mechanics? If we have

N

particles, the state vectors

Ψ

are elements of a Hilbert space, such as

H_{N} = L^{2} (V \times \dots \times V, d^{3} x_{1} \dots d^{3} x_{N})

for particles in a box

V \subset R^{3}

without additional quantum numbers. The probability density of finding the

N

particles at prescribed positions

{\vec{x}}_{1}, \dots, {\vec{x}}_{N}

is given by

{| Ψ ({\vec{x}}_{1}, \dots, {\vec{x}}_{N}) |}^{2}

. For identical particles, this should be the same as

{| Ψ ({\vec{x}}_{σ (1)}, \dots, {\vec{x}}_{σ (N)}) |}^{2}

for any permutation

σ : (\begin{array}{cccccc} 1 & 2 & 3 & \dots & N - 1 & N \\ σ (1) & σ (2) & σ (3) & \dots & σ (N - 1) & σ (N) \end{array})

Thus, the map

U_{σ} : Ψ ({\vec{x}}_{1}, \dots, {\vec{x}}_{N}) \mapsto Ψ ({\vec{x}}_{σ (1)}, \dots, {\vec{x}}_{σ (N)})

should be represented by a phase, i.e.

U_{σ} Ψ = η_{σ} Ψ, | η_{σ} | = 1

Every permutation

σ

can be expressed as a concatenation of transpositions, i.e., an interchange of two elements. Performing a transposition

π

twice yields the original wave function, hence,

U_{π}^{2} = 1

, so

η_{π}^{2} = 1

. It follows that,

η_{π} \in {\pm 1}

. Furthermore, from

U_{σ} U_{σ^{'}} = U_{σ σ^{'}}

it follows that

η_{σ} η_{σ^{'}} = η_{σ σ^{'}}

, and as any permutation

σ

can be expressed as a product of transpositions, the only possible constant assignments for

η_{σ}

are therefore
given by

\begin{matrix} (5.1) & η_{σ} = {\begin{cases} 1 & \forall σ & (Bosons) \\ sgn (σ) & \forall σ & (Fermions). \end{cases} \end{matrix}

Here the signum of

σ

is defined as

\begin{matrix} (5.2) & sgn (σ) = (- 1)^{# {transpositions in σ}} = (- 1)^{# {"crossings" in σ}} \end{matrix}

The second characterization also makes plausible the fact that

sgn (σ)

is an invariant satisfying

sgn (σ) sgn (σ^{'}) = sgn (σ σ^{'})

Example:

Consider the following permutation:

In this example we have

sgn (σ) = + 1 = (- 1)^{4}

In order to go from the Hilbert space

H_{N}

of distinguishable particles such as

\begin{aligned} H_{N} & = \underset{N factors}{\underset{⏟}{H_{1} \otimes \dots \otimes H_{1}}} \\ H_{N} ∋ | Ψ ⟩ & = | i_{1} ⟩ \otimes \dots \otimes | i_{N} ⟩ \equiv | i_{1} \dots i_{N} ⟩ \\ H | i ⟩ & = E_{i} | i ⟩ 1-particle Hamiltonian on H_{1} \end{aligned}

to the Hilbert space for Bosons/Fermions one can apply the projection operators

\begin{aligned} P_{+} = \frac{1}{N!} \sum_{σ \in S_{N}} U_{σ} \\ P_{-} = \frac{1}{N!} \sum_{σ \in S_{N}} sgn (σ) U_{σ} . \end{aligned}

As projectors the operators

P_{\pm}

fulfill the following relations:

P_{\pm}^{2} = P_{\pm}, P_{\pm}^{†} = P_{\pm}, P_{+} P_{-} = P_{-} P_{+} = 0

The Hilbert spaces for Bosons/Fermions, respectively, are then given by

\begin{matrix} (5.3) & H_{N}^{\pm} = {\begin{cases} P_{+} H_{N} & for Bosons \\ P_{-} H_{N} & for Fermions. \end{cases} \end{matrix}

In the following, we consider

N

non-interacting, non-relativistic particles of mass

m

in a box with volume

V = L^{3}

, together with Dirichlet boundary conditions. The Hamiltonian of the system in either case is given by

\begin{matrix} (5.4) & H_{N} = \sum_{i = 1}^{N} \frac{{\vec{p}}_{i}^{2}}{2 m} = \sum_{i = 1}^{N} - \frac{ℏ^{2}}{2 m} \partial_{{\vec{x}}_{i}}^{2} \end{matrix}

The eigenstates for a single particle are given by the wave functions

\begin{matrix} (5.5) & Ψ_{\vec{k}} (\vec{x}) = \sqrt{\frac{8}{V}} \sin (k_{x} x) \sin (k_{y} y) \sin (k_{z} z) \end{matrix}

where

k_{x} = \frac{π n_{x}}{L}, \dots

, with

n_{x} = 1, 2, 3, \dots

, and similarly for the

y, z

-components. The product wave functions

Ψ_{{\vec{k}}_{1}} ({\vec{x}}_{1}) \dots Ψ_{{\vec{k}}_{N}} ({\vec{x}}_{N})

do not satisfy the symmetry requirements for Bosons/Fermions. To obtain these we have to apply the projectors

P_{\pm}

to the states

| {\vec{k}}_{1} ⟩ \otimes \dots \otimes | {\vec{k}}_{N} ⟩ \in H_{N}

. We define:

\begin{matrix} (5.6) & {| {\vec{k}}_{1}, \dots, {\vec{k}}_{N} ⟩}_{\pm} := \frac{N!}{\sqrt{c_{\pm}}} P_{\pm} (\underset{| k_{1}, \dots, k_{N} ⟩}{\underset{⏟}{| k_{1} ⟩ \otimes \dots \otimes | k_{N} ⟩}}) \end{matrix}

where

c_{\pm}

is a normalization constant, defined by demanding that

_{\pm} {⟨ {\vec{k}}_{1}, \dots, {\vec{k}}_{N} ∣ {\vec{k}}_{1}, \dots, {\vec{k}}_{N} ⟩}_{\pm} =

1. (We have used the Dirac notation

⟨ x ∣ \vec{k} ⟩ \equiv Ψ_{\vec{k}} (\vec{x})

.) Explicitly, we have:

\begin{aligned} (5.7) & {| {\vec{k}}_{1}, \dots, {\vec{k}}_{N} ⟩}_{+} = \frac{1}{\sqrt{c_{+}}} \sum_{σ \in S_{N}} | {\vec{k}}_{σ (1)}, \dots, {\vec{k}}_{σ (N)} ⟩ for Bosons, \\ (5.8) & {| {\vec{k}}_{1}, \dots, {\vec{k}}_{N} ⟩}_{-} = \frac{1}{\sqrt{c_{-}}} \sum_{σ \in S_{N}} sgn (σ) | {\vec{k}}_{σ (1)}, \dots, {\vec{k}}_{σ (N)} ⟩ for Fermions. \end{aligned}

Note, that the factor

\frac{1}{N!}

coming from

P_{\pm}

has been absorbed into

c_{\pm}

Examples:

(a) Fermions with

N = 2

: A normalized two-particle fermion state is given by

{| {\vec{k}}_{1}, {\vec{k}}_{2} ⟩}_{-} = \frac{1}{\sqrt{2}} (| {\vec{k}}_{1}, {\vec{k}}_{2} ⟩ - | {\vec{k}}_{2}, {\vec{k}}_{1} ⟩)

with

{| {\vec{k}}_{1}, {\vec{k}}_{2} ⟩}_{-} = 0

{\vec{k}}_{1} = {\vec{k}}_{2}

. This implements the Pauli principle.

More generally, for an

N

-particle fermion state we have

\begin{matrix} (5.9) & {| \dots, {\vec{k}}_{i}, \dots, {\vec{k}}_{j}, \dots ⟩}_{-} = 0 whenever {\vec{k}}_{i} = {\vec{k}}_{j} \end{matrix}

(b) Bosons with

N = 2

: A normalized two-particle boson state is given by

\begin{matrix} (5.10) & {| {\vec{k}}_{1}, {\vec{k}}_{2} ⟩}_{+} = {\begin{cases} \frac{1}{\sqrt{2}} (| {\vec{k}}_{1} {\vec{k}}_{2} ⟩ + | {\vec{k}}_{2} {\vec{k}}_{1} ⟩) & {\vec{k}}_{1} \neq {\vec{k}}_{2} \\ | {\vec{k}}_{1}, {\vec{k}}_{2} ⟩ & {\vec{k}}_{1} = {\vec{k}}_{2} \end{cases} \end{matrix}

N = 3

: A normalized three-particle boson state with

{\vec{k}}_{1} = \vec{k}, {\vec{k}}_{2} =

{\vec{k}}_{3} = \vec{p}

is given by

\begin{matrix} (5.11) & | \vec{k}, \vec{p}, \vec{p} ⟩_{+} = \frac{1}{\sqrt{3}} (| \vec{p}, \vec{p}, \vec{k} ⟩ + | \vec{p}, \vec{k}, \vec{p} ⟩ + | \vec{k}, \vec{p}, \vec{p} ⟩) . \end{matrix}

The normalization factors

c_{+}, c_{-}

are given in general as follows:
(a) Bosons: Let

n_{\vec{k}}

be the number of appearances of the mode

\vec{k}

{| {\vec{k}}_{1}, \dots, {\vec{k}}_{N} ⟩}_{+}

, i.e.

n_{\vec{k}} = \sum_{i} δ_{\vec{k}, {\vec{k}}_{i}}

. Then

c_{+}

is given by

\begin{matrix} (5.12) & c_{+} = N! \prod_{\vec{k}} n_{\vec{k}}! \end{matrix}

In example (c) above we have

n_{\vec{k}} = 1, n_{\vec{p}} = 2

and thus

c_{+} = 3! 2! 1! = 12

Note, that this is correct since

| \vec{k}, \vec{p}, \vec{p} ⟩_{+} = \frac{1}{\sqrt{12}} (| \vec{p}, \vec{p}, \vec{k} ⟩ + | \vec{p}, \vec{k}, \vec{p} ⟩ + | \vec{k}, \vec{p}, \vec{p} ⟩ + | \vec{p}, \vec{p}, \vec{k} ⟩ + | \vec{p}, \vec{k}, \vec{p} ⟩ + | \vec{k}, \vec{p}, \vec{p} ⟩,)

because there are

3! = 6

permutations in

S_{3}

.
(b) Fermions: In this case we have

c_{-} = N

!. To check this, we note that

\begin{aligned} ⟨ {\vec{k}} ∣ {\vec{k}} ⟩ & = \sum_{σ, σ^{'} \in S_{N}} \frac{1}{c_{-}} ⟨ {\vec{k}}_{σ (1)}, \dots, {\vec{k}}_{σ (N)} ∣ {\vec{k}}_{σ^{'} (1)}, \dots, {\vec{k}}_{σ^{'} (N)} ⟩ \\ = \frac{N!}{c_{-}} \sum_{σ \in S_{N}} ⟨ {\vec{k}}_{1}, \dots, {\vec{k}}_{N} ∣ {\vec{k}}_{σ (1)}, \dots, {\vec{k}}_{σ (N)} ⟩ \\ = \frac{N! n_{{\vec{k}}_{1}}! n_{{\vec{k}}_{2}}! \dots n_{{\vec{k}}_{N}}!}{c_{-}} = \frac{N!}{c_{-}} = 1 \end{aligned}

because the term under the second sum is zero unless the permuted

{\vec{k}}

's are
identical (this happens

\prod_{\vec{k}} n_{\vec{k}}

! times for either bosons or fermions), and because for fermions, the occupation numbers

n_{\vec{k}}

can be either zero or one.

The canonical partition function

Z^{\pm}

is now defined as:

\begin{matrix} (5.13) & Z^{\pm} (N, V, β) := {tr}_{H_{N}^{\pm}} (e^{- β H}) \end{matrix}

In general the partition function is difficult to calculate. It is easier to momentarily move onto the grand canonical ensemble, where the particle number

N

is variable, i.e. it is given by a particle number operator

\hat{N}

with eigenvalues

N = 0, 1, 2, \dots

. The Hilbert space is then given by the bosonic

(+)

or fermionic

(-)

Fock space

\begin{matrix} (5.14) & H^{\pm} = ⨁_{N ⩾ 0} H_{N}^{\pm} = C \oplus H_{1}^{\pm} \oplus \dots \end{matrix}

H_{N}^{\pm}

the particle number operator

\hat{N}

has eigenvalue

N

. The grand canonical partition function

Y^{\pm}

is then defined as before as (cf. (4.103) and (4.107)):

\begin{matrix} (5.15) & Y^{\pm} (μ, V, β) := {tr}_{H^{\pm}} (e^{- β (H - μ \hat{N})}) = \sum_{N = 0}^{\infty} e^{+ μ β N} Z^{\pm} (N, V, β) \end{matrix}

Another representation of the states in

H^{\pm}

is the one based on the occupation numbers

n_{\vec{k}}

:
(a)

{| {n_{\vec{k}}} ⟩}_{+}, n_{\vec{k}} = 0, 1, 2, 3, \dots

for Bosons,
(b)

{| {n_{\vec{k}}} ⟩}_{-}, n_{\vec{k}} = 0, 1

for Fermions.

In the bosonic case, one defines the creation annihilation operator for a mode

\vec{k}

\begin{aligned} (5.16) & a_{\vec{k}}^{†} {| \dots, n_{\vec{k}}, \dots ⟩}_{+} = \sqrt{1 + n_{\vec{k}}} {| \dots, n_{\vec{k}} + 1, \dots ⟩}_{+}, \\ (5.17) & a_{\vec{k}} {| \dots, n_{\vec{k}}, \dots ⟩}_{+} = \sqrt{n_{\vec{k}}} {| \dots, n_{\vec{k}} - 1, \dots ⟩}_{+} \end{aligned}

These raise/lower the number of particles in mode

\vec{k}

by one. One easily checks that these fulfill the commutation relations

\begin{matrix} (5.18) & [a_{\vec{k}}, a_{\vec{p}}^{†}] = δ_{\vec{k}, \vec{p}} [a_{\vec{k}}, a_{\vec{p}}] = [a_{\vec{k}}^{†}, a_{\vec{p}}^{†}] = 0 \end{matrix}

so they behave as the ladder operators of a system of independent harmonic oscillators (one for each mode

\vec{k}

). In particular,

{\hat{N}}_{\vec{k}} = a_{\vec{k}}^{†} a_{\vec{k}}

is the operator counting the number of particles in mode

\vec{k}

To arrive at similar creation/annihilation operators in the fermionic case, we first consider a single mode, which can be occupied by no or one particle. In the corresponding basis

{| 0 ⟩, | 1 ⟩}

, the annihilation and creation operator have the following matrix representation

\begin{matrix} (5.19) & a = (\begin{array}{cc} 0 & 1 \\ 0 & 0 \end{array}), a^{†} = (\begin{array}{cc} 0 & 0 \\ 1 & 0 \end{array}) \end{matrix}

In particular, we find that

\begin{aligned} a a^{†} & = (\begin{array}{cc} 1 & 0 \\ 0 & 0 \end{array}), & a^{†} a & = (\begin{array}{cc} 0 & 0 \\ 0 & 1 \end{array}), \\ a a & = 0, & a^{†} a^{†} = 0, \end{aligned}

which in particular imply that

\begin{matrix} (5.22) & {a, a^{†}} = 1, {a, a} = {a^{†}, a^{†}} = 0 \end{matrix}

where

{A, B} = A B + B A

denotes the anticommutator of two operators. Furthermore, we see that

a^{†} a = \hat{N}

can again be interpreted as the number operator. It is natural to require anticommutation relations also for creation/annihilation operator corresponding to different modes, i.e.

\begin{matrix} (5.23) & {a_{\vec{k}}^{†}, a_{\vec{p}}} = δ_{\vec{k}, \vec{p}} {a_{\vec{k}}, a_{\vec{p}}} = {a_{\vec{k}}^{†}, a_{\vec{p}}^{†}} = 0 \end{matrix}

To implement these we can define their action in the above basis as

\begin{aligned} (5.24) & a_{\vec{k}}^{†} {| \dots, n_{\vec{k}}, \dots ⟩}_{-} = (- 1)^{\sum_{\vec{p} < \vec{k}} n_{\vec{p}}} (1 - n_{\vec{k}}) {| \dots, n_{\vec{k}} + 1, \dots ⟩}_{-}, \\ (5.25) & a_{\vec{k}} {| \dots, n_{\vec{k}}, \dots ⟩}_{-} = (- 1)^{\sum_{\vec{p} < \vec{k}} n_{\vec{p}}} n_{\vec{k}} {| \dots, n_{\vec{k}} - 1, \dots ⟩}_{-} . \end{aligned}

For the sign factors on the right hand side, which implement the anticommutation relations between operators corresponding to different modes, one chooses an arbitrary ordering of the wave vectors

\vec{k}

Both for bosons and fermion, the operator counting the number of particles in mode

\vec{k}

{\hat{N}}_{\vec{k}} = a_{\vec{k}}^{†} a_{\vec{k}}

, with eigenvalues

n_{\vec{k}}

. The Hamiltonian may then be written as

\begin{matrix} (5.26) & H = \sum_{\vec{k}} ϵ (\vec{k}) {\hat{N}}_{\vec{k}} = \sum_{\vec{k}} ϵ (\vec{k}) a_{\vec{k}}^{†} a_{\vec{k}} \end{matrix}

where

ϵ (\vec{k}) = \frac{ℏ^{2} {\vec{k}}^{2}}{2 m}

for non-relativistic particles. With the formalism of creation and destruction operators at hand, the grand canonical partition function for bosons and fermions, respectively, may now be calculated as follows:
(a) Bosons ("+"):

\begin{aligned} Y^{+} (μ, V, β) & = \sum_{{n_{\vec{k}}}^{+}} ⟨ {n_{\vec{k}}} | e^{- β (H - μ \hat{N})} {| {n_{\vec{k}}} ⟩}_{+} \\ = \sum_{{n_{\vec{k}}}} e^{- β \sum_{\vec{k}} n_{\vec{k}} (ϵ (\vec{k}) - μ)} \\ = \prod_{\vec{k}} (\sum_{n = 0}^{\infty} e^{- β (ϵ (\vec{k}) - μ) n}) \\ (5.27) & = \prod_{\vec{k}} {(1 - e^{- β (ϵ (\vec{k}) - μ)})}^{- 1} . \end{aligned}

(b) Fermions ("-"):

\begin{aligned} Y^{-} (μ, V, β) & = \sum_{{n_{\vec{k}}}^{-}} ⟨ {n_{\vec{k}}} | e^{- β (H - μ \hat{N})} | {n_{\vec{k}}} ⟩ \\ (5.28) & = \prod_{\vec{k}} {(1 + e^{- β (ϵ (\vec{k}) - μ)})}^{1} \end{aligned}

The expected number densities

{\bar{n}}_{\vec{k}}

, which are defined as

\begin{matrix} (5.29) & {\bar{n}}_{\vec{k}} := {⟨ {\hat{N}}_{\vec{k}} ⟩}_{\pm} = {tr}_{H^{\pm}} (\frac{{\hat{N}}_{\vec{k}}}{Y^{\pm}} e^{- β (H - μ \hat{N})}) \end{matrix}

can be calculated by means of a trick. Let us consider the bosonic case ("+"). From the above commutation relations we obtain

\begin{matrix} (5.30) & {\hat{N}}_{\vec{k}} a_{\vec{p}}^{†} = a_{\vec{p}}^{†} ({\hat{N}}_{\vec{k}} + δ_{\vec{k}, \vec{p}}) and a_{\vec{p}} {\hat{N}}_{\vec{k}} = ({\hat{N}}_{\vec{k}} + δ_{\vec{k}, \vec{p}}) a_{\vec{p}} \end{matrix}

From this it follows by a straightforward calculation that

\begin{aligned} {\bar{n}}_{\vec{k}} & = {tr}_{H^{+}} (\frac{1}{Y^{+}} a_{\vec{k}}^{†} a_{\vec{k}} e^{- β (H - μ \hat{N})}) = {tr}_{H^{+}} (\frac{1}{Y^{+}} a_{\vec{k}}^{†} a_{\vec{k}} e^{- \sum_{\vec{p}} β (ϵ (\vec{p}) - μ) {\hat{N}}_{\vec{p}}}) \\ = {tr}_{H^{+}} (\frac{1}{Y^{+}} a_{\vec{k}}^{†} e^{- \sum_{\vec{p}} β (ϵ (\vec{p}) - μ) ({\hat{N}}_{\vec{p}} + δ_{\vec{k}, \vec{p}})} a_{\vec{k}}) \\ = {tr}_{H^{+}} (\frac{1}{Y^{+}} a_{\vec{k}} a_{\vec{k}}^{†} e^{- \sum_{\vec{p}} β (ϵ (\vec{p}) - μ) ({\hat{N}}_{\vec{p}} + δ_{\vec{k}, \vec{p}})}) \\ = e^{- β (ϵ (\vec{k}) - μ)} {tr}_{H^{+}} (\frac{1}{Y^{+}} \underset{1 + {\hat{N}}_{\vec{k}}}{\underset{⏟}{a_{\vec{k}} a_{\vec{k}}^{†}}} e^{- \sum_{\vec{p}} β (ϵ (\vec{p}) - μ) {\hat{N}}_{\vec{p}}}) \\ = e^{- β (ϵ (\vec{k}) - μ)} (1 + {\bar{n}}_{\vec{k}}) . \end{aligned}

Applying similar arguments in the fermionic case we find for the expected number densities:

\begin{matrix} (5.32) & \begin{array}{ll} {\bar{n}}_{\vec{k}} = \frac{1}{e^{β (ϵ (\vec{k}) - μ)} - 1}, & for bosons \\ {\bar{n}}_{\vec{k}} = \frac{1}{e^{β (ϵ (\vec{k}) - μ)} + 1}, for fermions. \end{array} \end{matrix}

These distributions are called Bose-Einstein distribution and Fermi-Dirac distribution, respectively. Note that for the case of bosons, we have to require that the chemical potential is lower than the ground state energy,

μ < ϵ (0)

, in order to avoid a diverging particle number. Also note, that the particular form of

ϵ (\vec{k})

was not important in the derivation. In particular, (5.31) and (5.32) also hold for relativistic particles (see section 5.3). The classical distribution

{\bar{n}}_{\vec{k}} \propto e^{- β ϵ (\vec{k})}

is obtained in the limit

β ϵ (\vec{k}) ≫ 1

i.e.

ϵ (\vec{k}) ≫ k_{B} T

, consistent with our experience that quantum effects are usually only important for energies that are small compared to the temperature.

The mean energy

E_{\pm}

is given by

\begin{matrix} (5.33) & E_{\pm} = ⟨ H ⟩_{\pm} = \sum_{\vec{k}} {⟨ ϵ (\vec{k}) {\hat{N}}_{\vec{k}} ⟩}_{\pm} = \sum_{\vec{k}} ϵ (\vec{k}) {\bar{n}}_{\vec{k}}^{\pm} \end{matrix}

5.2 Degeneracy pressure for free fermions

Let us now go back to the canonical ensemble, with density matrix

ρ^{\pm}

given by

\begin{matrix} (5.34) & ρ^{\pm} = \frac{1}{Z_{\frac{N}{\pm}}^{\pm}} P_{\pm} e^{- β H_{N}} \end{matrix}

Our aim is now to calculate the canonical partition function

Z_{N}^{\pm}

[for more details concerning, see e.g. Ch. 7 of M. Kardar: "Statistical Physics of Particles" Cambridge (2007),
which we mostly follow in this section]. Let

| {\vec{x}} ⟩_{\pm}

be an eigenbasis of the position operators. Then, with

η \in {+, -}

:
where

Ψ_{\vec{k}} (\vec{x}) \in H_{1}

are the 1-particle wave functions and

\begin{matrix} (5.36) & η_{σ} = {\begin{cases} 1 & for bosons \\ sgn (σ) & for fermions \end{cases} \end{matrix}

The sum

\sum_{{{\vec{k}}_{1}, \dots, {\vec{k}}_{N}}}^{'}

is restricted in order to ensure that each identical particle state appears only once. We may equivalently work in terms of the occupation number representation

{| {n_{\vec{k}}} ⟩}_{\pm}

. It is then clear that

\begin{matrix} (5.37) & \sum_{{\vec{k}}}^{'} = \sum_{{\vec{k}}} \frac{\prod_{{\vec{k}}} n_{\vec{k}}!}{N!}, \end{matrix}

where the factor in the unrestricted sum compensates the over-counting. This gives with the formulas for

c_{η}

derived above

η ⟨ {{\vec{x}}^{'}} | ρ | {\vec{x}} ⟩_{η} = \sum_{{\vec{k}}} \frac{\prod_{\vec{k}} n_{\vec{k}}!}{N!} \frac{1}{\prod_{\vec{k}} n_{\vec{k}}! N!} \sum_{σ, σ^{'} \in S_{N}} \frac{η_{σ} η_{σ^{'}}}{Z_{N}} e^{- β \sum_{i} \frac{ℏ^{2} {\vec{k}}_{i}^{2}}{2 m}} Ψ_{σ^{'} {\vec{k}}}^{+} ({{\vec{x}}^{'}}) Ψ_{σ {\vec{k}}} ({\vec{x}})

.
It is now convenient to work with period boundary conditions instead of the Dirichlet boundary conditions used so far, i.e., we require that

Ψ (0, y, z) = Ψ (L, y, z)

, with

L

the length of the cube, and similarly for the

y

and

z

direction. The normalized eigenmodes are then

Ψ_{\vec{k}} = \frac{1}{\sqrt{V}} e^{i \vec{k} \cdot \vec{x}}

with

\vec{k} = \frac{2 π}{L} (n_{x}, n_{y}, n_{z})

, where

n_{x}, n_{y}, n_{z} \in Z

. Considering that the spacing between two wave vectors is

\frac{2 π}{L}

in every direction, we may replace the sum

\sum_{\vec{k}}

\frac{V}{(2 π)^{3}} \int d^{3} k

in the limit

V \to \infty

, which yields

\begin{aligned} {_{η} ⟨ {{\vec{x}}^{'}} | ρ | {\vec{x}} ⟩ ⟩}_{η} & = \frac{1}{Z_{N} N!^{2}} \frac{V^{N}}{(2 π)^{3 N}} \sum_{σ, σ^{'}} η_{σ} η_{σ^{'}} \int \frac{1}{V^{N}} d^{3 N} k e^{- β \sum_{i = 1}^{N} \frac{ℏ^{2} {\vec{k}}_{i}^{2}}{2 m}} e^{- i \sum_{j = 1}^{N} ({\vec{k}}_{σ j} {\vec{x}}_{j} - {\vec{k}}_{σ^{'} j} {\vec{x}}_{j}^{'})} \\ = \frac{1}{Z_{N} N!^{2}} \sum_{σ, σ^{'}} η_{σ} η_{σ^{'}} \prod_{j} \int \frac{d^{3} k}{(2 π)^{3}} e^{- i \vec{k} ({\vec{x}}_{σ j} - {\vec{x}}_{σ / j}^{'})} e^{- β \frac{{\vec{k}}^{2} ℏ^{2}}{2 m}} . \end{aligned}

The Gaussian integrals can be explicitly performed, giving the result

\frac{1}{λ^{3}} e^{- \frac{π}{λ^{2}} {({\vec{x}}_{σ j} - {\vec{x}}_{σ^{'} j}^{'})}^{2}}

with the thermal deBroglie wavelength

λ = \frac{h}{\sqrt{2 π m k_{B} T}}

. Relabeling the summation indices then results in

\begin{matrix} (5.39) & η ⟨ {{\vec{x}}^{'}} | ρ | {\vec{x}} ⟩_{η} = \frac{1}{Z_{N} λ^{3 N} N!} \sum_{σ} η_{σ} e^{- \frac{π}{λ^{2}} \sum_{j} {({\vec{x}}_{j} - {\vec{x}}_{σ j})}^{2}} \end{matrix}

Setting

{\vec{x}}^{'} = \vec{x}

, taking

\int d^{3 N} x

on both sides gives, and using

tr ρ \overset{!}{=} 1

gives:

\begin{matrix} (5.40) & Z_{N} = \frac{1}{N! λ^{3 N}} \int d^{3 N} x \sum_{σ \in S_{N}} η_{σ} e^{- \frac{π}{λ^{2}} \sum_{j} {({\vec{x}}_{j} - {\vec{x}}_{σ j})}^{2}} \end{matrix}

The terms with

σ \neq id

are suppressed for

λ \to 0

(i.e. for

h \to 0

T \to \infty

), so the leading order contribution comes from

σ = id

. The next-to leading order corrections come from those

σ

having precisely 1 transposition (there are

\frac{N (N - 1)}{2}

of them). A permutation with precisely one transposition corresponds to an exchange of two particles. Neglecting next-to-next-to leading order corrections, the canonical partition function is given by

\begin{aligned} Z_{N} = \frac{1}{N! λ^{3 N}} \int d^{3 N} x [1 + \frac{N}{2} (N - 1) η e^{- \frac{2 π}{λ^{2}} {({\vec{x}}_{1} - {\vec{x}}_{2})}^{2}} + \dots] \\ (5.41) & \int d^{3} x = V \frac{1}{N!} {(\frac{V}{λ^{3}})}^{N} [1 + \frac{N (N - 1)}{2 V} η \int d^{3} r e^{- \frac{2 π}{λ^{2}} {\vec{r}}^{2}} + \dots] \\ = \frac{1}{N!} {(\frac{V}{λ^{3}})}^{N} [1 + \frac{N (N - 1)}{2 V} η {(\frac{2 π λ^{2}}{4 π})}^{\frac{3}{2}} + \dots] . \end{aligned}

The free energy

F := - k_{B} T \log Z_{N}

is now calculated as

\begin{matrix} (5.42) & F = \underset{using N! \approx N^{N} e^{- N}}{\underset{⏟}{- N k_{B} T \log [\frac{e}{λ^{3}} \cdot \frac{V}{N}]}} - \underset{using \log (1 + ϵ) \approx ϵ}{\underset{⏟}{\frac{k_{B} T N^{2}}{2 V}}} \frac{λ^{3}}{2^{\frac{3}{2}}} η + \dots \end{matrix}

Together with the following relation for the pressure (cf. (4.17)),

\begin{matrix} (5.43) & P = - {\frac{\partial F}{\partial V} |}_{T} \end{matrix}

it follows that

\begin{matrix} (5.44) & P = n k_{B} T (1 - η n \frac{λ^{3}}{2^{\frac{5}{2}}} + \dots) \end{matrix}

where

n = \frac{N}{V}

is the particle density. Comparing to the classical ideal gas, where we had

P = n k_{B} T

, we see that when

n λ^{3}

is of order 1 , quantum effects significantly increase the pressure for fermions

(η = - 1)

while they decrease the pressure for bosons

(η = + 1)

. As we can see comparing the expression (5.41) with the leading order term in the cluster expansion of the classical gas (see chapter 4.6), this effect is also present for a classical gas to leading order if we include a 2-body potential

V (\vec{r})

, such that

\begin{matrix} (5.45) & e^{- β V (\vec{r})} - 1 = η e^{\frac{- 2 π {\vec{r}}^{2}}{λ^{2}}} (from (5.41)) \end{matrix}

It follows that for the potential

V (\vec{r})

it holds

\begin{matrix} (5.46) & V (\vec{r}) = - k_{B} T \log [1 + η e^{- \frac{2 π {\vec{r}}^{2}}{λ^{2}}}] \approx - k_{B} T η e^{- \frac{2 π {\vec{r}}^{2}}{λ^{2}}}, for r ≳ λ \end{matrix}

A sketch of

V (\vec{r})

is given in the following picture:

Figure 5.1: The potential

V (\vec{r})

ocurring in (5.46).

Thus, we can say that quantum effects lead to an effective potential. For fermions the resulting correction to the pressure

P

in (5.44) is called degeneracy pressure. Note that according to (5.44) the degeneracy pressure is proportional to

k_{B} T n^{2} λ^{3}

for fermions, which increases strongly for increasing density

n

. It provides a mechanism to support very dense objects against gravitational collapse, e.g. in neutron stars.

5.3 Spin Degeneracy

For particles with spin the energy levels have a corresponding

g

-fold degeneracy. Since different spin states have the same energy the Hamiltonian is now given by

\begin{matrix} (5.47) & H = \sum_{\vec{k}, s} ϵ (\vec{k}) a_{\vec{k}, s}^{†} a_{\vec{k}, s}, s = 1, \dots, g = 2 S + 1 \end{matrix}

where the creation/destruction operators

a_{\vec{k}, s}^{†}

and

a_{\vec{k}, s}

fulfill the commutation relations

\begin{matrix} (5.48) & [a_{\vec{k}, s}, a_{{\vec{k}}^{'}, s^{'}}^{†}] = δ_{\vec{k}, {\vec{k}}^{'}} δ_{s, s^{'}}, {a_{\vec{k}, s}, a_{{\vec{k}}^{'}, s^{'}}^{†}} = δ_{\vec{k}, {\vec{k}}^{'}} δ_{s, s^{'}} \end{matrix}

for bosons/fermions respectively. For the grand canonical ensemble the Hilbert space of particles with spin is given by

\begin{matrix} (5.49) & H^{\pm} = ⨁_{N ⩾ 0} H_{N}^{\pm}, H_{1} = L^{2} (V, d^{3} x) \otimes C^{g} \end{matrix}

It is easy to see that for the grand canonical ensemble this results in the following expressions for the expected number densities

{\bar{n}}_{\vec{k}}

and the mean energy

E_{\pm}

\begin{aligned} (5.50) & {\bar{n}}_{\vec{k}}^{\pm} = {⟨ {\hat{N}}_{\vec{k}} ⟩}_{\pm} = \frac{g}{e^{β (ϵ (\vec{k}) - μ)} \mp 1} \\ (5.51) & E = ⟨ H ⟩_{\pm} = g \sum_{\vec{k}} \frac{ϵ (\vec{k})}{e^{β (ϵ (\vec{k}) - μ)} \mp 1} \end{aligned}

In the canonical ensemble we find similar expressions. For a non-relativistic gas we get, with

\sum_{\vec{k}} \to V \int \frac{d^{3} k}{(2 π)^{3}}

for

V \to \infty

\begin{matrix} (5.52) & ϵ_{\pm} := \frac{E_{\pm}}{V} = g \int \frac{d^{3} k}{(2 π)^{3}} \frac{ℏ^{2} k^{2}}{2 m} \frac{1}{e^{β (\frac{ℏ^{2} k^{2}}{2 m} - μ)} \mp 1} . \end{matrix}

Setting

x = \frac{ℏ^{2} k^{2}}{2 m k_{B} T}

or equivalently

k = \frac{2 π^{\frac{1}{2}}}{λ} x^{\frac{1}{2}}

and defining the fugacity

z := e^{β μ}

, we find

\begin{matrix} (5.53) & \frac{ϵ^{\pm}}{k_{B} T} = \frac{g}{λ^{3}} \frac{2}{\sqrt{π}} \int_{0}^{\infty} \frac{d x x^{\frac{3}{2}}}{z^{- 1} e^{x} \mp 1} . \end{matrix}

or similarly

\begin{matrix} (5.54) & {\bar{n}}^{\pm} = \frac{⟨ \hat{N} ⟩_{\pm}}{V} = \frac{g}{λ^{3}} \frac{2}{\sqrt{π}} \int_{0}^{\infty} \frac{d x x^{\frac{1}{2}}}{z^{- 1} e^{x} \mp 1} . \end{matrix}

Furthermore, we also have the following relation for the pressure

P_{\pm}

and the grand canonical potential

G_{\pm} = - k_{B} T \log Y^{\pm} (cf

. section 4.4):

\begin{matrix} (5.55) & P_{\pm} = - {\frac{\partial G^{\pm}}{\partial V} |}_{T, μ} \end{matrix}

From (5.27), (5.28) it follows that in the case of spin degeneracy the grand canonical partition function

Y^{\pm}

is given by

\begin{matrix} (5.56) & Y^{\pm} = {[\prod_{\vec{k}} (1 \mp z e^{- β ϵ (\vec{k})})]}^{\mp g} \end{matrix}

Taking the logarithm on both sides and taking a large volume

V \to \infty

to approximate the sum by an integral as before yields

\begin{aligned} \frac{P_{\pm}}{k_{B} T} & = \mp g \int \frac{d^{3} k}{(2 π)^{3}} \log [1 \mp z e^{- \frac{ℏ^{2} k^{2}}{2 m k_{B} T}}] \\ (5.57) & = \frac{g}{λ^{3}} \frac{4}{3} {\sqrt{π}}^{- 1} \int_{0}^{\infty} \frac{d x x^{\frac{3}{2}}}{z^{- 1} e^{x} \mp 1} \end{aligned}

To go to the last line, we used a partial integration in

x

. For

z ≪ 1

, i.e.

μ β = \frac{μ}{k_{B} T} ≪ 0

one can expand

{\bar{n}}^{\pm}

z

around

z = 0

. Using the relation

\int \frac{d x x^{m - 1}}{z^{- 1} e^{x} - η} = η (m - 1)! \sum_{n = 1}^{\infty} \frac{(η z)^{n}}{n^{m}}

(which for

η z = 1

yields the Riemann

ζ

-function), one finds that

\begin{aligned} (5.58) & \frac{{\bar{n}}_{\pm} λ^{3}}{g} & = z \pm \frac{z^{2}}{2^{\frac{3}{2}}} + \frac{z^{3}}{3^{\frac{3}{2}}} \pm \frac{z^{4}}{4^{\frac{3}{2}}} + \dots \\ (5.59) & \frac{β P_{\pm} λ^{3}}{g} & = z \pm \frac{z^{2}}{2^{\frac{5}{2}}} + \frac{z^{3}}{3^{\frac{5}{2}}} \pm \frac{z^{4}}{4^{\frac{5}{2}}} + \dots \end{aligned}

Solving (5.58) for

z

and substituting into (5.59) gives

\begin{matrix} (5.60) & P_{\pm} = {\bar{n}}_{\pm} k_{B} T [1 \mp \frac{1}{2^{\frac{5}{2}}} (\frac{{\bar{n}}_{\pm} λ^{3}}{g}) + \dots] \end{matrix}

which for

g = 1

gives the same result for the degeneracy pressure we obtained previously in (5.44). Note again the "+" sign for fermions.

5.4 Black Body Radiation

We know that the dispersion relation for photons is given by (note that the momentum is

\vec{p} = ℏ \vec{k})

\begin{matrix} (5.61) & ϵ (\vec{k}) = ℏ c | \vec{k} | \end{matrix}

There are two possibilities for the helicity ("spin") of a photon which is either parallel or anti-parallel to

\vec{p}

, corresponding to the polarization of the light. Hence, the degeneracy factor for photons is

g = 2

and the Hamiltonian is given by

\begin{matrix} (5.62) & H = \sum_{\vec{p}, s = \pm 1} ϵ (\vec{p}) a_{\vec{p}, s}^{+} a_{\vec{p}, s} + \underset{interaction}{\underset{⏟}{\dots}} (\vec{p} \neq 0) \end{matrix}

Under normal circumstances there is practically no interaction beween the photons, so the interaction terms indicated by "..." can be neglected in the previous formula. The following picture is a sketch of a 4 -photon interaction, where

σ

denotes the cross section for the corresponding 2-2 scattering process obtained from the computational rules of quantum electrodynamics:

Figure 5.2: Lowest-order Feynman diagram for photon-photon scattering in Quantum Electrodynamics.

The mean collision time of the photons is given by

\begin{matrix} (5.63) & \frac{1}{τ} = \frac{c σ N}{V} = c σ n \approx 10^{- 44} \times n \frac{{cm}^{3}}{s} \end{matrix}

where

N = ⟨ \hat{N} ⟩

is the average number of photons inside

V

and

n = N / V

their density. Even in extreme places like the interior sun, where

T \approx 10^{7} K

, this leads to a mean collision time of

10^{18} s

. This is more than the age of the universe, which is approximately

10^{17}

s. From this we conclude that we can safely treat the photons as an ideal gas!

By the methods of the previous subsection we find for the grand canonical partition function, with

μ = 0

\begin{matrix} (5.64) & Y = tr (e^{- β H}) = {[\prod_{\vec{p} \neq 0} \frac{1}{1 - e^{- β ϵ (\vec{p})}}]}^{2} \end{matrix}

since the degeneracy factor is

g = 2

and photons are bosons. For the Gibbs free energy
(in the limit

V \to \infty

) we get

^{1}

\begin{aligned} G = - k_{B} T \log Y = \frac{2 V}{β} \int \frac{d^{3} p}{(2 π ℏ)^{3}} \log (1 - e^{- β c p}) = \frac{V {(k_{B} T)}^{4}}{π^{2} (ℏ c)^{3}} \int_{0}^{\infty} d x x^{2} \log (1 - e^{- x}) \\ = \frac{V {(k_{B} T)}^{4}}{π^{2} (ℏ c)^{3}} (- \frac{1}{3}) \int_{0}^{\infty} \frac{d x x^{3}}{e^{x} - 1} = - \frac{V {(k_{B} T)}^{4}}{(ℏ c)^{3}} \frac{π^{2}}{45} . \\ = - 2 ζ (4) = - \frac{π^{4}}{45} \\ (5.66) & \Rightarrow G = - \frac{4 σ}{3 c} V T^{4} . \end{aligned}

Here,

σ = 5.67 \times 10^{- 8} \frac{J}{s m^{2} K^{4}}

is the Stefan-Boltzmann constant.
The entropy was defined as

S := - k_{B} tr (ρ \log ρ)

with

ρ = \frac{1}{Y} e^{- β H}

. One easily finds the relation

\begin{matrix} (5.67) & S = - {\frac{\partial G}{\partial T} |}_{V, μ = 0} = {\frac{\partial}{\partial T} (k_{B} T \log Y) |}_{V, μ = 0} \end{matrix}

(see chapter 6.5 for a systematic review of such formulas) or

\begin{matrix} (5.68) & \Rightarrow S = \frac{16 σ}{3 c} V T^{3} \end{matrix}

The mean energy

E

is found as

\begin{matrix} E = ⟨ H ⟩ = 2 \sum_{\vec{p} \neq 0} ϵ (\vec{p}) \frac{1}{e^{β ϵ (\vec{p})} - 1} = 2 V \int \frac{d^{3} p}{(2 π ℏ)^{3}} \frac{| \vec{p} |}{e^{β c | \vec{p} |} - 1} . \\ (5.69) & \Rightarrow E = \frac{4 σ}{c} V T^{4} \end{matrix}

Finally, the pressure

P

can be calculated as

\begin{matrix} (5.70) & P = - {\frac{\partial G}{\partial V} |}_{T, μ = 0} = {\frac{\partial}{\partial V} (k_{B} T \log Y) |}_{T, μ = 0} \end{matrix}

see again chapter 6.5 for systematic review of such formulas. This gives

\begin{matrix} (5.71) & \Rightarrow P = \frac{4 σ}{3 c} T^{4} \end{matrix}

As an example, for the sun, with

T_{sun} = 10^{7} K

, the pressure is

P = 25, 000, 000 atm

and

\begin{matrix} (5.65) & ζ (s) = \sum_{n ⩾ 1} n^{- s}, for Re (s) > 1 \end{matrix}

f o r a H - b o m b, w i t h $ T_{bomb} = 10^{5} K $, t h e p r e s s u r e i s $ P = 0.25 atm $ . F r o m (5.69), (5.70) o n e o b t a i n s

\begin{matrix} (5.72) & P = \frac{1}{3} \frac{E}{V} \Leftrightarrow E = 3 P V \end{matrix}

This is also known as the Stefan-Boltzmann law.
Let

u (ν)

be the spectral energy density, i.e.,

u (ν) d ν

is the contribution to the total energy density due to radiation in the range of frequencies

[ν, ν + d ν]

. To derive an expression for this, we recall that the average number of photons with wave vector

\vec{k}

\begin{matrix} (5.73) & {\bar{n}}_{\vec{k}} = \frac{2}{e^{β c ℏ k} - 1} . \end{matrix}

Hence, taking the usual conversion from discrete to continuous wave vectors into account, the expected number of photons with wave vector in the range

[\vec{k}, \vec{k} + d \vec{k}]

\frac{2}{e^{β c h k} - 1} V \frac{d^{3} k}{(2 π)^{3}}

. For the expected number of photons with the modulus of the wave vector in the range

[k, k + d k]

, we thus get

\frac{V}{π^{2}} \frac{k^{2}}{e^{β c h k} - 1} d k

. The frequency of a wave with wave vector

\vec{k}

ν = \frac{c}{2 π} k

, so that the number of photons in the frequency range

[ν, ν + d ν]

\frac{8 π V}{c^{3}} \frac{ν^{2}}{e^{β h ν} - 1} d ν

. Multiplication with the energy

h ν

per photon and dividing by

V

, we obtain the Planck distribution

\begin{matrix} (5.74) & u (ν) = \frac{h}{π c^{3}} \frac{ν^{3}}{e^{\frac{h ν}{k_{B} T}} - 1} \end{matrix}

This is the famous law found by Planck in 1900 which lead to the development of quantum theory! The Planck distribution looks as follows:

Figure 5.3: Sketch of the Planck distribution for different temperatures.

This can be measured by drilling a hole in a cavity and measuring the spectral intensity of the outgoing radiation. An almost perfect black body spectrum is observed in the cosmic microwave background, at

T ≃ 2.7 K

Solving

u^{'} (ν_{max}) = 0

one finds that the maximum of

u (ν)

lies at

h ν_{max} \approx 2.82 k_{B} T

, a relation also known as Wien's law. The following limiting cases are noteworthy:
(i)

h ν ≪ k_{B} T

In this case we have

\begin{matrix} (5.75) & u (ν) \approx \frac{k_{B} T ν^{2}}{π c^{3}} \end{matrix}

This formula is valid in particular for

h \to 0

, i.e. it represents the classical limit. It was known before the Planck formula. It is not only inaccurate for larger frequencies but also fundamentally problematic since it suggests

⟨ H ⟩ = E \propto \int d ν u (ν) = \infty

, which indicates an instability not seen in reality.
(ii)

h ν ≫ k_{B} T

In this case we have

\begin{matrix} (5.76) & u (ν) \approx \frac{h ν^{3}}{π c^{3}} e^{\frac{- h ν}{k_{B} T}} \end{matrix}

This formula had been found empirically by Wien without proper interpretation of the constants (and in particular without identifying

h

We can also calculate the mean total particle number:

\begin{aligned} ⟨ \hat{N} ⟩ & = \sum_{\vec{p} \neq 0} \frac{2}{e^{β c | \vec{p} |} - 1} \approx 2 V \int \frac{d^{3} p}{(2 π ℏ)^{3}} \frac{1}{e^{β c | \vec{p} |} - 1} \\ (5.77) & = \frac{2 ζ (3)}{π^{2}} V {(\frac{k_{B} T}{ℏ c})}^{3} \end{aligned}

Combining this formula with that for the entropy

S

, eq. (5.68), gives the relation

\begin{matrix} (5.78) & S = \frac{8 π^{4}}{3 ζ (3)} k_{B} N \approx 3.6 N k_{B} \end{matrix}

where

N \equiv ⟨ \hat{N} ⟩

is the mean total particle number from above. Thus, for an ideal photon gas we have

S = O (1) k_{B} N

, i.e. each photon contributes one unit to

\frac{S}{k_{B}}

on average (see problem B. 18 for an application of this elementary relation).

5.5 Degenerate Bose Gas

Ideal quantum gases of bosonic particles show a particular behavior for low temperature

T

and large particle number densities

n = \frac{⟨ \hat{N} ⟩}{V}

. We first discuss the ideal Bose gas in a finite volume. In this case, the expected particle density was given by

\begin{matrix} (5.79) & n = \frac{⟨ \hat{N} ⟩}{V} = \frac{g}{V} \sum_{\vec{k}} \frac{1}{e^{β (ϵ (\vec{k}) - μ)} - 1} . \end{matrix}

The sum is calculated for sufficiently large volumes again by replacing

\sum_{\vec{k}}

V \int \frac{d^{3} k}{(2 π)^{3}}

, which yields

\begin{aligned} n & \approx g \int \frac{d^{3} k}{(2 π)^{3}} \frac{1}{e^{β (ϵ (\vec{k}) - μ)} - 1} \\ (5.80) & = \frac{g}{2 π^{2}} \int_{0}^{\infty} d k \frac{k^{2}}{e^{β (ϵ (\vec{k}) - μ)} - 1} \end{aligned}

The particle density is clearly maximal for

μ \to 0

and its maximal value is given by

n_{c}

where, with

ϵ (k) = \frac{ℏ^{2} k^{2}}{2 m}

\begin{aligned} n_{c} & = \frac{g}{2 π^{2}} \int_{0}^{\infty} d k \frac{k^{2}}{e^{β ϵ (\vec{k})} - 1} \\ = \frac{g}{2 π^{2}} {(\frac{2 m}{β ℏ^{2}})}^{\frac{3}{2}} \int_{0}^{\infty} \frac{d x x^{2}}{e^{x^{2}} - 1} \\ = \frac{g}{2 π^{2}} {(\frac{2 m}{β ℏ^{2}})}^{\frac{3}{2}} \sum_{n = 1}^{\infty} \int_{0}^{\infty} d x x^{2} e^{- n x^{2}} \\ = \frac{g}{λ^{3}} ζ (\frac{3}{2}) \end{aligned}

and where

λ = \sqrt{\frac{h^{2}}{2 π m k_{B} T}}

is the thermal deBroglie wavelength. From this wee see that

n ⩽ n_{c}

. For a given density

n

, the minimal temperate is (for

T < T_{c}

, we would have

n > n_{c}

)

\begin{matrix} (5.81) & T_{c} = \frac{h^{2}}{2 π m k_{B}} {(\frac{n}{g ζ (\frac{3}{2})})}^{\frac{2}{3}} \end{matrix}

Equilibrium states with higher densities

n > n_{c}

are not possible at finite volume. A new phenomenon happens, however, for infinite volume, i.e. in the thermodynamic limit,

V \to \infty

. Here, we must be careful because density matrices are only formal (e.g. the partition function

Y \to \infty

), so it is better to characterize equilibrium states by the so-called KMS condition (for Kubo-Martin-Schwinger) for equilibrium states. As we will see, new interesting equilibrium states that can be found in this way in the thermodynamic limit. They correspond to a Bose-condensate, or a gas in a superfluid state.

In the present context, the KMS condition for a Gibbs state

⟨ \dots ⟩

for the ideal Bose gas is simply

\begin{matrix} (5.82) & ⟨ a_{\vec{p}}^{†} a_{\vec{k}} ⟩ = e^{- β (ϵ (\vec{k}) - μ)} ⟨ a_{\vec{k}} a_{\vec{p}}^{†} ⟩, \end{matrix}

which was already derived earlier. We can put this relation in a more convenient form recalling that in the case of no spin

(g = 1)

we had the commutation relations

[a_{\vec{k}}, a_{\vec{p}}^{+}] =

δ_{\vec{k}, \vec{p}}

for the creation/destruction operators. From this it follows that

\begin{matrix} (5.83) & (1 - e^{- β (ϵ (\vec{k}) - μ)}) ⟨ a_{\vec{p}}^{†} a_{\vec{k}} ⟩ = e^{- β (ϵ (\vec{k}) - μ)} δ_{\vec{k}, \vec{p}} \end{matrix}

So far, we are still at finite volume

V

. In the thermodynamic limit (infinite volume),

V \to \infty

, we should make the replacements
finite volume:

\vec{k} \in {(\frac{π}{L} Z)}^{3}

and

{\begin{cases} a_{\vec{k}} \\ δ_{\vec{k}, \vec{p}} \end{cases} ⟶

infinite volume:

\vec{k} \in R^{3}

and

{\begin{cases} a (\vec{k}) \\ δ^{3} (\vec{k} - \vec{p}) \end{cases}

Thus, we expect that in the thermodynamic limit:

\begin{matrix} (5.84) & (1 - e^{- β (\frac{ℏ^{2} {\hat{k}}^{2}}{2 m} - μ)}) ⟨ a^{†} (\vec{p}) a (\vec{k}) ⟩ = e^{- β (\frac{{\vec{k}}^{2} ℏ^{2}}{2 m} - μ)} δ^{3} (\vec{p} - \vec{k}) . \end{matrix}

In that limit, the statistical operator

ρ

of the grand canonical ensemble does not make mathematical sense, because

e^{- β H + β μ \hat{N}}

does not have a finite trace (i.e.

Y = \infty

). Nevertheless, the KMS condition (5.84) still makes perfect sense. We view it as the appropriate substitute for the notion of Gibbs state in the thermodynamic limit. There, it can be possible to get new equilibrium states at given temperature

T

and chemical potential

μ

that are described by certain additional "order parameters", and that are impossible at finite volume. We think of these as describing different phases.

To see this concretely in the case of the ideal Bose gas, we must therefore ask: What are the solutions of the KMS-condition (5.84)? For

μ < 0

the unique solution is the usual Bose-Einstein distribution:

⟨ a^{†} (\vec{k}) a (\vec{p}) ⟩ = \frac{δ^{3} (\vec{p} - \vec{k})}{e^{β (\frac{ℏ^{2} k^{2}}{2 m} - μ)} - 1} .

The point is that for

μ = 0

other solutions are also possible, for instance
for some

n_{0} ⩾ 0

(this follows from

⟨ A^{+} A ⟩ ⩾ 0

for operators

A

in any state). The particle number density in the thermodynamic limit

(V \to \infty)

is best expressed in terms of the creation operators at sharp position

\vec{x}

\begin{matrix} (5.85) & a (\vec{p}) = \frac{1}{(2 π)^{\frac{3}{2}}} \int d^{3} x e^{- i \vec{p} \vec{x}} a (\vec{x}) . \end{matrix}

The particle number density at the point

\vec{x}

is then defined as

\hat{N} (\vec{x}) := a^{†} (\vec{x}) a (\vec{x})

and therefore we have, for

μ = 0

\begin{matrix} (5.86) & n = ⟨ \hat{N} (\vec{x}) ⟩ = \frac{1}{(2 π)^{3}} \int d^{3} p d^{3} k ⟨ a^{†} (\vec{p}) a (\vec{k}) ⟩ e^{- i (\vec{p} - \vec{k}) \vec{x}} = n_{c} + n_{0} \end{matrix}

Thus, in this equilibrium state we have a macroscopically large occupation number

n_{0}

of the zero mode causing a different particle density at

μ = 0

. The fraction of zero modes, that is, that of the modes in the "condensate", can be written using our definition of

T_{c}

\begin{matrix} (5.87) & n_{0} = n (1 - {(\frac{T}{T_{c}})}^{3 / 2}) \end{matrix}

for

T

below

T_{c}

, and

n_{0} = 0

above

T_{c}

. The formation of the condensate can thereby be seen as a phase transition at

T = T_{c}

We can also write down more general solutions to the KMS-condition, for example:

\begin{matrix} (5.88) & ⟨ a^{†} (\vec{x}) a (\vec{y}) ⟩ = \int \frac{d^{3} k}{(2 π)^{3}} \frac{e^{i \vec{k} (\vec{x} - \vec{y})}}{e^{β \frac{ℏ^{2} k^{2}}{2 m}} - 1} + \overset{―}{f (\vec{x})} f (\vec{y}) \end{matrix}

where

f

is any harmonic function, i.e. a function such that

{\vec{\nabla}}^{2} f = 0

. To understand the physical meaning of these states, we define the particle current operator

\vec{j} (\vec{x})

\begin{matrix} (5.89) & \vec{j} (\vec{x}) := \frac{- i}{2 m} (a^{†} (\vec{x}) \vec{\nabla} a (\vec{x}) - \vec{\nabla} a^{†} (\vec{x}) a (\vec{x})) . \end{matrix}

An example of a harmonic function is

f (\vec{x}) = 1 + i m \vec{v} \cdot \vec{x}

, and in this case one finds the expectation value

\begin{matrix} (5.90) & ⟨ \vec{j} (\vec{x}) ⟩ = \frac{- i}{2 m} (\overset{―}{f (\vec{x})} \vec{\nabla} f (\vec{x}) - f (\vec{x}) \vec{\nabla} \overset{―}{f (\vec{x})}) = \vec{v} \end{matrix}

This means that the condensate flows in the direction of

\vec{v}

without leaving equilibrium. Another solution is

f (\vec{x}) = f (x, y, z) = x + i y

. In this case one finds

⟨ \vec{j} (x, y, z) ⟩ = \frac{1}{m} (- y, x, 0)

describing a circular motion around the origin (vortex). The condensate can hence flow or form vortices without leaving equilibrium. This phenomenon goes under the name of superfluidity.

Chapter 6 The Laws of Thermodynamics

The laws of thermodynamics predate the ideas and techniques from statistical mechanics, and are, to some extent, simply consequences of more fundamental ideas derived in statistical mechanics. However, they are still in use today, mainly because:
(i) they are easy to remember.
(ii) they are to some extent universal and model-independent.
(iii) microscopic descriptions are sometimes not known (e.g. black hole thermodynamics) or are not well-developed (non-equilibrium situations).
(iv) they are useful!

The laws of thermodynamics are based on:
(o) The empirical evidence that systems approach a new thermal equilibrium state after being pushed out of equilibrium by an external influence.
(i) The empirical evidence that, for a very large class of macroscopic systems, equilibrium states can generally be characterized by very few parameters. These thermodynamic parameters, often called

X_{1}, \dots, X_{n}

in the following, can hence be viewed as "coordinates" on the space of equilibrium systems.
(ii) The idea to perform mechanical work on a system, or to bring equilibrium systems into "thermal contact" with reservoirs in order to produce new equilibrium states in a controlled way. The key idea here is that these changes (e.g. by "heating up a system" through contact with a reservoir system) should be extremely gentle so that the system is not pushed out of equilibrium too much. One thereby imagines that one can describe such a gradual change of the system by a succession of equilibrium states, i.e. a curve in the space of coordinates

X_{1}, \dots, X_{n}

characterizing the different equilibrium states. This idealized notion of an infinitely gentle/slow change is often referred to as "quasi-static".
(iii) Given the notions of quasi-static changes in the space of equilibrium states, one can then postulate certain rules guided by empirical evidence that tell us which kind of changes should be possible, and which ones should not. These are, in essence, the laws of thermodynamics. For example, one knows that if one has access to equilibrium systems at different temperature, then one system can perform work on the other system. The first and second law state more precise conditions about such processes and imply, respectively, the existence of an energy- and entropy function on equilibrium states. The zeroth law just states that being in thermal equilibrium with each other is an equivalence relation for systems, i.e. in particular transitive. It implies the existence of a temperature function labelling the different equivalence classes.

6.1 The Zeroth Law

0^{th}

law of thermodynamics: If two subsystems I,II are separately in thermal contact with a third system, III, then they are in thermal equilibrium with each other.

The

0^{th}

law implies the existence of a function

Θ : {equilibrium systems} \to R

such that

Θ

is equal for systems in thermal equilibrium with each other. To see this, let us imagine that the equilibrium states of the systems I,II and III are parametrized by some coordinates

{A_{1}, A_{2}, \dots}, {B_{1}, B_{2}, \dots}

and

{C_{1}, C_{2}, \dots}

. Since a change in I implies a corresponding change in III, there must be a constraint

^{1}

\begin{matrix} (6.1) & f_{I, III} ({A_{1}, A_{2}, \dots; C_{1}, C_{2}, \dots}) = 0 \end{matrix}

and a similar constraint

\begin{matrix} (6.2) & f_{II, III} ({B_{1}, B_{2}, \dots; C_{1}, C_{2}, \dots}) = 0 \end{matrix}

which we can write as

\begin{matrix} (6.3) & C_{1} = {\tilde{f}}_{I, IIII} ({A_{1}, A_{2}, \dots; C_{2}, C_{3}, \dots}) = {\tilde{f}}_{II, IIII} ({B_{1}, B_{2}, \dots; C_{2}, C_{3}, \dots}) \end{matrix}

Since, according to the

0^{th}

law, we also must have the constraint

\begin{matrix} (6.4) & f_{I, II} ({A_{1}, A_{2}, \dots, B_{1}, B_{2}, \dots}) = 0 \end{matrix}

we can proceed by noting that for

{A_{1}, A_{2}, \dots, B_{1}, B_{2}, \dots}

which satisfy the last equation, (6.3) must be satisfied for any

{C_{2}, C_{3}, \dots}

! Thus, we let III be our reference system and set

{C_{2}, C_{3}, \dots}

to any convenient but fixed value. This reduces the condition (6.4) for equilibrium between I and II to:
$$

\begin{aligned} Θ ({A_{1}, A_{2}, \dots}) : & = {\tilde{f}}_{I, III} ({A_{1}, A_{2}, \dots, C_{2}, C_{3}, \dots}) \\ (6.5) & = {\tilde{f}}_{II, III} ({B_{1}, B_{2}, \dots, C_{2}, C_{3}, \dots}) = Θ ({B_{1}, B_{2}, \dots}) . \end{aligned}

This means that equilibrium is characterized by some function

Θ

of thermodynamic coordinates, which has the properties of a temperature.

We may choose as our reference system III an ideal gas, with

\begin{matrix} (6.6) & \frac{P V}{N k_{B}} = const. = T [K] =: Θ \end{matrix}

By bringing this system (for

V \to \infty

) in contact with any other system, we can measure the (absolute) temperature of the latter. For example, one can define the triple point of the system water-ice-vapor to be at 273.16 K . Together with the definition of

k_{B} = 1.4 \times 10^{- 23} \frac{J}{K}

) this then defines, in principle, the Kelvin temperature scale. Of course in practice the situation is more complicated because ideal gases do not exist.

Figure 6.1: The triple point of ice water and vapor in the

(P, T)

phase diagram

The Zeroth Law implies in particular: The temperature of a system in equilibrium is constant throughout the system. This has to be the case since subsystems obtained by imaginary walls are in equilibrium with each other, see the following figure:

Figure 6.2: A large system divided into subsystems I and II by an imaginary wall.

6.2 The First Law

1^{st}

law of thermodynamics: The amount of work required to change adiabatically a thermally isolated system from an initial state

i

to a final state

f

depends only on

i

and

f

, not on the path of the process.

Figure 6.3: Change of system from initial state

i

to final state

f

along two different paths.

Here, by an 'adiabatic change", one means a change without heat exchange. Consider a particle moving in a potential. By fixing an arbitrary reference point

X_{0}

, we can define an energy landscape

\begin{matrix} (6.7) & E (X) = \int_{X_{0}}^{X} δ W \end{matrix}

where the integral is along any path connecting

X_{0}

with

X

, and where

X_{0}

is a reference point corresponding to the zero of energy.

δ W

is the infinitesimal change of work done along the path. In order to define more properly the notion of such integrals of "infinitesimals", we will now make a short mathematical digression on differential forms.

Differentials ("differential forms")

A 1-form (or differential) is an expression of the form

\begin{matrix} (6.8) & α = \sum_{i = 1}^{N} α_{i} (X_{1}, \dots, X_{N}) d X_{i} \end{matrix}

We define

\begin{matrix} (6.9) & \int_{γ} α := \int_{0}^{1} \sum_{i = 1}^{N} α_{i} (X_{1} (t), \dots, X_{N} (t)) \underset{" = d X_{i} "}{\underset{⏟}{\frac{d X_{i} (t)}{d t} d t}} \end{matrix}

which in general is

γ

-dependent. Given a function

f (X_{1}, \dots, X_{N})

R^{N}

, we write

\begin{matrix} (6.10) & d f (X_{1}, \dots, X_{N}) = \frac{\partial f}{\partial X_{1}} (X_{1}, \dots, X_{N}) d X_{1} + \dots + \frac{\partial f}{\partial X_{N}} (X_{1}, \dots, X_{N}) d X_{N} \end{matrix}

d f

is called an "exact" 1-form. From the definition of the path integral along

γ

it is obvious that

\begin{matrix} (6.11) & \int_{γ} d f = \int_{0}^{1} \frac{d}{d t} {f (X_{1} (t), \dots, X_{N} (t))} d t = f (γ (1)) - f (γ (0)), \end{matrix}

so the integral of an exact 1-form only depends on the beginning and endpoint of the path. An example of a curve

γ : [0, 1] \to R^{2}

is given in the following figure:

Figure 6.4: A curve

γ : [0, 1] \to R^{2}

The converse is also true: The integral is independent of the path

γ

if and only if there
exists a function

f

R^{N}

, such that

d f = α

, or equivalently, if and only if

α_{i} = \frac{\partial f}{\partial X_{i}}

.
The notion of a

p

-form generalizes that of a 1 -form. It is an expression of the form

\begin{matrix} (6.12) & α = \sum_{i_{1}, \dots, i_{p}} α_{i_{1} \dots i_{p}} d X_{i_{1}} \dots d X_{i_{p}} \end{matrix}

where

α_{i_{1} \dots i_{p}}

are (smooth) functions of the coordinates

X_{i}

. We declare the

d X_{i}

to anticommute,

\begin{matrix} (6.13) & d X_{i} d X_{j} = - d X_{j} d X_{i} . \end{matrix}

Then we may think of the coefficient tensors as totally anti-symmetric, i.e. we can assume without loss of generality that

\begin{matrix} (6.14) & α_{i_{σ (1) \dots} \dots i_{σ (p)}} = sgn (σ) α_{i_{1} \dots i_{p}}, \end{matrix}

where

σ

is any permutation of

p

elements and

s g n

is its signum (see the discussion of fermions in the chapter on the ideal quantum gas). We may now introduce an operator d with the following properties:
(i)

d (f g) = d f g + (- 1)^{p} f d g

f

is a

p

form and

g

q

form,
(ii)

d (λ f + η g) = λ d f + η d g

f, g

are

p

forms and

λ, η

are constants.
(iii)

d f = \sum_{i} \frac{\partial f}{\partial X_{i}} d X_{i}

for 0 -forms

f

,
(iv)

d^{2} X_{i} = 0

On scalars (i.e. 0 -forms) the operator d is defined as before, and the rules (i)-(iv) then determine it for any

p

-form. The relation (6.13) can be interpreted as saying that we should think of the differentials

d X_{i}, i = 1, \dots, N

as "fermionic-" or "anti-commuting variables".

^{2}

For instance, we then get for a 1-form

α

\begin{aligned} (6.15) & d α & = \sum_{i, j} \frac{\partial α_{i}}{\partial X_{j}} \underset{= - d X_{i} d X_{j}}{\underset{⏟}{d X_{j} d X_{i}}} \\ (6.16) & = \frac{1}{2} \sum_{i, j} (\frac{\partial α_{i}}{\partial X_{j}} - \frac{\partial α_{j}}{\partial X_{i}}) d X_{j} d X_{i} . \end{aligned}

The expression for

d α

of a

p

-form follows similarly by applying the rules (i)-(iv). The rules imply the most important relation for

p

forms,

\begin{matrix} (6.17) & d^{2} α = d (d α) = 0 \end{matrix}

Conversely, it can be shown that for any

p + 1

form

f

R^{N}

such that

d f = 0

we must have

f = d α

for some

p

-form

α

. This result is often referred to as the

P o i n c a r e

lemma. An important and familiar example for this from field theory is provided by force fields

\vec{f}

R^{3}

. The components

f_{i}

of the force field may be identified with the components of a 1-form called

F = \sum f_{i} d X_{i}

. The condition

d F = 0

is seen to be equivalent to

\vec{\nabla} \times \vec{f} = 0

, i.e. we have a conservative force field. Poincaré's lemma implies the existence of a potenial

- W

, such that

F = - d W

; in vector notation,

\vec{f} = - \vec{\nabla} W

. A similar statement is shown to hold for

p

-forms

Just as a 1-form can be integrated over oriented curves (1-dimensional surfaces), a

p

form can be integrated over an oriented

p

-dimensional surface

Σ

. If that surface is parameterized by

N

functions

X_{i} (t_{1}, \dots, t_{p})

p

parameters

(t_{1}, \dots, t_{p}) \in U \subset R^{p}

(the ordering of which defines an orientation of the surface), we define the corresponding integral as

\begin{matrix} (6.18) & \int_{Σ} α = \int_{U} d t_{1} \dots d t_{p} \sum_{i_{1}, \dots, i_{p}} α_{i_{1} \dots i_{p}} (X (t_{1}, \dots, t_{p})) \frac{\partial X_{i_{1}}}{\partial t_{1}} \dots \frac{\partial X_{i_{p}}}{\partial t_{p}} \end{matrix}

The value of this integral is independent of the chosen parameterization up to a sign which corresponds to our choice of orientation. The most important fact pertaining to integrals of differential forms is Gauss' theorem (also called Stokes' theorem in this context):

\begin{matrix} (6.19) & \int_{Σ} d α = \int_{\partial Σ} α \end{matrix}

In particular, the integral of a form

d α

vanishes if the boundary

\partial Σ

Σ

is empty.

Using the language of differentials, the

1^{st}

law of thermodynamics may also be stated as saying that, in the absence of heat exchange, the infinitesimal work is an exact 1-form,

\begin{matrix} (6.20) & d E = δ W \end{matrix}

or alternatively,

\begin{matrix} (6.21) & d δ W = 0 \end{matrix}

We can break up the infinitesimal work change into the various forms of possible work such as in

\begin{matrix} (6.22) & d E = δ W = \sum_{i} \underset{force}{\underset{⏟}{J_{i}}} \underset{displacement}{\underset{⏟}{d X_{i}}} = - P d V + μ d N + {other types of work, see table}, \end{matrix}

if the change of state is adiabatic (no heat transfer!). If there is heat transfer, then the

1^{st}

law gets replaced by

\begin{matrix} (6.23) & d E = δ Q + \sum_{i} \underset{force}{\underset{⏟}{J_{i}}} \underset{displacement}{\underset{⏟}{d X_{i}}} . \end{matrix}

This relation is best viewed as the definition of the infinitesimal heat change

δ Q

. Thus, we could say that the first law is just energy conservation, where energy can consist of either mechanical work or heat. We may then write

\begin{matrix} (6.24) & δ Q = d E - \sum_{i} \underset{force}{\underset{⏟}{J_{i}}} \underset{displacement}{\underset{⏟}{d X_{i}}} \end{matrix}

from which it can be seen that

δ Q

is a 1-form depending on the variables

(E, X_{1}, \dots, X_{n})

.
An overview over several thermodynamic forces and displacements is given in the following table:

System	Force $J_{i}$	Displacement $X_{i}$
wire	tension $F$	length $L$
film	surface tension $τ$	area $A$
fluid/gas	pressure $P$	volume $V$
magnet	magnetic field $\vec{B}$	magnetization $\vec{M}$
electricity	electric field $\vec{E}$	polarization $\vec{Π}$
	stat. potential $ϕ$	charge $q$
chemical	chemical potential $μ$	particle number $N$

Table 6.1: Some thermodynamic forces and displacements for various types of systems.

Since

δ Q

is not an exact differential (in particular

d δ Q \neq 0

) we have

Δ Q_{1} = \int_{γ_{1}} δ Q \neq \int_{γ_{2}} δ Q = Δ Q_{2}

So, there does not exist a function

Q = Q (V, A, N, \dots)

such that

δ Q = d Q

! Traditionally, one refers to processes where

δ Q \neq 0

as "non-adiabatic", i.e. heat is transferred.

6.3 The Second Law

2^{nd}

law of thermodynamics (Kelvin): There are no processes in which heat goes over from a reservoir, is completely converted to other forms of energy, and nothing else happens.

One important consequence of the

2^{nd}

law is the existence of a state function

S

, called entropy. As before, we denote the

n

"displacement variables" generically by

X_{i} \in

{V, N, \dots}

and the "forces" by

J_{i} \in {- P, μ, \dots}

, and consider equilibrium states labeled by

(E, {X_{i}})

in an

n + 1

-dimensional space. We consider within this space the "adiabatic" submanifold

A

of all states that can be reached from a given state

(E^{*}, {X_{i}^{*}})

by means of a reversible and quasi-static (i.e. sufficiently slowly performed) process. On this submanifold we must have

\begin{matrix} (6.25) & δ Q = d E - \sum_{i = 1}^{n} J_{i} d X_{i} = 0 \end{matrix}

i.e.

\int_{γ} δ Q = 0

for any closed curve

γ

A

. Otherwise there would exist processes disturbing the energy balance (through the exchange of heat), and we could then choose a sign of

δ Q

such that work is performed on a system by converting heat energy into work, which is impossible by the

2^{nd}

law.

We choose a (not uniquely defined) function

S

labeling different submanifolds

A

Figure 6.5: Sketch of the submanifolds

A

This means that

d S

is proportional to

d E - \sum_{i = 1}^{n} J_{i} d X_{i}

. Thus, at each point

(E, {X_{i}})

there is a function

Θ (E, X_{1}, \dots, X_{n})

such that

\begin{matrix} (6.26) & Θ d S = d E - \sum_{i = 1}^{n} J_{i} d X_{i} \end{matrix}

Θ

can be identified with the temperature

T [K]

for suitable choice of

S = S (E, X_{1}, \dots, X_{n})

,
which then uniquely defines

S

. This is seen for instance by comparing the coefficients in

\begin{matrix} (6.27) & T d S = T (\frac{\partial S}{\partial E} d E + \sum_{i = 1}^{n} \frac{\partial S}{\partial X_{i}} d X_{i}) = d E - \sum_{i = 1}^{n} J_{i} d X_{i} \end{matrix}

which yields

\begin{matrix} (6.28) & 0 = \underset{= 0}{\underset{⏟}{(T \frac{\partial S}{\partial E} - 1)}} d E + \sum_{i = 1}^{n} \underset{= 0}{\underset{⏟}{(T \frac{\partial S}{\partial X_{i}} + J_{i})}} d X_{i} \end{matrix}

Therefore, the following relations hold:

\begin{matrix} (6.29) & \frac{1}{T} = \frac{\partial S (E, {X_{j}})}{\partial E} and - \frac{J_{i}}{T} = \frac{\partial S (E, {X_{j}})}{\partial X_{i}} \end{matrix}

We recognize the first of those relations as the defining relation for temperature, whereas the second includes the definitions of the pressure and chemical potential. These relations were already stated in the microcanocical ensemble (cf. section 4.2.1.). We can now rewrite (6.26) as

\begin{matrix} (6.30) & d E = T d S + \sum_{i = 1}^{n} J_{i} d X_{i} = T d S - P d V + μ d N + \dots \end{matrix}

By comparing this formula with that for energy conservation for a process without heat transfer, we identify

\begin{matrix} (6.31) & δ Q = heat transfer = T d S \Rightarrow d S = \frac{δ Q}{T} (noting that d (δ Q) \neq 0!). \end{matrix}

Equation (6.30), which was derived for quasi-static processes, is the most important equation in thermodynamics.

Example: As an illustration, we calculate the adiabatic curves

A

for an ideal gas. The defining relation is, with

n = 1

and

X_{1} = V

in this case

0 = d E + P d V

Since

P V = N k_{B} T

and

E = \frac{3}{2} N k_{B} T

for the ideal gas, we find

\begin{matrix} (6.32) & P = P (E, V) = \frac{2}{3} \frac{E}{V} \end{matrix}

and therefore

\begin{matrix} (6.33) & 0 = d E + \frac{2}{3} \frac{E}{V} d V \end{matrix}

Thus, we can parametrize the adiabatic

A

E = E (V)

, such that

d E = \frac{\partial E (V)}{\partial V} d V

A

. We then obtain

\begin{aligned} 0 = \underset{= 0}{\underset{⏟}{(\frac{\partial E}{\partial V} + \frac{2}{3} \frac{E}{V})}} d V \\ \Rightarrow & E (V) = E^{*} {(\frac{V^{*}}{V})}^{2 / 3} \end{aligned}

Figure 6.6: Adiabatics of the ideal gas

Of course, we may also switch to other thermodynamic variables, like

(S, V)

, such that

E

now becomes a function of

(S, V)

\begin{matrix} (6.34) & d E = T d S - P d V = (\frac{\partial E}{\partial V}) d V + (\frac{\partial E}{\partial S}) d S \end{matrix}

The defining relation for the adiabatics then reads

\begin{matrix} (6.35) & 0 = \underset{= 0}{\underset{⏟}{(\frac{\partial E}{\partial V} + P)}} d V + \underset{= 0}{\underset{⏟}{(\frac{\partial E}{\partial S} - T)}} d S \end{matrix}

from which it follows that

\begin{matrix} (6.36) & T = {\frac{\partial E}{\partial S} |}_{V} and P = - {\frac{\partial E}{\partial V} |}_{S} \end{matrix}

which hold generally (cf. section 4.2 .1 , eq. (4.17)). For an ideal gas

(P V = N k_{B} T

and

E = \frac{3}{2} N k_{B} T

) we thus find

\begin{aligned} - \frac{\partial E}{\partial V} V & = k_{B} N \frac{\partial E}{\partial S} \\ E & = \frac{3}{2} k_{B} N \frac{\partial E}{\partial S} \end{aligned}

which we can solve as

\begin{matrix} (6.37) & E (S, V) = E (S^{*}, V) e^{\frac{2}{3} \frac{S - S^{*}}{k_{B} N}} \end{matrix}

Since we also have

\begin{matrix} (6.38) & \frac{1}{E} \frac{\partial E}{\partial V} = - \frac{2}{3} \frac{1}{V} \end{matrix}

we find for the function

E (S, V)

\begin{matrix} (6.39) & E (S, V) = E (S^{*}, V^{*}) {(\frac{V^{*}}{V})}^{\frac{2}{3}} e^{\frac{2}{3} \frac{S - S^{*}}{k_{B} N}} \end{matrix}

Solving this relation for

S

, we obtain the relation

\begin{matrix} (6.40) & S = S^{*} + N k_{B} \log [{(\frac{E}{E^{*}})}^{\frac{3}{2}} \frac{V}{V^{*}}] \end{matrix}

The dependence on

V, E

of this expression coincides with the expression (4.16), found in section 4.2.1. Indeed, we find

\begin{matrix} (6.41) & S = N k_{B} \log [\frac{V}{N} {(\frac{4 π e m}{3} \frac{E}{N})}^{\frac{3}{2}}] \end{matrix}

i.e. the formula found before in the micro canonical ensemble for a suitable choice of the entropy

S^{*} = S (E^{*}, V^{*})

at the reference point.

^{3}

The entropy at the reference point depends on

N

and on the microscopic parameter of the system (i.e. the particle mass

m

), which clearly cannot be determined in the present context, since this information is neither contained in the equations of state, nor, of course, in the first law of thermodynamics.

6.4 Cyclic processes

6.4.1 The Carnot Engine

We next discuss the Carnot engine for an ideal (mono atomic) gas. As discussed in section 4.2., the ideal gas is characterized by the relations:

\begin{matrix} (6.42) & E = \frac{3}{2} N k_{B} T = \frac{3}{2} P V . \end{matrix}

We consider the cyclic process consisting of the following steps:

I \to II :

isothermal expansion at

T = T_{H}

\to

III: adiabatic expansion

(δ Q = 0)

,
III

\to

IV: isothermal compression at

T = T_{C}

,
IV

\to

I: adiabatic compression
where we assume

T_{H} > T_{C}

.
We want to work out the efficiency

η

, which is defined as

\begin{matrix} (6.43) & η := \frac{Δ W}{Δ Q_{in}} \end{matrix}

where

Δ Q_{in} = \int_{I}^{I I} δ Q

is the total heat added to the system (analogously,

Δ Q_{out} = \int_{I I I}^{I V} δ Q

is the total heat given off by system into a colder reservoir), and where

Δ W = ∮ δ W = (\int_{I}^{I I} + \int_{I I}^{I I I} + \int_{I I I}^{I V} + \int_{I V}^{I}) δ W

is the total work done by the system. We may also write

δ Q = T d S

and

δ W = P d V

(or more generally

δ W = - \sum_{i = 1}^{n} J_{i} d X_{i}

if other types of mechanical/ chemical work are performed by the system). By definition no heat exchange takes place during II

\to

III and IV

\to

We now wish to calculate

η_{Carnot}

. We can for instance take

P

and

V

as the variables to describe the process. We have

P V =

const. for isothermal processes by (6.42). To calculate the adiabatics, we could use the results from above and change the variables from

(E, V) \to (P, V)

using (6.42), but it is just as easy to do this from scratch: We start with

δ Q = 0

for an adiabatic process. From this follows that

\begin{matrix} (6.44) & 0 = d E + P d V \end{matrix}

Since on adiabatics we may take

P = P (V)

, this yields

\begin{matrix} (6.45) & d E = \frac{3}{2} d (P V) = \frac{3}{2} (V \frac{\partial P}{\partial V} + P) d V \end{matrix}

and therefore

\begin{matrix} (6.46) & 0 = \frac{3}{2} d (P V) + P d V = \underset{= 0}{\underset{⏟}{(\frac{3}{2} V \frac{\partial P}{\partial V} + \frac{5}{2} P)}} d V \end{matrix}

This yields the following relation:

\begin{matrix} (6.47) & V \frac{\partial P}{\partial V} = - \frac{5}{3} P, \Rightarrow P V^{γ} = const., γ = \frac{5}{3} \end{matrix}

So in a

(P, V)

-diagram the Carnot process looks as follows:

Figure 6.7: Carnot cycle for an ideal gas. The solid lines indicate isotherms and the dashed lines indicate adiabatics.

From

E = \frac{3}{2} P V

, which gives

d E = 0

on isotherms, it follows that the total heat added to the system is given by

\begin{aligned} Δ Q_{in} = \int_{I}^{I I} \underset{\begin{array}{c} 1^{st} law \\ δ Q = d E + P d V \end{array}}{\underset{⏟}{(d E + P d V)}} = \int_{I}^{I I} \underset{\begin{array}{c} from d E = 0 \\ on isotherms \end{array}}{\underset{⏟}{P d V}} \\ = \underset{P V = N k_{B} T_{H} on isotherm}{\underset{⏟}{N k_{B} T_{H} \int_{I}^{I I} V^{- 1} d V}} \\ (6.48) & = N k_{B} T_{H} \log \frac{V_{I I}}{V_{I}} . \end{aligned}

Using this result together with

P d V = - d E

on adiabatics we find for the total mechanical work done by the system:

\begin{aligned} Δ W & = \int_{I}^{I I} P d V + \int_{I I}^{I I I} P d V + \int_{I I I}^{I V} P d V + \int_{I V}^{I} P d V \\ = N k_{B} T_{H} \log \frac{V_{I I}}{V_{I}} - \int_{I I}^{I I I} d E - N k_{B} T_{C} \log \frac{V_{I I I}}{V_{I V}} - \int_{I V}^{I} d E \\ = E_{I I} - E_{I I I} + E_{I V} - E_{I} + N k_{B} (T_{H} \log \frac{V_{I I}}{V_{I}} - T_{C} \log \frac{V_{I I I}}{V_{I V}}) \end{aligned}

By conservation of energy,

∮ d E = 0

, we get

\begin{aligned} E_{I I} - E_{I I I} + E_{I V} - E_{I} & = E_{I I} - E_{I} + E_{I V} - E_{I I I} \\ = \int_{I}^{I I} d E + \int_{I I I}^{I V} d E = 0 \end{aligned}

since

d E = d (\frac{3}{2} N k_{B} T) = 0

on isotherms. From this it follows that

\begin{matrix} (6.49) & Δ W = N k_{B} (T_{H} \log \frac{V_{I I}}{V_{I}} - T_{C} \log \frac{V_{I I I}}{V_{I V}}) \end{matrix}

We can now use (6.48) and (6.49) to find

\begin{matrix} (6.50) & η_{Carnot} = \frac{Δ W}{Δ Q_{in}} = 1 - \frac{T_{C}}{T_{H}} \frac{\log V_{I I I} / V_{I V}}{\log V_{I I} / V_{I}} \end{matrix}

The relation (6.47) for the adiabatics, together with the ideal gas condition (6.42) implies

\begin{aligned} P_{I I} V_{I I}^{γ} = P_{I I I} V_{I I I}^{γ} & \Rightarrow T_{H} V_{I I}^{γ - 1} = T_{C} V_{I I I}^{γ - 1} \\ P_{I} V_{I}^{γ} = P_{I V} V_{I V}^{γ} & \Rightarrow T_{H} V_{I}^{γ - 1} = T_{C} V_{I V}^{γ - 1}, \\ \Rightarrow \frac{V_{I I}}{V_{I}} = \frac{V_{I I I}}{V_{I V}} . \end{aligned}

We thus find for the efficiency of the Carnot cycle

\begin{matrix} (6.51) & η = 1 - \frac{T_{C}}{T_{H}} \end{matrix}

This fundamental relation for the efficiency of a Carnot cycle can be derived also using the variables

(T, S)

instead of

(P, V)

, which also reveals the distinguished role played by this process. As

d T = 0

for isotherms and

d S = 0

for adiabatic processes, the Carnot cycle is just a rectangle in the

T

S

-diagram:

Figure 6.8: The Carnot cycle in the

(T, S)

-diagram.

We evidently have for the total heat added to the system:

\begin{matrix} (6.52) & Δ Q_{in} = \int_{I}^{I I} δ Q = \int_{I}^{I I} T d S = T_{H} (S_{I I} - S_{I}) \end{matrix}

To compute

Δ W

, the total mechanical work done by the system, we observe that (as

∮ d E = 0)

\begin{aligned} Δ W & = ∮ δ W = ∮ P d V \\ = ∮ (P d V + d E) \\ = ∮ T d S . \end{aligned}

A

is the domain enclosed by the rectangular curve describing the process in the

T - S

diagram, Gauss' theorem gives

\begin{aligned} Δ W & = ∮ T d S = \int_{A} d (T d S) = \int_{A} d T d S \\ = (T_{H} - T_{C}) (S_{I I} - S_{I}) \end{aligned}

from which it immediately follows that the efficiency

η_{Carnot}

is given by

\begin{matrix} (6.53) & η_{Carnot} = \frac{Δ W}{Δ Q_{in}} = \frac{(T_{H} - T_{C}) Δ S}{T_{H} Δ S} = 1 - \frac{T_{C}}{T_{H}} < 1 \end{matrix}

as before. Since

T_{H} > T_{C}

, the efficiency can never be

100 %

6.4.2 General Cyclic Processes

Consider now the more general cycle given by the curve

C

in the

(T, S)

-diagram depicted in the figure below:

Figure 6.9: A generic cyclic process in the

(T, S)

-diagram.

We define

C_{\pm}

to be the part of of the boundary curve

C

where heat is injected resp. given off. Then we have

d S > 0

C_{+}

and

d S < 0

C_{-}

. For such a process, we define the efficiency

η = η (C)

as before by the ratio of net work

Δ W

and injected heat

Δ Q_{in}

\begin{matrix} (6.54) & η = \frac{Δ W}{Δ Q_{in}} \end{matrix}

The quantities

Δ W

and

Δ Q_{in}

are then calculated as

\begin{aligned} Δ W & = - ∮_{C} δ W = ∮_{C} (T d S - d E) = ∮_{C} T d S \\ Δ Q_{in} & = \int_{C_{+}} T d S \end{aligned}

from which it follows that the efficiency

η = η (C)

is given by

\begin{matrix} (6.55) & η = \frac{∮_{C} T d S}{\int_{C_{+}} T d S} = 1 + \frac{\int_{C_{-}} T d S}{\int_{C_{+}} T d S} = 1 - \frac{Δ Q_{out}}{Δ Q_{in}} \end{matrix}

Now, if the curve

C

is completely contained between two isotherms at temperatures

T_{H} > T_{C}

, as in the above figure, then

\begin{aligned} 0 ⩽ & \int_{C_{+}} T d S ⩽ T_{H} \int_{C_{+}} d S (as d S < 0 on C_{+}), \\ \int_{C_{-}} T d S ⩽ T_{C} \int_{C_{-}} d S ⩽ 0 (as d S ⩽ 0 on C_{-}) . \end{aligned}

The efficiency

η_{C}

of our general cycle

C

can now be estimated as

\begin{matrix} (6.56) & η_{C} = 1 + \frac{\int_{C_{-}} T d S}{\int_{C_{+}} T d S} ⩽ 1 + \frac{T_{C} \int_{C_{-}} d S}{T_{H} \int_{C_{+}} d S} = 1 - \frac{T_{C}}{T_{H}} = η_{Carnot} \end{matrix}

where we used the above inequalities as well as

0 = ∮ d S = \int_{C_{+}} d S + \int_{C_{-}} d S

. Thus, we conclude that an arbitrary process is always less efficient than the Carnot process. This is why the Carnot process plays a distinguished role.

We can get a more intuitive understanding of this important finding by considering the following process:

The heat

Δ Q_{in}

is given by

Δ Q_{in} = T_{H} Δ S

, and as before

Δ W = \int_{C} T d S = \int_{A} d T d S

. Thus,

Δ W

is the area

A

enclosed by the closed curve

C

. This is clearly smaller than the area enclosed by the corresponding Carnot cycle (dashed rectangle). Now divide a general cyclic process into

C = C_{1} \cup C_{2}

, as sketched in the following figure:

Figure 6.10: A generic cyclic process divided into two parts by an isotherm at temperature

T_{I}

。

This process describes two cylic processes acting one after the other, where the heat dropped during cycle

C_{1}

is injected during cycle

C_{2}

at temperature

T_{I}

. It follows from the discussion above that

\begin{matrix} (6.57) & η (C_{2}) = \frac{Δ W_{2}}{Δ Q_{2, in}} ⩽ \frac{T_{I} - T_{C}}{T_{I}} = 1 - \frac{T_{C}}{T_{I}} \end{matrix}

which means that the cycle

C_{2}

is less efficient than the Carnot process acting between temperatures

T_{I}

and

T_{C}

. It remains to show that the cycle

C_{1}

is also less efficient than the Carnot cycle acting betweeen temperatures

T_{H}

and

T_{I}

. The work

Δ W_{1}

done along

C_{1}

is again smaller than the area enclosed by the latter Carnot cycle, i.e. we have

Δ W_{1} ⩽ (T_{H} - T_{I}) Δ S

. Furthermore, we must have

Δ Q_{1, in} ⩾ Δ Q_{1, out} = T_{I} Δ S

, which yields

η (C_{1}) = \frac{Δ W_{1}}{Δ Q_{1, in}} ⩽ \frac{T_{H} - T_{I}}{T_{I}} ⩽ 1 - \frac{T_{I}}{T_{H}} .

Thus, the cycle

C_{1}

is less efficient than the Carnot cycle acting between temperatures

T_{H}

and

T_{I}

. It follows that the cycle

C = C_{1} \cup C_{2}

must be less efficient than the Carnot cycle acting between temperatures

T_{H}

and

T_{C}

6.4.3 The Diesel Engine

Another example of a cyclic process is the Diesel engine. The idealized version of this process consists of the following 4 steps:

I \to II

: isentropic (adiabatic) compression
II

\to

III: reversible heating at constant pressure
III

\to

IV: adiabatic expansion with work done by the expanding fluid
IV

\to

I: reversible cooling at constant volume

Figure 6.11: The process describing the Diesel engine in the

(P, V)

-diagram.

As before, we define the thermal efficiency to be

η_{Diesel} = \frac{Δ W}{Δ Q_{in}} = \frac{(\int_{I}^{I I} + \int_{I I}^{I I I} + \int_{I I I}^{I V} + \int_{I V}^{I}) T d S}{\int_{I I}^{I I I} T d S}

As in the discussion of the Carnot process we use an ideal gas, with

V = N k_{B} T, E =

\frac{3}{2} P V

, and

d E = T d S - P d V

. Since

d S = 0

on the paths I

\to

II and III

\to IV

, it follows
that

\begin{matrix} (6.58) & η_{Diesel} = 1 + \frac{\int_{I V}^{I} T d S}{\int_{I I}^{I I I} T d S} \end{matrix}

Using (6.42), the integrals in this expression are easily calculated as

\begin{aligned} \int_{I V}^{I} T d S & = \int_{I V}^{I} (d E + P d V) = \int_{I V}^{I} (\frac{3}{2} V d P + \frac{5}{2} P \underset{= 0}{\underset{⏟}{d V}}) \\ = \frac{3}{2} N k_{B} (T_{I} - T_{I V}), \\ \int_{I I}^{I I I} T d S & = \int_{I I}^{I I I} (d E + P d V) = \int_{I I}^{I I I} (\frac{3}{2} V \underset{= 0}{\underset{⏟}{d P}} + \frac{5}{2} P d V) \\ = \frac{5}{2} N k_{B} (T_{I I I} - T_{I I}), \end{aligned}

which means that the efficiency

η_{Diesel}

is given by

\begin{matrix} (6.59) & η_{Diesel} = 1 - \frac{3}{5} \frac{T_{I V} - T_{I}}{T_{I I I} - T_{I I}} \end{matrix}

6.5 Thermodynamic potentials

The first law can be rewritten in terms of other "thermodynamic potentials", which are sometimes useful, and which are naturally related to different equilibrium ensembles.

We start from the

1^{st}

law of thermodynamics in the form

\begin{matrix} (6.60) & d E = T d S - P d V + μ d N + \dots (= T d S + \sum_{i = 1}^{n} J_{i} d X_{i}) \end{matrix}

By (6.60)

E

is naturally viewed as a function of

(S, V, N)

(or more generally of

S

and

{X_{i}}

). To get a thermodynamic potential that naturally depends on (

T, V, N

) (or more generally,

T

and

{X_{i}}

), we form the free energy

\begin{matrix} (6.61) & F = E - T S . \end{matrix}

Taking the differential of this, we get

\begin{aligned} d F & = d E - S d T - T d S \\ = T d S - P d V + μ d N + \dots - S d T - T d S \\ = - S d T - P d V + μ d N + \dots \\ ( & = - S d T + \sum_{i = 1}^{n} J_{i} d X_{i}) \end{aligned}

Writing out the differential

d F

\begin{matrix} (6.62) & d F = {\frac{\partial F}{\partial T} |}_{V, N} d T + {\frac{\partial F}{\partial V} |}_{T, N} d V + \dots \end{matrix}

and comparing the coefficients, we get

\begin{matrix} (6.63) & 0 = ({\frac{\partial F}{\partial T} |}_{V, N} + S) d T + ({\frac{\partial F}{\partial V} |}_{T, N} + P) d V + ({\frac{\partial F}{\partial N} |}_{T, V} - μ) d N + \dots \end{matrix}

This yields the following relations:

\begin{matrix} (6.64) & S = - {\frac{\partial F}{\partial T} |}_{V, N}, P = - {\frac{\partial F}{\partial V} |}_{T, N}, μ = {\frac{\partial F}{\partial N} |}_{T, V}, \dots \end{matrix}

By the first of these equations, the entropy

S = S (T, V, N)

is naturally a function of

(T, V, N)

, which suggests a relation between

F

and the canonical ensemble. As discussed in section 4.3, in this ensemble we have

\begin{matrix} (6.65) & ρ = ρ (T, V, N) = \frac{1}{Z} e^{- \frac{H (N, V)}{k_{B} T}} and S = - k_{B} tr ρ \log ρ \end{matrix}

We now seek an

F

satisfying

S = - {\frac{\partial F}{\partial T} |}_{V, N}

. A simple calculation shows

\begin{matrix} (6.66) & F (T, V, N) = - k_{B} T \log Z (T, V, N) \end{matrix}

Indeed:

\begin{aligned} \frac{\partial F}{\partial T} & = - k_{B} {\log tr e^{- \frac{H}{k_{B} T}} + \frac{1}{k_{B} T} \frac{tr H e^{- \frac{H}{k_{B} T}}}{tr e^{- \frac{H}{k_{B} T}}}} \\ = k_{B} tr ρ \log ρ = - S \end{aligned}

In the same way, we may look for a function

G

of the variables

(T, μ, V)

. To this end,
we form the grand potential

\begin{matrix} (6.67) & G = E - T S - μ N = F - μ N \end{matrix}

The differential of

G

\begin{aligned} d G & = d F - μ d N - N d μ \\ = - S d T - P d V + μ d N - μ d N - N d μ \\ = - S d T - P d V - N d μ \end{aligned}

Writing out

d G

d G = {\frac{\partial G}{\partial T} |}_{V, μ} d T + {\frac{\partial G}{\partial V} |}_{T, μ} d V + \dots

and comparing the coefficients, we get

0 = ({\frac{\partial G}{\partial T} |}_{V, μ} + S) d T + ({\frac{\partial G}{\partial V} |}_{T, μ} + P) d V + ({\frac{\partial G}{\partial μ} |}_{T, V} + N) d μ

which yields the relations

\begin{matrix} (6.68) & S = - {\frac{\partial G}{\partial T} |}_{V, μ}, N = - {\frac{\partial G}{\partial μ} |}_{T, V}, P = - {\frac{\partial G}{\partial V} |}_{T, μ} \end{matrix}

In the first of these equations,

S

is naturally viewed as a function of the variables

(T, μ, V)

, suggesting a relationship between

G

and the grand canonical ensemble. As discussed in section 4.4 , in this ensemble we have

\begin{matrix} (6.69) & ρ (T, μ, V) = \frac{1}{Y} e^{- \frac{(H (V) - μ \hat{N})}{k_{B} T}} and S = - k_{B} tr ρ \log ρ \end{matrix}

We now seek a function

G

satisfying

S = - {\frac{\partial G}{\partial T} |}_{μ, V}

and

N = - {\frac{\partial G}{\partial μ} |}_{T, V}

. An easy calculation reveals

\begin{matrix} (6.70) & G (T, μ, V) = - k_{B} T \log Y (T, μ, V) \end{matrix}

Indeed:

\begin{aligned} \frac{\partial G}{\partial T} & = - k_{B} {\log tr e^{- \frac{(H - μ \hat{N})}{k_{B} T}} + \frac{1}{k_{B} T} \frac{tr (H - μ \hat{N}) e^{- \frac{(H - μ \hat{N})}{k_{B} T}}}{tr e^{- \frac{(H - μ \hat{N})}{k_{B} T}}}} \\ = k_{B} tr ρ \log ρ = - S \end{aligned}

The second relation can be demonstrated in a similar way (with

N = ⟨ \hat{N} ⟩

). To get a function

H

which naturally depends on the variables

(P, T, N)

, we form the free enthalpy (or Gibbs potential)

\begin{matrix} (6.71) & H = E - T S + P V = F + P V . \end{matrix}

It satisfies the relations

\begin{matrix} (6.72) & S = - {\frac{\partial H}{\partial T} |}_{P, N}, μ = {\frac{\partial H}{\partial N} |}_{P, T}, V = {\frac{\partial H}{\partial P} |}_{N, T} \end{matrix}

or equivalently

\begin{matrix} (6.73) & d H = - S d T + V d P + μ d N \end{matrix}

The free

^{4}

enthalpy is often used in the context of chemical processes, because these naturally occur at constant atmospheric pressure. For processes at constant pressure

P

(isobaric processes) we have

\begin{matrix} (6.74) & d H = - S d T + μ d N \end{matrix}

Assuming that the entropy

S = S (E, V, N_{i}, \dots)

is an extensive quantity, we can derive relations between the various potentials. The extensive property of

S

means that

\begin{matrix} (6.75) & S (λ E, λ V, λ N_{i}) = λ S (E, V, N_{i}), for λ > 0 \end{matrix}

Taking the partial derivative

\frac{\partial}{\partial λ}

of this expression gives

\begin{matrix} (6.76) & S = {\frac{\partial S}{\partial E} |}_{V, N_{i}} E + {\frac{\partial S}{\partial V} |}_{E, N_{i}} V + {\sum_{i} \frac{\partial S}{\partial N_{i}} |}_{V, E} N_{i} \end{matrix}

Together with the relations

\begin{matrix} (6.77) & {\frac{\partial S}{\partial E} |}_{V, N_{i}} = \frac{1}{T},, {\frac{\partial S}{\partial V} |}_{E, N_{i}} = \frac{P}{T}, {\frac{\partial S}{\partial N_{i}} |}_{V, E} = - \frac{μ_{i}}{T} \end{matrix}

we find the Gibbs-Duhem relation (after multiplication with

T

\begin{matrix} (6.78) & E + P V - \sum_{i} μ_{i} N_{i} - T S = 0 \end{matrix}

or equivalently

\begin{matrix} (6.79) & H = \sum_{i} μ_{i} N_{i} . \end{matrix}

Let us summarize the properties of the potentials we have discussed so far in a table:

Thermodynamic

potential

Definition

First Law

Natural

variables

entropy

S

fundamental

T d S = d E + P d V - μ d N

E, V, N

free energy

F

F = E - T S

d F = - S d T - P d V + μ d N

T, V, N

grand potential

G

G = E - T S - μ N

d G = - S d T - P d V - N d μ

T, V, μ

free enthalpy

H

H = E - T S + P V

d H = - S d T + V d P + μ d N

T, P, N

Table 6.2: Relationship between various thermodynamic potentials

The relationship between the various potentials can be further elucidated by means of the Legendre transform. This characterization is important because it makes transparent the convexity respectively concavity properties of

G, F

following from the convexity of

S

Example: Virial expansion and van der Waals equation of state

As an example for the use of potentials, we employ the cluster expansion discussed in section 4.6 to derive the equation of state for a realistic monoatomic gas. Calculations are left as an exercise (problem B.26).

Recall that the cluster expansion is given by

\frac{1}{V} \log Y = \frac{1}{λ^{3}} \sum_{l = 1}^{\infty} b_{l} (V, β) z^{l}

where

Y

is the grand canonical partition function,

λ

is the thermal de Broglie wavelength, and

z = e^{β μ}

is the fugacity. Using the Gibbs-Duhem relation and (6.70), the cluster expansion gives an expansion of the pressure,

P = \frac{k_{B} T}{λ^{3}} \sum_{l = 1}^{\infty} b_{l} (V, β) z^{l}

By (6.68) and (6.70), it also yields an expansion of the particle density

n = N / V

n = \frac{1}{λ^{3}} \sum_{l = 1}^{\infty} l b_{l} (V, β) e^{l β μ}

We would like to eliminate

z

in favor of the more accessible number density

n

. For this,
we make the ansatz

z = λ^{3} n + a_{2} {(λ^{3} n)}^{2} + a_{3} {(λ^{3} n)}^{3} + \dots

and determine the coefficients

a_{2}, a_{3}, \dots

from the expansion of

n

a_{2} = - 2 b_{2}, a_{3} = 8 b_{2}^{2} - 3 b_{2}, \dots

Plugging this into the expansion of

P

, we obtain

P = k_{B} T n (1 + B_{2} (T) n + B_{3} (T) n^{2} + \dots)

where

B_{2} = - b_{2} λ^{3}, B_{3} = (4 b_{2}^{2} - 3 b_{3}) λ^{6}, \dots

This is known as the virial expansion, and the

B_{l}

are known as the virial coefficients. The first order contribution yields the equation of state of the classical monoatomic ideal gas. It corresponds to the situation where the particles do not interact. For a realistic dilute gas, it is reasonable to expect that low orders in the expansion give a good approximation; we consider second order. At this order, we obtain

\begin{matrix} (6.80) & P \approx k_{B} T n (1 + B_{2} (T) n) . \end{matrix}

Let us compute

B_{2}

under the assumption that the interaction is described by the spherically symmetric two-body potential

V (r) = {\begin{cases} + \infty & ∣ r < r_{0} \\ - u_{0} {(\frac{r_{0}}{r})}^{6} & ∣ r ⩾ r_{0} \end{cases}

where

r_{0}

is twice the radius of the atoms. This potential models a hard core repulsion at atomic distances and a moderately decreasing attraction at large distances. Using the integral formula (4.119) for

b_{2}

, in the high temperature limit we find

B_{2} \approx \frac{V_{a}}{2} (1 - \frac{u_{0}}{k_{B} T}),

where

V_{a} = 4 π / 3 r_{0}^{3}

is the effective volume of one particle. Plugging this into the second order virial expansion (6.80), we get

P \approx k_{B} T n + \frac{V_{a}}{2} (k_{B} T - u_{0}) n^{2} .

Substituting

1 + x \approx \frac{1}{1 - x}

this can be written in the form

\begin{matrix} (6.81) & (P + a {(\frac{N}{V})}^{2}) (V - N b) \approx N k_{B} T \end{matrix}

with the coefficients

a = \frac{V_{a} u_{0}}{2}, b = \frac{V_{a}}{2} .

Equation (6.81) is known as the van-der-Waals equation. The coefficient

b

can be interpreted as a volume per particle by which the volume available to the system is reduced due to exclusion of particles. Since

r_{0}

is the distance of minimal approach, i.e., twice the radius of the particles,

b

amounts to twice the effective volume of the particles. The coefficient

a

decreases the pressure

P

in the system due attractive particle interaction. In applications we should bear in mind that the equation can be exptected to give a good approximation in case the volume per particle available is much larger than the effective volume of the particles,

V ≫ b / 2

, and for high temperature,

u_{0} ≪ k_{B} T

6.6 Chemical Equilibrium

We consider chemical reactions characterized by a

k

-tuple

\underset{―}{r} = (r_{1}, \dots, r_{k})

of integers corresponding to a chemical reaction of the form

\begin{matrix} (6.82) & \sum_{r_{i} < 0} | r_{i} | χ_{i} ⇆ \sum_{r_{i} > 0} | r_{i} | χ_{i} \end{matrix}

where

χ_{i}

is the chemical symbol of the

i

-th compound. For example, the reaction

C + O_{2} ⇆ {CO}_{2}

is described by

χ_{1} = C, χ_{2} = O_{2}, χ_{3} = {CO}_{2}

and

r_{1} = - 1, r_{2} = - 1, r_{3} = + 1

, or

\underset{―}{r} =

(- 1, - 1, + 1)

. The full system is described by some complicated Hamiltonian

H (V)

and number operators

{\hat{N}}_{i}

for the

i

-th compound. Since the dynamics can change the particle number, we will have

[H (V), {\hat{N}}_{i}] \neq 0

in general. We imagine that an entropy

S (E, V, {N_{i}})

can be assigned to an ensemble of states with energy between

E - Δ E

and

E

, and average particle numbers

{N_{i} = ⟨ {\hat{N}}_{i} ⟩}

, but we note that the definition of

S

in microscopic terms is far from obvious because

{\hat{N}}_{i}

is not a constant of motion.

The entropy should be maximized in equilibrium. Since

\underset{―}{N} = (N_{1}, \dots, N_{k})

changes
by

\underset{―}{r} = (r_{1}, \dots, r_{k})

in a reaction, the necessary condition for equilibrium is

\begin{matrix} (6.83) & {\frac{d}{d n} S (E, V, \underset{―}{N} + n \underset{―}{r}) |}_{n = 0} = 0 \end{matrix}

Since by definition

{\frac{\partial S}{\partial N_{i}} |}_{V, E} = - \frac{μ_{i}}{T}

, in equilibrium we must have

\begin{matrix} (6.84) & 0 = \underset{―}{μ} \cdot \underset{―}{r} = \sum_{i = 1}^{k} μ_{i} r_{i} \end{matrix}

Let us now assume that in equilibrium we can use the expression for

μ_{i}

of an ideal gas with

k

distinguishable components and

N_{i}

indistinguishable particles of the

i

-th component. This is basically the assumption that interactions contribute negligibly to the entropy of the equilibrium state. According to the discussion in section 4.2.3 the total entropy is given by

\begin{matrix} (6.85) & S = \sum_{i = 1}^{k} S_{i} + Δ S \end{matrix}

where

S_{i} = S (E_{i}, V_{i}, N_{i})

is the entropy of the

i

-th species,

Δ S

is the mixing entropy, and we have

\begin{matrix} (6.86) & \frac{N_{i}}{V_{i}} = \frac{N}{V}, \sum N_{i} = N, \sum V_{i} = V, \sum E_{i} = E \end{matrix}

The entropy of the

i

-th species is given by

\begin{matrix} (6.87) & S_{i} = N_{i} k_{B} [\log \frac{e V_{i}}{N_{i}} + \log {(\frac{4}{3} \frac{E_{i}}{N_{i}} π e m_{i})}^{\frac{3}{2}}] \end{matrix}

The mixing entropy is given by

\begin{matrix} (6.88) & Δ S = - N k_{B} \sum_{i = 1}^{k} (c_{i} \log c_{i} - c_{i}), \end{matrix}

where

c_{i} = \frac{N_{i}}{N}

is the concentration of the

i

-th component. Let

{\bar{μ}}_{i}

be the chemical potential of the

i

-th species without taking into account the contribution due to the mixing:

\begin{aligned} \frac{{\bar{μ}}_{i}}{T} = - {\frac{\partial S_{i}}{\partial N_{i}} |}_{V_{i}, E_{i}} & = k_{B} \log [\frac{V_{i}}{N_{i}} {(\frac{4 π m_{i} E_{i}}{3 N_{i}})}^{\frac{3}{2}}] \\ = - \frac{S_{i}}{N_{i}} + \frac{5}{2} k_{B} \end{aligned}

We have for the total chemical potential for the

i

-speicies:

\begin{aligned} μ_{i} & = {\bar{μ}}_{i} + k_{B} T \log c_{i} \\ = \frac{5}{2} k_{B} T - \frac{S_{i} T}{N_{i}} + k_{B} T \log c_{i} \\ = \frac{1}{N_{i}} (E_{i} + P V_{i} - T S_{i}) + k_{B} T \log c_{i} \\ = \underset{\begin{array}{c} H_{i} / N_{i} = free enthalpy \\ per particle for species i \end{array}}{\underset{⏟}{h_{i}}} + k_{B} T \log c_{i}, \end{aligned}

where we have used the equations of state for the ideal gas for each species. From this it follows that the condition for equilibrium becomes

\begin{matrix} (6.89) & 0 = \sum_{i} μ_{i} \cdot r_{i} = \sum_{i} (h_{i} r_{i} + k_{B} T \log c_{i}^{r_{i}}), \end{matrix}

which yields

\begin{matrix} (6.90) & 1 = e^{\frac{Δ h}{k_{B} T}} \prod_{i} c_{i}^{r_{i}} \end{matrix}

or equivalently

\begin{matrix} (6.91) & e^{- \frac{Δ h}{k_{B} T}} = \frac{\prod_{r_{i} > 0} c_{i}^{| r_{i} |}}{\prod_{r_{i} < 0} c_{i}^{| r_{i} |}} \end{matrix}

with

Δ h = \sum_{i} r_{i} h_{i} =

enthalpy increase for one reaction. The above relation is sometimes called the "mass-action law". It is clearly in general not an exact relation, because we have treated the constituents as ideal gases. Nevertheless, it is often a surprisingly good approximation.

6.7 Phase Co-Existence and Clausius-Clapeyron Relation

We consider a system comprised of

k

compounds with particle numbers

N_{1}, \dots, N_{k}

. It is assumed that chemical reactions are not possible, so each

N_{i}

is conserved. The entropy is assumed to be given as a function of

S = S (\underset{―}{X})

, where

\underset{―}{X} = (E, V, N_{1}, \dots, N_{k})

(here we also include

E

into the thermodynamic coordinates.) We assume that the system is in an equilibrium state with

φ

coexisting pure phases which are labeled by

α = 1, \dots, φ

. The equilibrium state for each phase

α

is thus characterized by some vector

{\underset{―}{X}}^{(α)}

, or rather the corresponding ray

{λ {\underset{―}{X}}^{(α)} ∣ λ > 0}

since we can scale up the volume, energy, and numbers of particles by a positive constant.

Examples:

Consider the following example of a phase boundary between coffee and sugar:

Figure 6.12: The phase boundary between solution and a solute.

In this example we have

k = 2

compounds (coffee, sugar) with

φ = 2

coexisting phases (solution, sugar at bottom). The solid phase and coffee/sugar solution phases are described by vectors

\begin{matrix} (6.92) & {\underset{―}{X}}^{(solid)} = (E^{(1)}, N_{sugar}^{(1)}, N_{coffee}^{(1)} = 0, V^{(1)}), {\underset{―}{X}}^{(solution)} (E^{(2)}, N_{sugar}^{(2)}, N_{coffee}^{(2)}, V^{(2)}), \end{matrix}

respectively. In this example we need 2 independent parameters to describe phase equilibrium, such as the temperature

T

of the coffee and the concentration

c

of sugar, i.e. sweetness of the coffee.
2) Another example is the ice-vapor-water diagram where we only have

k = 1

substance (water). At the triple point, we have

φ = 3

coexisting phases. At the water-vapor boundary, we have

φ = 2

coexisting phases, and we need one parameter to fix where we are on this phase boundary. Away from any phase boundary, only

φ = 1

phase is possible.

The temperature

T

, pressure

P

, and chemical potentials

μ_{i}

must have the same value in each phase, i.e. we have for all

α

\begin{matrix} (6.93) & \frac{\partial S}{\partial E} ({\underset{―}{X}}^{(α)}) = \frac{1}{T}, \frac{\partial S}{\partial V} ({\underset{―}{X}}^{(α)}) = \frac{P}{T}, \frac{\partial S}{\partial N_{i}} ({\underset{―}{X}}^{(α)}) = - \frac{μ_{i}}{T} . \end{matrix}

We define a

(k + 2)

-component vector

\underset{―}{ξ}

, which is is independent of

α

, as follows:

\begin{matrix} (6.94) & \underset{―}{ξ} = (\frac{1}{T}, \frac{P}{T}, - \frac{μ_{1}}{T}, \dots, - \frac{μ_{k}}{T}) \end{matrix}

As an example consider the following phase diagram for 6 phases:

Figure 6.13: Imaginary phase diagram for the case of 6 different phases. At each point on a phase boundary which is not an intersection point,

φ = 2

phases are supposed to coexist. At each intersection point

φ = 4

phases are supposed to coexist.

From the discussion in the previous sections we know that
(1)

S

is extensive in equilibrium:

\begin{matrix} (6.95) & S (λ \underset{―}{X}) = λ S (\underset{―}{X}), λ > 0 . \end{matrix}

(2)

S

is a concave function in

\underset{―}{X} \in R^{k + 2}

(subadditivity), and

\begin{matrix} (6.96) & \sum_{α} λ^{(α)} S ({\underset{―}{X}}^{(α)}) ⩽ S (\sum_{α} λ^{(α)} {\underset{―}{X}}^{(α)}), \end{matrix}

as long as

\sum_{α} λ^{(α)} = 1, λ^{(α)} ⩾ 0

. Since the coexisting phases are in equilibrium with each other, we must have "=" rather than "

<

" in the above inequality. Otherwise, the entropy would be maximized for some non-trivial linear combination

{\underset{―}{X}}_{min} =

\sum_{α} λ^{(α)} {\underset{―}{X}}^{(α)}

, and only one homogeneous phase given by this minimizer

{\underset{―}{X}}_{min}

could be realized.

By (1) and (2) it follows that in the region

C \subset R^{2 + k}

, where several phases can coexist,

S

is linear,

S (\underset{―}{X}) = \underset{―}{ξ} \cdot \underset{―}{X}

for all

\underset{―}{X} \in C, \underset{―}{ξ} =

const. in

C

, and

C

consists of positive linear combinations

\begin{matrix} (6.97) & C = {\underset{―}{X} = \sum_{α = 1}^{φ} λ^{(α)} {\underset{―}{X}}^{(α)} : λ^{(α)} ⩾ 0} \end{matrix}

in other words, the coexistence region

C

is a convex cone generated by the vectors

{\underset{―}{X}}^{(α)}, α = 1, \dots, φ

. The set of points in the space

(P, T, {c_{i}})

where equilibrium between

φ

phases holds (i.e. the phase boundaries in a

P - T - {c_{i}}

-diagram) can be characterized as follows. Since

\underset{―}{ξ}

is constant within the convex cone

C

, we have for any

\underset{―}{X} \in C

and any

α = 1, \dots, φ

and any

I

\begin{aligned} 0 = {\frac{d}{d λ} ξ_{I} (\underset{―}{X} + λ {\underset{―}{X}}^{(α)}) |}_{λ = 0} & = \sum_{J} X_{J}^{(α)} \frac{\partial}{\partial X_{J}} ξ_{I} (\underset{―}{X}) = \sum_{J} X_{J}^{(α)} \frac{\partial^{2}}{\partial X_{J} \partial X_{I}} S (\underset{―}{X}) \\ = \sum_{J} X_{J}^{(α)} \frac{\partial^{2}}{\partial X_{I} \partial X_{J}} S (\underset{―}{X}) \\ = \sum_{J} X_{J}^{(α)} \frac{\partial}{\partial X_{I}} ξ_{J} (\underset{―}{X}) \end{aligned}

where we denote the

k + 2

components of

\underset{―}{X}

{X_{I}}

. Multiplying this equation by

d X_{I}

and summing over

I

, this relation can be written as

\begin{matrix} (6.98) & {\underset{―}{X}}^{(α)} \cdot d \underset{―}{ξ} = 0 \end{matrix}

which must hold in the coexistence region

C

. Since the equation must hold for all

α = 1, \dots, φ

, the coexistence region is is subject to

φ

constraints, and we therefore need

f = (2 + k - φ)

parameters to describe the coexistence region in the phase diagram. This statement is sometimes called the Gibbs phase rule.

Examples: Consider again the example of a phase boundary between coffee and sugar, where we had

k = 2

compounds (coffee, sugar) with

φ = 2

coexisting phases (solution, sugar at bottom). The phase rule tells us that we need

f = 2 + 2 - 2 = 2

independent parameters to describe phase equilibrium, which is correct. In the ice-vapor-water diagram we only had

k = 1

substance (water). At the triple point, we have

φ = 3

coexisting phases and

f = 1 + 2 - 3 = 0

, which is consistent because a point is a 0 -dimensional manifold. At the water-ice coexistence line, we have

φ = 2

and

f = 1 + 2 - 2 = 1

, which is the correct dimension of a line.

Now consider a 1-component system

(k = 1)

, such that

\underset{―}{X} = (E, N, V)

and

\underset{―}{ξ} =

(\frac{1}{T}, \frac{P}{T}, - \frac{μ}{T})

The

φ

different phases are described by

{\underset{―}{X}}^{(1)} = (E^{(1)}, N^{(1)}, V^{(1)}), \dots, {\underset{―}{X}}^{(φ)} = (E^{(φ)}, N^{(φ)}, V^{(φ)}) .

In the case of

φ = 2

different phases we thus have

\begin{aligned} E^{(1)} d (\frac{1}{T}) + V^{(1)} d (\frac{P}{T}) - N^{(1)} d (\frac{μ}{T}) = 0 \\ E^{(2)} d (\frac{1}{T}) + V^{(2)} d (\frac{P}{T}) - N^{(2)} d (\frac{μ}{T}) = 0 \end{aligned}

We assume that the particle numbers are equal in both phases,

N^{(1)} = N^{(2)} \equiv N

, which
means that

f = 2 + k - φ = 1

.Thus,

\begin{matrix} (6.99) & [E^{(1)} - E^{(2)} + P (V^{(1)} - V^{(2)})] \frac{d T}{T^{2}} = (V^{(1)} - V^{(2)}) \frac{d P}{T} \end{matrix}

or, equivalently,

\begin{matrix} (6.100) & \frac{d P (T)}{d T} = \frac{Δ E + P Δ V}{T Δ V} \end{matrix}

Together with the relation

Δ E = T Δ S - P Δ V

we find the Clausius-Clapeyronequation

\begin{matrix} (6.101) & \frac{d P}{d T} = \frac{Δ S}{Δ V} \end{matrix}

As an application, consider a solid (phase 1) in equilibrium with its vapor (phase 2). For the volume we should have

V^{(1)} ≪ V^{(2)}

, from which it follows that

Δ V = V^{(1)} - V^{(2)} \approx

- V^{(2)}

. For the vapor phase, we assume the relations for an ideal gas,

P V^{(2)} = k_{B} T N^{(2)} =

k_{B} T N

. Substitution for

P

gives

\begin{matrix} (6.102) & \frac{d P}{d T} = \frac{Δ Q}{N} \frac{P}{k_{B} T^{2}}, with Δ Q = - Δ S \cdot T \end{matrix}

Assuming

Δ q = \frac{Δ Q}{N}

to be roughly independent of

T

, we obtain

\begin{matrix} (6.103) & P (T) = P_{0} e^{- \frac{Δ q}{k_{B} T}} \end{matrix}

on the phase boundary, see the following figure:

Figure 6.14: Phase boundary of a vapor-solid system in the

(P, T)

-diagram

6.8 Osmotic Pressure

We consider a system made up of two compounds and define

\begin{aligned} N_{1} = particle number of "ions" (solute) \\ N_{2} = particle number of "water molecules" (solvent). \end{aligned}

The corresponding chemical potentials are denoted

μ_{1}

and

μ_{2}

. The grand canonical partition function,

Y (μ_{1}, μ_{2}, V, β) = tr [e^{- β (H (V) - μ_{1} {\hat{N}}_{1} - μ_{2} {\hat{N}}_{2})}]

can be written as

\begin{matrix} (6.104) & Y (μ_{1}, μ_{2}, V, β) = \sum_{N_{1} = 0}^{\infty} Y_{N_{1}} (μ_{2}, β, V) e^{β μ_{1} N_{1}} \end{matrix}

where

Y_{N_{1}}

is the grand canonical partition function for substance 2 with a fixed number

N_{1}

of particles of substance

1^{5}

. Let now

y_{N} := \frac{1}{V} \frac{Y_{N}}{Y_{0}}

. It then follows that

\begin{matrix} (6.105) & \log Y \equiv \log Y_{0} + \log \frac{Y}{Y_{0}} = \log Y_{0} + \log [1 + \sum_{N_{1} > 0} V y_{N_{1}} e^{β μ_{1} N_{1}}] \end{matrix}

hence

\begin{matrix} (6.106) & \log Y = \log Y_{0} + V y_{1} (\underset{\begin{matrix} no V dependence for large \\ systems as free energy \\ G = - k_{B} T \log Y \sim V \end{matrix}}{\underset{⏟}{μ_{2}, β}}) e^{β μ_{1}} + O (e^{2 β μ_{2}}) \end{matrix}

For the (expected) particle number of substance 1 we therefore have

\begin{matrix} (6.107) & N_{1} = - \frac{1}{β} \frac{\partial}{\partial μ_{1}} \log Y (μ_{1}, μ_{2}, V, β) \end{matrix}

which follows from

\begin{matrix} (6.108) & d G = - S d T - P d V - N_{1} d μ_{1} - N_{2} d μ_{2} \end{matrix}

using the manipulations with thermodynamic potentials reviewed in section 6.5. Because

\log Y_{0}

does not depend on

μ_{1}

, we find

\begin{matrix} (6.109) & N_{1} / V = n_{1} = y_{1} (μ_{2}, β) e^{β μ_{1}} + O (e^{2 β μ_{1}}) \end{matrix}

On the other hand, we have for the pressure (see section 6.5)

\begin{matrix} (6.110) & P = - \frac{1}{β} \frac{\partial}{\partial V} \log Y (μ_{1}, μ_{2}, V, β) \end{matrix}

which follows again from (6.108). Using that

y_{1}

is approximately independent of

V

for large volume, we obtain the following relation:

P (μ_{2}, N_{1}, β) = P (μ_{2}, N_{1} = 0, β) + y_{1} (μ_{2}, T) e^{β μ_{1}} / β + O (e^{2 β μ_{1}})

Using

e^{β μ_{1}} = \frac{n_{1}}{y_{1}} + O (n_{1}^{2})

, which follows from (6.109), we get

\begin{matrix} (6.111) & P (μ_{2}, N_{1}, T) = P (μ_{2}, N_{1} = 0, T) + k_{B} T n_{1} + O (n_{1}^{2}) \end{matrix}

Here we note that

y_{1}

, which in general is hard to calculate, fortunately does not appear on the right hand side at this order of approximation.

Consider now two copies of the system called A and B, separated by a wall which leaves through water, but not the ions of the solute. The concentration

n_{1}^{(A)}

of ions on one side of the wall need not be equal to the concentration

n_{1}^{(B)}

on the other side. So we have different pressures

P^{(A)}

and

P^{(B)}

. Their difference is

Δ P = P^{(A)} - P^{(B)} = k_{B} T (n_{1}^{(A)} - n_{1}^{(B)})

hence, writing

Δ n = n_{1}^{(A)} - n_{1}^{(B)}

, we obtain the osmotic formula, due to van 't Hoff:

\begin{matrix} (6.112) & Δ P = k_{B} T Δ n \end{matrix}

In the derivation of this formula we neglected terms of the order

n_{1}^{2}

, which means that the formula is valid only for dilute solutions!

Appendix A

Dynamical Systems and Approach to Equilibrium

A. 1 The Master Equation

In this section, we will study a toy model for dynamically evolving ensembles (i.e. non-stationary ensembles) [in this section, we follow mostly Ch. 6 of "Physique Statistique" by A. Georges, M. Mézard, École Polytechnique (2010)]. We will not start from a Hamiltonian description of the dynamics, but rather work with a phenomenological description which is already probabilistic. In this approach, the ensemble is described by a time-dependent probability distribution

{p_{n} (t)}

, where

p_{n} (t)

is the probability of the system to be in state

n

at time

t

. Since

p_{n} (t)

are to be probabilities, we evidently should have

\sum_{i = 1}^{N} p_{i} (t) = 1, p_{i} (t) ⩾ 0

for all

t

We assume that the time dependence is determined by the dynamical law

\begin{matrix} (A.1) & \frac{d p_{i} (t)}{d t} = \sum_{j \neq i} [T_{i j} p_{j} (t) - T_{j i} p_{i} (t)] \end{matrix}

where

T_{i j} > 0

is the transition amplitude for going from state

j

to the state

i

per unit of time. We call this law the "master equation." As already discussed in sec. 3.2 , the master equation can be thought of as a version of the Boltzmann equation. In the context of quantum mechanics, the transition amplitudes

T_{i j}

induced by some small perturbation of the dynamics

H_{1}

would e.g. be given by Fermi's golden rule,

T_{i j} = 2 π n / h | ⟨ i | H_{1} | j ⟩ |^{2}

and would therefore be symmetric in

i

and

j, T_{i j} = T_{j i}

. In this section, we do not assume that the transition amplitude is symmetric as this would exclude interesting examples.

It is instructive to check that the master equation has the desired property of keeping

p_{i} (t) ⩾ 0

and

\sum_{i} p_{i} (t) = 1

. The first property is seen as follows. Suppose that

t_{0}

is the first time that some

p_{i} (t_{0}) = 0

. From the structure of the master equation, it then follows
that

d p_{i} (t_{0}) / d t > 0

, unless in fact all

p_{j} (t_{0}) = 0

. This is impossible, because the sum of the probabilities equal to 1 for all times. Indeed,

\begin{aligned} \frac{d}{d t} \sum_{i} p_{i} & = \sum_{i} \frac{d}{d t} p_{i} \\ = \sum_{i} \sum_{j : j \neq i} (T_{i j} p_{j} - T_{j i} p_{i}) \\ = \sum_{i, j : i \neq j} T_{i j} p_{j} - \sum_{i, j : j \neq i} T_{j i} p_{i} = 0 . \end{aligned}

An equilibrium state corresponds to a distribution

{p_{i}^{eq}}

which is constant in time and is a solution to the master equation, i.e.

\begin{matrix} (A.2) & \sum_{j : j \neq i} T_{i j} p_{j}^{eq} = p_{i}^{eq} \sum_{j : j \neq i} T_{j i} . \end{matrix}

An important special case is the case of symmetric transition amplitudes. We are in this case for example if the underlying microscopic dynamics is reversible. In that case, the uniform distribution

p_{i}^{eq} = \frac{1}{N}

is always stationary (micro canonical ensemble).

Example: Time evolution of a population of bacteria

Consider a population of some kind of bacteria, characterized by the following quantities:

\begin{aligned} n & = number of bacteria in the population \\ M & = mortality rate \\ R & = reproduction rate \\ p_{n} (t) & = probability that the population consists of n bacteria at instant t \end{aligned}

In this case the master equation (A.1) reads:

\begin{matrix} (A.3) & \frac{d}{d t} p_{0} = M p_{1} \end{matrix}

It means that the transition amplitudes are given by

\begin{matrix} (A.5) & {\begin{cases} T_{n (n + 1)} & = M (n + 1) \\ T_{n (n - 1)} & = R (n - 1) \\ T_{i j} & = 0 otherwise \end{cases} \end{matrix}

and the condition for equilibrium becomes

\begin{matrix} (A.6) & R (n - 1) p_{n - 1}^{eq} + M (n + 1) p_{n + 1}^{eq} = (R + M) n p_{n}^{eq}, with n ⩾ 1 and p_{1} = 0 \end{matrix}

It follows by induction that in this example the only possible equilibrium state is given by

\begin{matrix} (A.7) & p_{n}^{eq} = {\begin{cases} 1 & if n = 0 \\ 0 & if n ⩾ 1 \end{cases} \end{matrix}

i.e. we have equilibrium if and only if all bacteria are dead.

A. 2 Properties of the Master Equation

We may rewrite the master equation (A.1) as

\begin{matrix} (A.8) & \frac{d p_{i} (t)}{d t} = \sum_{j} X_{i j} p_{j} (t) \end{matrix}

where

\begin{matrix} (A.9) & X_{i j} = {\begin{cases} T_{i j} & if i \neq j \\ - \sum_{k \neq i} T_{k i} & if i = j \end{cases} \end{matrix}

We immediately find that

X_{i j} ⩾ 0

for all

i \neq j

and

X_{i i} ⩽ 0

for all

i

. We can obtain

X_{i i} < 0

if we assume that for each

i

there is at least one state

j

with nonzero transition amplitude

T_{i j}

. We make this assumption from now on. The formal solution of (A.8) is given by the following matrix exponential:

\begin{matrix} (A.10) & \underset{―}{p} (t) = e^{t X} \underset{―}{p} (0) \underset{―}{p} (t) = (p_{1} (t), \dots, p_{N} (t)) . \end{matrix}

(We also assume that the total number

N

of states is finite).
We would now like to understand whether there must always exist an equilibrium state, and if so, how it is approached. An equilibrium distribution must satisfy

0 =

\sum_{j} X_{i j} p_{j}^{eq}

, which is possible if and only if the matrix

X

has a zero eigenvalue. Thus, we must have some information about the eigenvalues of

X

. We note that this matrix need
not be symmetric, so its eigenvalues,

E

, need not be real, and we are not necessarily able to diagonalize it! Nevertheless, it turns out that the master equation gives us a sufficient amount of information to understand the key features of the eigenvalue distribution. If we define the evolution matrix

A (t)

\begin{matrix} (A.11) & A (t) := e^{t X} \end{matrix}

then, since

A (t)

maps element-wise positive vectors

\underset{―}{p} = (p_{1}, \dots, p_{N})

to vectors with the same property, it easily follows that

A_{i j} (1) ⩾ 0

for all

i, j

. Hence, by the PerronFrobenius theorem, the eigenvector

\underset{―}{v}

A (1)

whose eigenvalue

λ_{max}

has the largest real part must be element wise positive,

v_{i} ⩾ 0

for all

i

, and

λ_{max}

must be real and positive,

\begin{matrix} (A.12) & A (1) \underset{―}{v} = λ_{max \underset{―}{v}}, λ_{max} > 0 . \end{matrix}

This (up to a rescaling) unique vector

\underset{―}{v}

must also be an eigenvector of

X

, with real eigenvalue

\log λ_{max} = E_{max}

. We next show that any eigenvalue

E

X

(possibly

\in C

) has

Re (E) ⩽ 0

by arguing as follows: Let

\underset{―}{w}

be an eigenvector of

X

with eigenvalue

E

, i.e.

X \underset{―}{w} = E \underset{―}{w}

. Then

\begin{matrix} (A.13) & \sum_{j \neq i} X_{i j} w_{j} = (E - X_{i i}) w_{i} \end{matrix}

and therefore

\begin{matrix} (A.14) & \sum_{j \neq i} X_{i j} | w_{j} | ⩾ | E - X_{i i} | | w_{i} |, \end{matrix}

which follows from the triangle inequality and

X_{i j} ⩾ 0

for

i \neq j

. Taking the sum

\sum_{i}

and using (A.9) then yields

\sum_{i} (X_{i i} + | E - X_{i i} |) | w_{i} | ⩽ 0

and therefore

(X_{i i} + | E - X_{i i} |) | w_{i} | ⩽ 0

for at least one

i

. Since

X_{i i} < 0

, this is impossible unless

Re (E) ⩽ 0

. Then it follows that

E_{max} ⩽ 0

and then also

λ_{max} ⩽ 1

. We would now like to argue that

E_{max} = 0

, in fact. Assume on the contrary

E_{max} < 0

. Then

\underset{―}{v} (t) = A (t) \underset{―}{v} = e^{t E_{max}} \underset{―}{v} \to 0,

which is impossible as evolution preserves

\sum_{i} v_{i} (t) > 0

. From this we conclude that

E_{max} = 0

, or

X \underset{―}{v} = 0

, and thus

\begin{matrix} (A.15) & p_{j}^{eq} = \frac{v_{j}}{\sum_{i} v_{i}} \end{matrix}

is an equilibrium distribution. This equilibrium distribution is unique (from the PerronFrobenius theorem). Since any other eigenvalue

E

X

must have

Re (E) < 0

, any distribution

{p_{i} (t)}

must approach this equilibrium state. We summarize our findings:

There exists a unique equilibrium distribution ${p_{j}^{eq}}$ .
Any distribution ${p_{i} (t)}$ obeying the master equation must approach equilibrium as $| p_{j} (t) - p_{j}^{eq} | = O (e^{- t / τ_{relax}})$ for all states $j$ , where the relaxation timescale is given by $τ_{relax} = - 1 / E_{1}$ , where $E_{1} < 0$ is largest non-zero eigenvalue of $X$ .

In statistical mechanics, one often has

\begin{matrix} (A.16) & T_{i j} e^{- β ϵ_{j}} = T_{j i} e^{- β ϵ_{i}}, \end{matrix}

where

ϵ_{i}

is the energy of the state

i

. Equation (A.16) is called the detailed balance condition. It is easy to see that it implies

p_{i}^{eq} = e^{- β ϵ_{i}} / Z

Thus, in this case, the unique equilibrium distribution is the canonical ensemble, which was motivated already in chapter 4.

If the detailed balance condition is fulfilled, we may pass from

X_{i j}

, which need not be symmetric, to a symmetric (hence diagonalizable) matrix by a change of the basis as follows. If we set

q_{i} (t) = p_{i} (t) e^{\frac{β E_{i}}{2}}

, we get

\begin{matrix} (A.17) & \frac{d q_{i} (t)}{d t} = \sum_{j = 1}^{N} {\tilde{X}}_{i j} q_{j} (t) \end{matrix}

where

{\tilde{X}}_{i j} = e^{\frac{β ϵ_{i}}{2}} X_{i j} e^{\frac{- β ϵ_{j}}{2}}

is now symmetric. We can diagonalize it with real eigenvalues

λ_{n} ⩽ 0

and real eigenvectors

{\underset{―}{w}}^{(n)}

, so that

\tilde{X} {\underset{―}{w}}^{(n)} = λ_{n} {\underset{―}{w}}^{(n)}

. The eigenvalue

λ_{0} = 0

again corresponds to equilibrium and

w_{i}^{(0)} \propto e^{- β ϵ_{i} / 2}

. Then we can write

\begin{matrix} (A.18) & p_{i} (t) = p_{i}^{eq} + e^{- \frac{β ϵ_{i}}{2}} \sum_{n ⩾ 1} c_{n} e^{t λ_{n}} w_{i}^{(n)} \end{matrix}

where

c_{n} = q (0) \cdot {\underset{―}{w}}^{(n)}

are the Fourier coefficients. We see again that

p_{i} (t)

converges to the equilibrium state exponentially with relaxation time

- \frac{1}{λ_{1}} < \infty

, where

λ_{1} < 0

is the largest non-zero eigenvalue of

\tilde{X}

A. 3 Relaxation time vs. ergodic time

We come back to the question why one never observes in practice that a macroscopically large system returns to its initial state. We discuss this in a toy model consisting of

N

spins. A state of the system is described by a configuration

C

of spins:

\begin{matrix} (A.19) & C = (σ_{1}, \dots, σ_{N}) \in {+ 1, - 1}^{N} . \end{matrix}

The system has

2^{N}

possible states

C

, and we let

p_{C} (t)

be the probability that the system is in the state

C

at time

t

. Furthermore, let

τ_{0}

be the time scale for one update of the system, i.e. a spin flip occurs with probability

\frac{d t}{τ_{0}}

during the time interval

[t, t + d t]

. We assume that all spin flips are equally likely in our model. This leads to a master equation (A.1) of the form

\begin{matrix} (A.20) & \frac{d p_{C} (t)}{d t} = \frac{1}{τ_{0}} {\frac{1}{N} \sum_{i = 1}^{N} p_{C_{i}} (t) - p_{C} (t)} = \sum_{C^{'}} X_{C C^{'}} p_{C^{'}} (t) \end{matrix}

Here, the first term in the brackets

{\dots}

describes the increase in probability due to a change

C_{i} \to C

, where

C_{i}

differs from

C

by flipping the

i^{th}

spin. This change occurs with probability

\frac{1}{N}

per time

τ_{0}

. The second term in the brackets

{\dots}

describes the decrease in probability due to the change

C \to C_{i}

for any

i

. It can be checked from definition of

X

that

\begin{matrix} (A.21) & \sum_{C} X_{C C^{'}} = 0 \Rightarrow \sum_{C} p_{C} (t) = 1 \forall t . \end{matrix}

Furthermore it can be checked that the equilibrium configuration is given by

\begin{matrix} (A.22) & p_{C}^{eq} = \frac{1}{2^{N}} \forall C \in {- 1, + 1}^{N} \end{matrix}

Indeed:

\sum_{C^{'}} X_{C C^{'}} p_{C^{'}}^{eq} = 0

, so in the equilibrium distribution, all states

C

are equally likely for this model.

If we now imagine a discretized version of the process, where at each time step one randomly chosen spin is flipped, then the timescale over which the system returns to the initial condition is estimated by

τ_{ergodic} \approx 2^{N} τ_{0}

since we have to visit

O (2^{N})

sites before returning and each step takes time

τ_{0}

. We claim that this is much larger than the relaxation timescale. To estimate the latter, we choose an arbitrary but fixed spin, say the first spin. Then we define

p_{\pm} = ⟨ δ (σ_{1} \mp 1) ⟩

, where the time-dependent average is calculated with respect to the distribution

{p_{C} (t)}

, in other words

\begin{matrix} (A.23) & p_{\pm} (t) = \sum_{C : σ_{1} = \pm 1} p_{C} (t) = probability for finding the 1^{st} spin up/down at time t \end{matrix}

The master equation implies an evolution equation for

p_{+}

(and similarly

p_{-}

), which is
obtained by simply summing (A.20) subject to the condition

\sum_{C : σ_{1} = \pm 1}

. This gives:

\begin{matrix} (A.24) & \frac{d p_{+}}{d t} = \frac{1}{τ_{0}} {\frac{1}{N} (1 - p_{+}) - \frac{1}{N} p_{+}}, \end{matrix}

which has the solution

\begin{matrix} (A.25) & p_{+} (t) = \frac{1}{2} + (p_{+} (0) - \frac{1}{2}) e^{- \frac{2 t}{N τ_{0}}} \end{matrix}

So for

t \to \infty

, we have

p_{+} (t) \to \frac{1}{2}

at an exponential rate. This means

\frac{1}{2}

is the equilibrium value of

p_{+}

. Since this holds for any chosen spin, we expect that the relaxation time towards equilibrium is

τ_{relax} \approx \frac{N}{2} τ_{0}

and we see

\begin{matrix} (A.26) & τ_{ergodic} ≫ τ_{relax} . \end{matrix}

A more precise analysis of relaxation time involves finding the eigenvalues of the

2^{N_{-}}

dimensional matrix

X_{C C^{'}}

: we think of the eigenvectors

{\underset{―}{u}}_{0}, {\underset{―}{u}}_{1}, {\underset{―}{u}}_{2}, \dots

with eigenvalues

λ_{0} = 0, λ_{1}, λ_{2}, \dots

as functions

u_{0} (C), u_{1} (C), \dots

where

C = (σ_{1}, \dots, σ_{N})

. Then the eigenvalue equation is

\begin{matrix} (A.27) & \sum_{C^{'}} X_{C C^{'}} u_{n} (C^{'}) = λ_{n} u_{n} (C) \end{matrix}

and we have

\begin{matrix} (A.28) & u_{0} (C) \equiv u_{0} (σ_{1}, \dots, σ_{N}) = p_{C}^{eq} = \frac{1}{2^{N}} \forall C \end{matrix}

Now we define the next

N

eigenvectors

u_{1}^{j}, j = 1, \dots, N

\begin{matrix} (A.29) & u_{1}^{j} (σ_{1}, \dots, σ_{N}) = {\begin{cases} α & if σ_{j} = + 1 \\ β & if σ_{j} = - 1 \end{cases} \end{matrix}

Imposing the eigenvalue equation gives

α = - β

, and then

λ_{1} = - \frac{2}{N}

. The eigenvectors are orthogonal to each other. The next set of eigenvectors

u_{2}^{i j}, 1 ⩽ i < j ⩽ N

\begin{matrix} (A.30) & u_{2}^{i j} (σ_{1}, \dots, σ_{N}) = {\begin{cases} α & if σ_{i} = 1, σ_{j} = 1 \\ - α & if σ_{i} = 1, σ_{j} = - 1 \\ - α & if σ_{i} = - 1, σ_{j} = 1 \\ α & if σ_{i} = - 1, σ_{j} = - 1 \end{cases} \end{matrix}

The vectors

u_{2}^{i j}

are again found to be orthogonal, with the eigenvalue

λ_{2} = - \frac{4}{N}

. The subsequent vectors are constructed in the same fashion, and we find

λ_{k} = - \frac{2 k}{N}

for the

k

-th set. The general solution of the master equation is given by (A.10)

\begin{matrix} (A.31) & p_{C} (t) = \sum_{C^{'}} {(e^{t X})}_{C C^{'}} p_{C^{'}} (0) \end{matrix}

which we can now evaluate using our eigenvectors. If we write

p_{C} (t) = p_{C}^{eq} + \sum_{k = 1}^{N} \sum_{1 ⩽ i_{1} < \dots < i_{k} ⩽ N} a_{i_{1} \dots i_{k}} (t) u_{k}^{i_{1} \dots i_{k}} (C)

we get

a_{i_{1} \dots i_{k}} (t) = a_{i_{1} \dots i_{k}} (0) e^{- 2 k t / (N τ_{0})}

This gives the relaxation time for a general distribution. We see that the relaxation time is given by the exponential with the smallest decay (the term with

k = 1

in the sum), leading to the relaxation time

τ_{relax} = N τ_{0} / 2

already guessed before. This is exponentially small compared to the ergodic time! For

N = 1

mol we have, approximately

\begin{matrix} (A.32) & \frac{τ_{ergodic}}{τ_{relax}} = O (e^{(10^{23})}) \end{matrix}

A. 4 Monte Carlo methods and Metropolis algorithm

The Metropolis algorithm is based in an essential way on the fact that

τ_{relax} ≪ τ_{ergodic}

for typical systems. The general aim of the algorithm is to efficiently compute expectation values of the form

\begin{matrix} (A.33) & ⟨ F ⟩ = \sum_{C} F (C) \frac{e^{- β E (C)}}{Z (β)} \end{matrix}

where

E (C)

is the energy of the state

C

and

F

is some observable. A good example to have in mind is the Ising model on a

d

-dimensional square lattice, where the energy is given by:

E (C) = - J \sum_{bonds i j} σ_{i} σ_{j} + b \sum_{sites i} σ_{i},

and

F = σ_{i}

F = σ_{i} σ_{j}

. Here, a configuration is a set of spins

C = {σ_{1}, \dots, σ_{N}}

. In this example, as well as in most other models of statistical physics, the number of configurations scales exponentially with the system size; in the present example this number would be

2^{N}

. Furthermore, except for a rather special class of models, it seems impossible to do the sum in closed form. This happens for the Ising model for all dimensions

d ⩾ 3

. Then, already for a cubic lattice of very modest side-length such as 10 , we are faced with

2^{1000}

configurations, meaning that it is utterly out of question to do this sum by simply adding all terms up on a computer.

If we have to evaluate numerically an integral of the form

I = \int_{0}^{1} g (x) d x

then if

g (x)

is sufficiently regular and varies on a scale of order 1 , we can do the integral e.g. by generating a sample

X = {x_{1}, \dots, x_{m}}

, chosen according to the uniform probability distribution on

[0, 1]

(the latter means that we have no prior idea of what

g (x)

looks like. Then we expect that

I \approx m^{- 1} \sum_{x_{i} \in X} g (x_{i}),

already for a relatively small number

m

of points. However, this will fail e.g. if

g (x)

is very sharply peaked near some point

x_{0}

, say with peak width

10^{- 1000}

. It is clear that generically, none of the randomly chosen points

X = {x_{1}, \dots, x_{m}}

will hit the peak, unless we take

m \approx 10^{1000}

, which is out of the question. Roughly speaking, we typically run into the same kind of problem when evaluating the state sum in statistical physics.

Again, a simple minded method would be to simply generate a uniformly distributed sample

{\tilde{C}}_{1}, \dots, {\tilde{C}}_{u}

where

u ≫ 1

and to approximate

⟨ F ⟩ \approx \sum_{i = 1}^{u} F ({\tilde{C}}_{i}) \frac{e^{- β E ({\tilde{C}}_{i})}}{Z}

. But this is a very bad idea in most cases since the fraction of configurations out of which the quantity

e^{- β E ({\tilde{C}}_{i})}

is not practically 0 is exponentially small. The idea is instead to generate a sample

C_{1}, \dots, C_{m}

of configurations distributed according to

\propto e^{- β E (C)}

. But how to get such samples? We choose any (!)

T_{C, C^{'}}

satisfying the detailed balance condition (A.16):

\begin{matrix} (A.34) & T_{C, C^{'}} e^{- β E (C^{'})} = T_{C^{'}, C} e^{- β E (C)} . \end{matrix}

Then, according to the above discussion, we expect that an initial distribution

p_{C} (t)

will reach the equilibrium configuration

p_{C}^{eq} = e^{- β E (C)} / Z

after about

N

time-steps, where

N

is the system size. If we chose transition amplitudes such that also

\sum_{C^{'}} T_{C^{'}, C} = 1

for all

C

, the discretized version of the master equation then becomes

p_{C^{'}} (t + 1) = \sum_{C} T_{C^{'}, C} p_{C} (t)

In the simplest case, the sum is over all configurations

C

differing from

C^{'}

by flipping precisely one spin. If

C_{i}^{'}

is the configuration obtained from some configuration

C^{'}

by flipping spin

i

, we therefore assume that

T_{C^{'}, C}

is non-zero only if

C = C_{i}^{'}

for some

i

Stating the algorithm in a slightly different way, we can say that, for a given configuration, we accept the change

C \to C^{'}

randomly with probability

T_{C^{'}, C}

. A very simple and practical choice for the acceptance probability (in other words

T_{C^{'}, C}

) satisfying our
conditions is given by

\begin{matrix} (A.35) & p_{accept} = {\begin{cases} 1 & if E (C^{'}) ⩽ E (C) \\ e^{- β [E (C^{'}) - E (C)]} & if E (C^{'}) ⩾ E (C) \end{cases} \end{matrix}

We may then summarize the algorithm as follows:

Metropolis Algorithm

(1) Choose an initial configuration

C

.
(2) Choose randomly a spin

i

and determine the change in energy

E (C) - E (C_{i}) = δ_{i} E

for the new configuration

C_{i}

obtained by flipping one spin

i

.
(3) Choose a uniformly distributed random number

u \in [0, 1]

. If

u < e^{- β δ_{i} E}

, change

σ_{i} \to - σ_{i}

, otherwise leave

σ_{i}

unchanged.
(4) Rename

C_{i} \to C

.
(5) Go back to (2).

Running the algorithm

m

times, going through approximately

N

iterations each time, gives the desired sample

C_{1}, \dots, C_{m}

distributed approximately according to

e^{- β E (C)} / Z

. The expectation value

< F >

is then computed as the average of

F (C)

over the sample

C_{1}, \dots, C_{m}

. An important practical point is that the change in energy if we flip one spin is very easy to calculate in the example of the Ising model because the interaction is local (i.e. we have to compute only

2 d

terms associated with the nearest neighbors of

i

), as it is in most models.

A. 5 Eigenstate thermalization

[In this section, we shall be following Srednicki: J Phys A 32 (1999) 1163-1175, Sections 2 and 3.] We have seen previously that in classical systems, the ensemble average of an observables equals the time-average under typical conditions. This can be seen as a kind of statement about equilibration. For quantum systems showing quantum chaos one can give another argument why the system will equilibrate no matter what its initial state was. It does not rely on an incomplete knowledge of the dynamics (as discussed in chapter 3.2) but rather on the specific structure of observables in such systems. Assume that dynamics is ruled by the time-independent (exact) Hamiltonian

\hat{H}

with nondegenerate eigenvalues

E_{n}

and eigenstates

| n ⟩

. Then, in the energy eigenbasis, the diagonal matrix elements of an observable

A

can be expressed in terms of a function in one variable

E_{n}

and the off-diagonal matrix elements can be expressed in terms of a function in two
variables

E_{n}

and

E_{m}

, or in terms of their sum and difference

E_{n m} = \frac{E_{n} + E_{m}}{2}, ω_{n m} = \frac{E_{n} - E_{m}}{2} .

Thus, one can make the ansatz

\begin{matrix} (A.36) & A_{n m} = α (E_{n}) δ_{n m} + e^{- \frac{S (E_{n m)}}{2}} f (E_{n m}, ω_{n m}) R_{n m} \end{matrix}

where

R_{n m}

is some complex matrix,

α

and

f

are smooth real-valued functions, and where

S (E) = \log (E \sum_{n} h (E - E_{n}))

with a nonnegative smooth function

h

having unit integral and chosen so that

S

is monotonic. Calculation of the expectation value of

A

in the canonical ensemble at temperature

T

yields

⟨ A ⟩_{ens} = \frac{tr (A e^{- β \hat{H}})}{tr (e^{- β \hat{H}})} = \frac{\int_{0}^{\infty} \frac{d E}{E} α (E) e^{S (E) - β E}}{\int_{0}^{\infty} \frac{d E}{E} e^{S (E) - β E}} + O (e^{- S / 2}),

where in the last term,

S

stands for the entropy of the ensemble. Extending this interpretation to the function

S

in the integrals and using that the entropy is an extensive quantity, one may evaluate the integrals by the method of stationary phase (see eg. Appendix A. 2 in the script on quantum mechanics). This leads to

\begin{matrix} (A.37) & ⟨ A ⟩_{ens} = α (⟨ \hat{H} ⟩_{ens}) + O (N^{- 1}) + O (e^{- S / 2}), \end{matrix}

where

T, S

and

E

are related by

E = ⟨ \hat{H} ⟩_{ens}

and the usual relationship

T = {({\frac{\partial S}{\partial E} |}_{E})}^{- 1}

On the other hand, for the time average

⟨ A ⟩_{Ψ, time}

of the expectation value

⟨ Ψ_{t} ∣ A Ψ_{t} ⟩

in some given state

Ψ

, we find

\begin{matrix} (A.38) & ⟨ A ⟩_{Ψ, time} = lim_{τ \to \infty} \frac{1}{τ} \int_{0}^{τ} ⟨ Ψ_{t} | A | Ψ_{t} ⟩ d t = \sum_{n} {| γ_{n} |}^{2} A_{n n} \end{matrix}

where

γ_{n}

are the expansion coefficients of

Ψ (0)

relative to the energy eigenbasis,

Ψ (0) = \sum_{n} γ_{n} | n ⟩ .

In view of (A.36), this yields

\begin{matrix} (A.39) & ⟨ A ⟩_{Ψ, time} = \sum_{n} {| γ_{n} |}^{2} α (E_{n}) + O (e^{- S / 2}) . \end{matrix}

For a state of a macroscopic system that could be realistically prepared in a lab, the uncertainty of

\hat{H}

in the state

Ψ

typically satisfies

Δ_{Ψ} (\hat{H}) \sim \frac{⟨ \hat{H} ⟩_{Ψ}}{\sqrt{N}}

so it is small for large

N

. Thus, expanding

α

in (A.39) about

⟨ \hat{H} ⟩_{Ψ, time}

to second order, we obtain the approximation

\begin{aligned} ⟨ A ⟩_{Ψ, time} & = α (⟨ \hat{H} ⟩_{Ψ}) + \frac{1}{2} α^{''} (⟨ \hat{H} ⟩_{Ψ}) Δ_{Ψ}^{2} (\hat{H}) + \dots + O (e^{- S / 2}) \\ = α (⟨ \hat{H} ⟩_{Ψ}) + O (Δ_{Ψ}^{2} (\hat{H})) + O (e^{- S / 2}) \end{aligned}

Combining this with the approximation for the ensemble average of

A

given by (A.37) and choosing

T

so that

⟨ \hat{H} ⟩_{ens} = ⟨ \hat{H} ⟩_{Ψ}

, we finally obtain

⟨ A ⟩_{Ψ, time} = ⟨ A ⟩_{ens} + O (Δ_{Ψ}^{2} (\hat{H})) + O (N^{- 1}) + O (e^{- S / 2}) .

As a result, the time average of the expectation value of

A

in a realistically preparable state is approximately equal to the ensemble average of

A

at the appropriate temperature. Therefore, no matter what (realistic) state the system is prepared in, it will always equilibrate.

Appendix B

Exercises

B. 1 Exercises for chapter 2

Problem B.1. [Random walk] Let

w (x) d x

be an arbitrary probability distribution describing the probability for finding a real random variable

X

in the 'interval'

[x, x + d x]

. Let the mean and spread be defined as usual as

μ = \int x w (x) d x, σ = {(\int (x - μ)^{2} w (x) d x)}^{\frac{1}{2}} .

Consider now a random walk on the real axis

R

with increment/decrement

X_{i}

at step

i

. The mean distance covered from the origin after

N

steps is

Y = \frac{1}{N} (X_{1} + \dots + X_{N})

. The aim is to show that, for large

N, Y

has approximately a Gaussian probability distribution, with mean

μ

and spread

σ / \sqrt{N}

.
a) Introduce the variable

Z = \sum (X_{i} - μ) / \sqrt{N}

and demonstrate that its probability distribution

w_{Z} (z) d z

is given by

w_{Z} (z) = \int \frac{d k}{2 π} e^{i k z} \int d x_{1} \dots d x_{N} \prod w (x_{i}) \exp (- i k (x_{1} + \dots + x_{N}) / \sqrt{N} + i k μ \sqrt{N})

b) Introduce the 'characteristic function'

χ (q) = \tilde{w} (q) = \int d x w (x) e^{- i q x},

where

\tilde{w} (q)

is the Fourier transform of

w (x)

, and write

w_{Z} (z)

in terms of it.
c) Show that the first terms in the expansion

\log χ (q) = Σ {⟨ x^{n} ⟩}_{c} (- i q)^{n} / n

! are given by the cumulants

⟨ x ⟩_{c} = μ, {⟨ x^{2} ⟩}_{c} = σ^{2}

. Substitute this into (b) and show that the result may be written as

w_{Z} (z) = \int \frac{d k}{2 π} e^{i k z - \frac{1}{2} (σ k)^{2} + \dots}

where ... stand for terms going to zero as

1 / \sqrt{N}

or faster for large

N

.
d) Deduce that for large

N

one has

w_{Z} (z) \to \frac{1}{\sqrt{2 π σ}} \exp (- z^{2} / (2 σ^{2}))

. Relating this to the distribution of

Y

, get the desired result.
e) What is the wider significance of the result beyond a random walk on

R

Problem B. 2 (Entropy maximization 1). A system has

N

states occupied with probabilities

p_{n}, n = 0, 1, \dots, N

. The

n

-th state has energy

E_{n} = n

. The average energy,

U = \sum_{n} E_{n} p_{n}

, is assumed to be given.
a) Show that the entropy is maximized by a distribution of the form

p_{n} = e^{- β n} / Z

.
b) In the case

N = 2

, work out the explicit form of

β, Z

in terms of

U

.
c) Suppose now that the standard deviation,

\begin{matrix} (B.1) & Δ U = \sqrt{(\sum_{n} E_{n}^{2} p_{n}) - U^{2}} \end{matrix}

is also known. What is the form of the distribution maximizing the entropy in this case for general

N

Your answers should indicate why the distribution is an actual maximum of the entropy and not just a stationary point.

Problem B. 3 (Entropy maximization 2). Repeatedly throwing an ideal dice will evidently yield the average result of 3.5 . However, it is found for an-obviously manipulateddice that the average is instead 4. In the absence of further information, what is the probability distribution, to within 3 significant figures, assigned to this dice?

Hint: Maximize the information entropy. You may use a computer (e.g. MATHEMATICA) to help you with any equation that you need to solve numerically.

Problem B. 4 (Information entropy). This problem motivates the definition of information entropy. Consider an experiment with

N

possible outcomes that occur randomly with probabilities

p_{1}, \dots, p_{N}

. (Think of throwing a dice, where

N = 6, p_{i} = \frac{1}{6}

.) To determine which outcome

O

has occurred, we allow ourselves yes/no questions of the type: Is

O \in S

? where

S

is some subset of

{1, \dots, N}

. For example, for

S = {i}

, the corresponding question would be: Is

O \in {i}

, i.e. has event

i

occurred? Or, if

S = {1, 3}

, the question is: Has event 1 or event 3 occurred?
a) Consider the following question strategy to find out what

O

was: We first ask: Has outcome 1 occurred? If yes, we are done, if no, we ask: Has outcome 2 occurred? and so on. What is the maximum number of questions needed to determine

O

in this strategy? What is the average number of questions needed in this strategy?
b) Let

I = - \sum_{i} p_{i} \log_{2} p_{i}

be the information entropy. Show that in any strategy, the average number of questions needed to determine

O

⩾ I

.
c) Verify that this is indeed the case for strategy a), applied to a dice. For a dice, suggest a strategy which requires fewer questions on average than a).
d)* Show that there always exists a strategy such that the average number of questions needed is

⩽ I + 1 . b

) and d) show that

I

is an estimate of the average number of questions needed to find out what the outcome was.

Problem B.5. [Ising spin chain] We consider the 1-dimensional Ising spin chain with periodic boundary conditions. In this model, we have

n

spins

σ_{1}, \dots, σ_{n} \in {\pm 1}

, and the energy of a configuration is given by

H = - J (σ_{1} σ_{2} + σ_{2} σ_{3} + \dots + σ_{n - 1} σ_{n} + σ_{n} σ_{1})

where

J > 0

. The probability distribution in the canonical ensemble is

P ({σ_{j}}) = \frac{1}{Z} \exp (- β H ({σ_{j}})) .

a) Draw a picture of the spin chain for a configuration minimizing/maximizing

H

.
b) Show that

Z = \sum_{{σ_{j}}} \exp (- β H ({σ_{j}}))

.
c) Show that

Z

can be written alternatively as

Z = \sum_{σ_{1} = \pm 1} \dots \sum_{σ_{n} = \pm 1} T_{σ_{1} σ_{2}} T_{σ_{2} σ_{3}} \dots T_{σ_{n - 1} σ_{n}} T_{σ_{n} σ_{1}} = tr T^{n}

with the "transfer matrix"

T_{σ σ^{'}} = e^{β J σ σ^{'}}

. In matrix form,

T = (\begin{array}{cc} T_{+ +} & T_{+ -} \\ T_{- +} & T_{+ +} \end{array}) = (\begin{array}{cc} e^{β J} & e^{- β J} \\ e^{- β J} & e^{β J} \end{array})

Hint: write out

H

and the multiple sums in

Z = \sum_{{σ_{j}}} \exp (- β H ({σ_{j}}))

.
d) Show that

T

has eigenvalues

λ_{1} = 2 \cosh (J β), λ_{2} = 2 \sinh (J β)

. (This means that we can diagonalize

T

, i.e. we have

T = U D U^{†}

, where

D = diag (λ_{1}, λ_{2})

and

U

is a unitary matrix.) Use this and

Z = tr T^{n}

to show that

Z = 2^{n} [(\cosh (β J))^{n} + (\sinh (β J))^{n}] .

Note that you do not need to compute

U

(explain why!).

Problem B. 6 (Initial conditions).

1 {mm}^{3}

of a gas at normal pressure and temperature contains about

10^{15}

particles. Considering the particles as point-like and classical, provide a rough, conservative estimate for how many hard drives would be necessary to store the initial conditions of all gas particles. (As of 2013, a normal hard drive can store about 5 TB of data.)

Problem B. 7 (Time evolution of ensemble averages).
a) Let

(P, Q) \equiv ({\vec{p}}_{1}, {\vec{q}}_{1}, \dots, {\vec{p}}_{N}, {\vec{q}}_{N}) \in R^{6 N}

be a point in the phase space of

N

particles, and let the dynamical law be given through a time-independent Hamiltonian

H (P, Q)

, which defines the trajectories

(P (t), Q (t))

via Hamilton's equations. Let

Φ_{t} (P, Q) := (P (t), Q (t))

with initial condition

(P (0), Q (0)) = (P, Q)

. Show that for any observable

O (P, Q)

, the 'time-translated' observable

O_{t} (P, Q) = O [Φ_{t} (P, Q)]

have the same expectation value,

⟨ O ⟩ = ⟨ O_{t} ⟩

for all

t

, where the probability distribution describing the ensemble has constant value on each energy surface.
b) Is the phase space for

N

particles always of the form

R^{6 N}

? Hint: Think e.g. of a gas consisting of molecules.

Problem B. 8 (Phase space density). The purpose of this problem is to explain why the phase space density for an equilibrium ensemble must generically be a function of

E

alone.
a) Show that the "classical trace" of a rapidly decaying function

f (P, Q)

on the phase space

R^{6 N}

, defined by

tr (f) = \int f (P, Q) d^{3 N} Q d^{3 N} P

has the properties

tr {f, g} = 0, (tr (f))^{*} = tr (f^{*}), tr (f^{*} f) ⩾ 0

Hint: Use the results of problem B.7.
b) Let

ρ (P, Q)

be a classical phase space distribution,

O (P, Q)

an observable, and

⟨ O ⟩ = tr (O ρ)

, with the "classical trace" defined in a). Show that if

ρ = ρ (I_{0}, \dots, I_{N})

is a function of conserved quantities

I_{i}

of the system (which include at least

I_{0} = H

), then the ensemble defined by

ρ

is stationary, i.e.

⟨ O_{t} ⟩ = ⟨ O ⟩

for all

t

and all

O

. Is the converse statement also true? What are the conserved quantities for a generic Hamiltonian of the form

H (P, Q) = \sum_{i} \frac{{| {\vec{p}}_{i} |}^{2}}{2 m} + \sum_{i < j} V (| {\vec{x}}_{i} - {\vec{x}}_{j} |) + \sum_{i} W ({\vec{x}}_{i})

(note that

W

can be used to describe the "walls" of a box.) What if

W = 0

? What if

W ({\vec{x}}_{i}) = w (| {\vec{x}}_{i} |)

, i.e. a spherically symmetric external potential? What if

V = W = 0 ?

Problem B. 9 (Density matrices).
a) Verify the following elementary properties of the trace of matrices:

tr [A, B] = 0, (tr A)^{*} = tr (A^{†}), tr (A^{†} A) ⩾ 0

b) Let

ρ, σ

be density matrices. Show that for any nonnegative real numbers

p, q

satisfying

p + q = 1

, the matrix

p ρ + q σ

has the properties of a density matrix.

Problem B. 10 (Entanglement entropy). Ignoring all degrees of freedom other than spin, the Hilbert space of a single neutron is

H = C^{2}

, with spin-operators

σ_{x} = (\begin{array}{cc} 0 & 1 \\ 1 & 0 \end{array}), σ_{y} = (\begin{array}{cc} 0 & - i \\ i & 0 \end{array}), σ_{z} = (\begin{array}{cc} 1 & 0 \\ 0 & - 1 \end{array})

a) The neutron is aligned with probability

p

in the

+ z

direction and with probability

1 - p

in the

+ x

direction. What is the density matrix describing this ensemble? What are its eigenvalues?
b) Now consider two neutrons in the normalized state

| ψ ⟩ = α | ↑↑ ⟩ + β | ↓↓ ⟩

, where

| ↑ ⟩

resp.

| ↓ ⟩

are the normalized eigenstates for the

+ z

resp.

- z

direction and

| ↑↑ ⟩ = | ↑ ⟩ \otimes | ↑ ⟩

etc. What is the reduced density matrix for the first neutron? When is the corresponding entanglement entropy maximal/minimal?
c) Assume now that the two neutron system is in an eigenstate

| χ ⟩

with zero total spin in the

z

-direction. What is the reduced density matrix for the first neutron? Show that if

| χ ⟩

is either symmetric or anti-symmetric, then the entanglement entropy is maximized.
d) Can an arbitrary density matrix

ρ_{1}

for the first neutron arise as the reduced density matrix of a suitable (pure) state of the system with two neutrons?

B. 2 Exercises for chapter 3

Problem B. 11 (Boltzmann's H-theorem). Let

f (\vec{v}, \vec{x}, t)

be the 1-particle probability distribution, expressed in terms of the velocity

\vec{v} = \vec{p} / m

. Define a function

H (t)

H (t) = - \int d^{3} v d^{3} x f (\vec{v}, \vec{x}, t) \log f (\vec{v}, \vec{x}, t)

which is similar in nature to the von Neumann (

\propto

information-) entropy. The aim of this problem is to derive the important consequence

\dot{H} ⩾ 0

of the Boltzmann equation.
a) Explain why the Boltzmann equation can alternatively be written as

\begin{aligned} [\frac{\partial}{\partial t} + \vec{v} \frac{\partial}{\partial \vec{x}} + \frac{1}{m} \vec{F} (\vec{x}) \frac{\partial}{\partial \vec{v}}] f (\vec{x}, \vec{v}, t) \\ = \int d^{3} v_{2} d^{3} v_{3} d^{3} v_{4} W (\vec{v}, {\vec{v}}_{2}, {\vec{v}}_{3}, {\vec{v}}_{4}) [f (\vec{x}, {\vec{v}}_{3}, t) f (\vec{x}, {\vec{v}}_{4}, t) - f (\vec{x}, \vec{v}, t) f (\vec{x}, {\vec{v}}_{2}, t)] \end{aligned}

for some

W > 0

. Hint: "Undo" the momentum- and energy-conservation rule which has already been included in the Boltzmann equation by introducing

δ^{3} (\vec{v} + {\vec{v}}_{1} -

{\vec{v}}_{3} - {\vec{v}}_{4}

) and new integrations etc.
b) Argue physically why the following relations should hold:

\begin{aligned} W ({\vec{v}}_{1}, {\vec{v}}_{2}, {\vec{v}}_{3}, {\vec{v}}_{4}) = W ({\vec{v}}_{2}, {\vec{v}}_{1}, {\vec{v}}_{4}, {\vec{v}}_{3}) \\ W (- {\vec{v}}_{1}, - {\vec{v}}_{2}, - {\vec{v}}_{3}, - {\vec{v}}_{4}) = W ({\vec{v}}_{1}, {\vec{v}}_{2}, {\vec{v}}_{3}, {\vec{v}}_{4}) \\ W ({\vec{v}}_{1}, {\vec{v}}_{2}, {\vec{v}}_{3}, {\vec{v}}_{4}) = W (- {\vec{v}}_{3}, - {\vec{v}}_{4}, - {\vec{v}}_{1}, - {\vec{v}}_{2}) \end{aligned}

Hint: What is the physical meaning of these equations for the collision?
c) Let

H (\vec{x}, t)

be defined as

H (t)

but without the

d^{3} x

-integration. Show that

\dot{H} (\vec{x}, t) = \frac{\partial}{\partial \vec{x}} \int d^{3} v \vec{v} f (\vec{x}, \vec{v}, t) \log f (\vec{x}, \vec{v}, t) + I (\vec{x}, t)

where

I = \int d^{3} v_{1} d^{3} v_{2} d^{3} v_{3} d^{3} v_{4} W_{1234} (f_{1} f_{2} - f_{3} f_{4}) (1 + \log f_{1})

using the shorthand

f_{1} = f (\vec{x}, {\vec{v}}_{1}, t), f_{2} = f (\vec{x}, {\vec{v}}_{2}, t)

, etc.
d) Using b), show that

I

can be written as

I = \frac{1}{4} \int d^{3} v_{1} d^{3} v_{2} d^{3} v_{3} d^{3} v_{4} W_{1234} (f_{1} f_{2} - f_{3} f_{4}) \log \frac{f_{1} f_{2}}{f_{3} f_{4}}

and conclude that

I ⩾ 0

. Using c), show that

\dot{H} ⩾ 0

.
e) What is the physical significance of this result?

Problem B. 12 (Master equation). We consider a time dependent probability distribution

{p_{i} (t)}

described by the master equation:

\frac{d}{d t} p_{i} (t) = \sum_{j : j \neq i} [T_{i j} p_{j} (t) - T_{j i} p_{i} (t)]

Each transition amplitude

T_{i j}

is assumed to be positive. Assume the detailed balance condition:

T_{i j} e^{- β E_{j}} = T_{j i} e^{- β E_{i}}

a) Show that the time-independent distribution

p_{i} = e^{- β E_{i}} / Z

is a (stationary) solution to the master equation.
b) Show that the master equation implies

\sum_{i} p_{i} (t) = 1

for all

t

if this is true at

t = 0

. (Hint: differentiate this sum and use the master equation.) Give an argument why each

p_{i} (t)

has to remain positive for all times if this holds initially. (Hint: consider a

t_{0}

such that

p_{i} (t_{0}) = 0

and use the master equation.)
c) Consider a population of bacteria. Let

n

be the number of bacteria,

M

the mortality rate,

R

the reproduction rate.

p_{n} (t)

is the probability that the population consists of

n

bacteria. We consider the evolution equation

{\dot{p}}_{n} (t) = R (n - 1) p_{n - 1} (t) + M (n + 1) p_{n + 1} (t) - (M + R) n p_{n} (t)

for

n > 0

and

{\dot{p}}_{0} (t) = M p_{1} (t)

for

n = 0

. Show that the equation has the form of a master equation. Derive the possible equilibrium state(s) of this system.

B. 3 Exercises for chapter 4

Problem B. 13 (1-dimensional classical Ising model). The

d

-dimensional Ising model is exactly solvable in

d = 1, 2

, but not beyond. Here we look at the (easier) case when

d = 1

. Consider a 1-dimensional lattice of

N + 1

atoms, each of which is assumed to carry a spin

σ_{i} = \pm 1, i = 0, \dots, N

. The energy of the state described by

{σ_{i}} = (σ_{0}, \dots, σ_{N})

is assumed to be

H ({σ_{i}}) = - J \sum_{i = 1}^{N} σ_{i} σ_{i - 1}

where

J

is a constant which determines the strength of the interaction.
a) Neighboring spins can have equal or opposite signs, in which case they are called "parallel" resp. "anti-parallel". Let

ν

be the number of anti-parallel pairs in

{σ_{i}}

. Express the energy in terms of

ν

. Count the number of states with

ν

anti-parallel pairs. Hence, what is the number

W (E)

of configurations

{σ_{i}}

having

H ({σ_{i}}) = E

?
b) Using the result of a), calculate the canonical partition function

Z (β) = \sum_{{σ_{i}}} e^{- β H ({σ_{i}})}

Hint: Rewrite the sum as a sum over

ν

.
c) Calculate the free energy

F / N

per spin and the entropy per

spin S / N

in the canonical and micro-canonical ensembles for large

N

Problem B. 14 (Heat capacity of a crystal). We study a simplified microscopic model to understand the heat capacity of a crystal. We suppose that the crystal consists of

N

atoms (or ions) arranged in some sort of lattice. The equilibrium position of an atom is at some lattice site, about which it can oscillate. We assume that the oscillations are small and can be described by a harmonic oscillator, and that the individual oscillators are independent, i.e. do not interact with one another (to what extent is the last assumption realistic in practice?). The total Hamiltonian is hence the sum of the Hamiltonians for the individual oscillators:

H = \sum_{i = 1}^{N} H_{i}, H_{i} = \frac{1}{2 m} {\vec{p}}_{i}^{2} + \frac{m}{2} ω^{2} {\vec{x}}_{i}^{2}

where

{\vec{p}}_{i}

is the momentum,

{\vec{x}}_{i}

is the position relative to the equilibrium position, and

m

is the mass of the atom.
a) Einstein model, canonical approach.
i) Describe the eigenstates and eigenvalues (energy levels) of the crystal in terms of those of a single 1-dimensional harmonic oscillator.
ii) Give the quantum canonical partition function

Z_{N}

as a function of the temperature
iii) Deduce the mean energy

U

and the specific heat

C

of the system. Compare this to the specific heat of a paramagnetic (non-interacting) spin chain. What is the behavior of

C

for high/low temperatures? What is the numerical value for

C

at high temperature for a crystal containing

N_{A} = 6.022 \times 10^{23}

atoms? What is the characteristic temperature

T_{0} = ℏ ω / k_{B}

for a value

ℏ ω = 0.1 eV

?
iv) Evaluate also the classical canonical partition function

Z_{N}^{class}

N

distinguishable classical oscillators. Demonstrate that it is comparable to the quantum partition function for

T ≫ T_{0}

. Comment?
b) Micro-canonical approach. Here, one fixes the total energy,

E = (M + \frac{3}{2} N) ℏ ω

, where

M ≫ 1

.
i) What is the number

W (E)

of accessible micro-states of the system?
ii) Deduce the entropy

S (E)

of the system.
iii) Express

E

as a function of the temperature.
iv) Compute

C

and compare the result to that found in part 1.
c) Modified Einstein model. The prediction for the heat capacity as a function of

T

is qualitatively in accord with experiments for high temperatures, but not for low temperatures, where experiments show the behavior

C \sim T^{3}

. In order to have a model which is in accord with that behavior for low

T

, suppose that we have instead

N

oscillators with variable frequencies

ω_{i}, i = 1, \dots, N

between 0 and some maximum

ω_{max}

. The heat capacity is now given by a sum. Approximate this sum by an integral in terms of the frequency distribution function

D (ω)

, defined in such a way that

D (ω) d ω

is the number of atoms with frequency in the range

ω \dots ω + d ω

. Assuming that

D (ω)

behaves as

D (ω) \sim A ω^{ν}

for

ω ≪ 1

, find the correct value of

ν

to reproduce the behavior

C \sim T^{3}

for low

T

Problem B. 15 (Paramagnetism). Consider a system of a large number

N

of spins. Let

s_{i} = \pm 1

be the value of the

i

-th spin in the

z

-direction.
a) In the absence of a magnetic field, all spin configurations

(s_{1}, \dots, s_{N})

are equally probable.
i) What is the probability for a fixed configuration of spins?
ii) What is the probability of finding a configuration with

N_{+}

positive spins (and

N_{-} = N - N_{+}

negative spins)?
iii) Generalize the probability law in 2 to the case when a positive spin occurs with probability

p

and a negative spin with probability

1 - p

. Calculate, in this case, the mean value

⟨ N_{+} ⟩

and the spread

Δ N_{+}

. What is the dependence on

N

for a large system?
b) Now suppose the spins are associated with the electrons sitting at distinct lattice sites of a crystal. The magnetic moment associated with the

i

-th spin is

{\vec{μ}}_{i} = - μ {\vec{σ}}_{i}, μ = \frac{| q | ℏ}{2 m_{e}} = 9.27 \times 10^{- 24} J / T

where

\vec{σ} = (σ_{x}, σ_{y}, σ_{z})

are the Pauli matrices. What is the Hamiltonian in the presence of a constant magnetic field

\vec{B}

in the

z

-direction? What are the maximum and minimum values

E_{max / min}

of the energy of the system interacting with the external magnetic field? Express the degeneracy of a given energy level as a function of

N_{+}, N_{-}

.
c) Entropy:
i) Calculate the entropy of the system given the information that the energy is at the fixed value

E = μ B (N_{+} - N_{-}) .

Express the entropy in terms of

N

and the variable

ϵ = \frac{E}{N μ B}

(energy per spin in units of

μ B

). Sketch

S = S (N, ϵ)

. Verify that the entropy is a concave function of the energy, meaning that

\frac{\partial^{2} S (N, ϵ)}{\partial ϵ^{2}} ⩽ 0 .

(You may recall Stirling's formula which states

\log n! = n \log n - n + O (\log n)

for large

n

.) Why is it plausible that the entropy should be concave?
ii) Suppose the energy of the system is not known exactly, but only up to

Δ U

, where we assume

Δ U ≪ N μ B

. What is the entropy of the system? Is the result significantly different from that in 1 ?
d) Temperature: Recall that the absolute temperature of the system is defined by

T = \frac{1}{\partial S / \partial E}

i) Express

T (E)

as a function of energy for a spin system with

N

spins. Sketch this function, and comment on the behavior of

T (E)

for

E > 0

!
ii) Invert the relation between temperature and energy and obtain the energy as a function

E = E (T, N)

T, N

.
iii) Consider a spin system with positive energy. The spin system is put in thermal contact with an ideal monoatomic gas at temperature

T_{g}

. The energy of the gas is, as usual,

E_{g} = \frac{3}{2} N_{g} k_{B} T_{g}

. Once thermal equilibrium is reached, what can be said about the final temperature

T_{f}

? What are its limits for

N / N_{g} \to \infty

resp.

\to 0

?
e) Curie's law: The magnetization

\vec{M} = (M_{x}, M_{y}, M_{z})

is defined as the average magnetic moment in the spin system. The magnetic susceptibility per volume

V

is defined in the small field limit as

^{1}

χ = lim_{B \to 0} \frac{M}{V \cdot B} .

A substance having

χ > 0

is called paramagnetic, while a substance having

χ < 0

is called diamagnetic. For paramagnetic substances, one finds experimentally that, to a good precision,

χ

is inversely proportional to the absolute temperature. This behavior is called 'Curie's law'. Suppose the spin system is in thermal equilibrium at temperature

T

. Give the magnetic moment

M \equiv M_{z}

as a function of

β, B \equiv B_{z}, N

Deduce the susceptibility and verify Curie's law. Calculate also the heat capacity

C = \partial E / \partial T

and sketch

C

as a function of

B / T

and

T / B

Problem B. 16 (Mean field theory and ferromagnetism). A ferromagnetic material has a spontaneous magnetization below a critical temperature

T_{c}

, even in the absence of an external magnetic field

B

. Above

T_{c}

, the spontaneous magnetization is zero, and the material behaves like a paramagnet. To understand this effect, we study the famous Ising model. In this model, one considers independent spins

σ_{i} = \pm 1

on the sites

i

of a hypercubic lattice

Z^{d}

d

spatial dimensions. The energy of a configuration of spins

{σ_{i}}

is taken to be

H ({σ_{i}}) = - J \sum_{i j} σ_{i} σ_{j} - b \sum_{i} σ_{i}

The first sum is over all lattice bonds, i.e. pairs

(i, j)

with

i < j

such that spin

i

and spin

j

are nearest neighbors. The second sum is over all lattice sites.

b

is related to the background field by

b = μ B

, where

μ

is of the order of the Bohr magneton,

\sim - 9.3 \times 10^{- 24} J / T . J

is the ferromagnetic coupling between the spins. The probability distribution for the spin configurations is

ρ ({σ_{i}}) = \frac{1}{Z} \exp - β H ({σ_{i}})

with the partition function

Z = Z (β, J, b)

.
a) Write down a formula for the partition function.
b) Write down a formula for

ρ (σ_{1}, \dots, σ_{i} = + 1, \dots) / ρ (σ_{1}, \dots, σ_{i} = - 1, \dots)

in terms of

β

and

h_{i}

. Let

p_{\pm}

be the probabilities that the

i

-th spin is

\pm 1

, respectively. Show that the mean magnetization defined as

m = ⟨ σ_{i} ⟩

is independent of

i

and can be written as

m = \frac{p_{+} / p_{-} - 1}{p_{+} / p_{-} + 1}

(The mean value is defined wrt. the probability distribution

ρ

given above.)
c) In the mean field approximation, each individual spin can be thought of as being subject to an "effective" magnetic field

h_{i} = b + J \sum_{bonds containing i} σ_{j}

for a given configuration

{σ_{j}}

of the other spins

j \neq i

. One writes the energy in terms of these fields,

H = - \frac{1}{2} \sum_{i} σ_{i} (h_{i} + b)

and assumes that it is consistent to replace

h_{i}

with its mean value

h := ⟨ h_{i} ⟩

. Assuming the mean field approximation, derive, using a), the "self-consistency" relation

m = \tanh β (J v m + b)

where

v

is the number of nearest neighbors in the lattice, i.e.

v = 2

1 d, v = 4

2 d, v = 6

3 d

, etc.
d) The free energy is given as usual by

F = - k_{B} T \log Z

, where

β^{- 1} = k_{B} T

. Verify that

\begin{matrix} (B.2) & N m = - \frac{\partial F}{\partial b} \end{matrix}

Here

N

is the total number of lattice sites, which we assume to be finite in this part (box). To calculate

Z

, write

σ_{i} = ⟨ σ_{i} ⟩ + δ σ_{i}

, with

δ σ_{i} = σ_{i} - ⟨ σ_{i} ⟩

. Substitute this into the formula for

H

, and neglect terms that are quadratic in

δ σ_{i}

. Calculate

Z

and

F

in this approximation, and verify that

F = - β^{- 1} N \log [2 \cosh β (v J m + b)] + \frac{1}{2} N v J m^{2}

Verify that the self-cosistency relation of c) is consistent with eq. (B.2) in this approximation.
e) Now let

b = 0

(no external field). Show that the self-consistency equation for

m

in b) has

m = 0

as its only solution if

T > T_{c}

, where

T_{c} := v J / k_{B}

. Whence, in this case there is no spontaneous magnetization. Show that, for

T < T_{C}

, the self-consistency equation has two nonzero solutions. Thus, below the critical temperature, there is spontaneous magnetization. For

T_{c} - T > 0

and small, solve the self-consistency equation by expanding the tanh around

m = 0

. Show that the solution

m (T)

behaves as

m (T) \sim {(T_{c} - T)}^{1 / 2}

T \to T_{c}

. This behavior is characteristic in the theory of phase transitions. The exponent is called the critical exponent.

Problem B. 17 (Directed polymer). A polymer consists of atoms

i = 0, 1, 2, \dots, N

at positions

(x_{i}, y_{i}) \in Z^{2}

of a square lattice. The atom at the origin is fixed at the position

x_{0} = y_{0} = 0

, and the other atoms are chained together such that

x_{i} - x_{i - 1} = 1

and

| y_{i} - y_{i - 1} | = 1

. This polymer is hence oriented in the

x

-direction and it does not selfintersect.
a) Determine the total number of micro-states of the polymer.
b) Determine the number of microstates

W (N, y)

having the property that

y_{N} = y

.
c) Determine, more generally, the number of microstates

W (i, y)

having the property that

y_{i} = y

. Also calculate

y / W (i, y)

.
d) Determine the number of micro-states

W (i, y, N, y^{'})

having the property that

y_{i} =

y, y_{N} = y^{'}

. Also calculate

y y^{'} / W (i, y, N, y^{'})

.
e) Finally, calculate the typical deflection of the chain end,

⟨ y_{N}^{2} ⟩ = \frac{\sum_{y} y^{2} W (N, y)}{\sum_{y} W (N, y)}

Hint: Consider the partition function of the canonical ensemble

Z (N, β) = \sum_{y} e^{β y} W (N, y) .

B. 4 Exercises for chapter 5

Problem B. 18 (Entropy budget of the Earth). It is estimated that the mass of carbon bound in newly generated biomass on earth is about

10^{11} - 10^{12}

tons per year. Carbon is mostly taken out of the atmosphere by converting

{CO}_{2}

gas and water vapor into organic material via photosynthesis. Organic material consists of highly organized structures and consequently should have a much lower entropy than water vapor and

{CO}_{2}

gas. In order to reconcile this with the principle that the entropy of a system cannot decrease, one notes that the earth is not an isolated system, but receives high energy photons from the sun and emits heat in the form of low energy photons back into space. Through this process, the entropy of the photons is increased. The aim of this question is to estimate this gain and to show that it can account for the entropy decrease through newly generated biomass.
a) Most photons arriving from the sun have a wavelength of

\sim 520 nm

. Using the Einstein relation

E = h ν

for the energy of a single photon, and using the value

1400 \frac{J}{s \cdot m^{2}}

for the energy of solar radiation per area per unit of time, estimate the number of photons arriving on earth from the sun per year. Of the photons arriving on earth, only about

50 %

are absorbed on the surface, whereas the rest is reflected or absorbed by clouds etc. Of these, only about

0.1 %

participate in the actual photosynthesis that results in a net gain of glucose (and then biomass). Hence, what is the number of photons per year participating in the creation of new biomass?
b) The average temperature on the surface of the earth is about

T = 280 K

. Assuming that the intensity of low energy photons emitted from the earth back into space
follows a black body distribution,

I (ν) = \frac{2 π h ν^{3}}{c^{2}} \frac{1}{\exp (\frac{h ν}{k_{B} T}) - 1}

what is the most probable frequency

ν

of photons emitted from earth, i.e. that maximizing

I

(do this first for general

T

)? The total energy of photons absorbed by earth is approximately equal to that of the photons emitted back into space (this follows from energy conservation; we can ignore the chemical energy stored in new biomass). Hence, what is the ratio of photons received on earth to that emitted?
c) The entropy of a gas of

N_{γ}

photons in thermal equilibrium is

S_{γ} \sim 0.9 k_{B} N_{γ}

. Whence, what is the gain in entropy coming from those photons participating in the creation of new biomass (you can leave

k_{B}

in the formula)?
d) Now estimate the entropy decrease through the creation of new biomass. In an extremely simplified description of this process, we can say that

{CO}_{2}

gets converted to C which is bound in organic material, and

O_{2}

, which is released back into the atomosphere. The atmosphere is treated as an ideal gas. According to the formula for the entropy of an ideal gas in the lecture, the entropy contributions are, respectively

S_{α} = k_{B} N_{α} \log [V {(4 π m_{α} k_{B} T)}^{3 / 2}], α = O_{2}, {CO}_{2}

The entropy of bound carbon in organic material is neglected. Using the atomic masses

12 u

for C and

16 u

for O , what is the decrease in entropy due to the conversion of

{CO}_{2}

into

O_{2}

via the creation of new biomass per year (you can leave

k_{B}

in the formula)? Compare your answer to c). Comment?

Problem B. 19 (Athmospheric pressure). Consider a gas of

N

classical, non-interacting particles enclosed in an infinitely high cylinder with base occupying an area

S

. The cylinder is placed upright into a gravitational field i.e.

\vec{F} = m \vec{g}

, where the axis of the cylinder is parallel to

\vec{g}

.
a) What is the Hamiltonian of the system? What is the density function

ρ

for the canonical ensemble?
b) What is the average number of particles above height

h

?
c) Derive from b) a formula for the pressure as a function of

h

Problem B. 20 (Relativistic classical ideal gas). For a relativistic particle, the energymomentum relation is

ϵ (\underset{―}{p}) = \sqrt{m^{2} c^{4} + c^{2} p^{2}}

, where

p = | \underset{―}{p} |

. We first consider a classical gas of

N

indistinguishable massless particles enclosed in a box with volume

V

.
a) Show that the partition function (canonical ensemble) is given by

Z = \frac{1}{N!} {(8 π V {(\frac{k_{B} T}{h c})}^{3})}^{N}

b) By analogy with the non-relativistic case, where

λ_{cl} = h / \sqrt{m k_{B} T}

, we see that the thermal de Broglie wave length is now

λ_{rel} = h c / (k_{B} T)

. Using Stirling's formula, find the expression

F \approx - N k_{B} T [1 + \log (\frac{8 π V}{N λ_{rel}^{3}})]

for the free energy. Use the standard relation

P = - \partial F / {\partial V |}_{T}

to derive the equation of state for the relativistic gas. Compare your result to the non-relativistic case. Do the same for the internal energy

E = F - T \partial F / {\partial T |}_{V}

.
c) Show that the relativistic and non-relativistic de Broglie wave lengths are related by

\frac{λ_{rel}}{λ_{cl}} = \sqrt{\frac{m c^{2}}{k_{B} T}}

so that, for non-relativistic particles with

k_{B} T ≪ m c^{2}

we have

λ_{rel} ≫ λ_{cl}

. We can consider

d = (V / N)^{1 / 3}

as the mean distance between the particles. Quantum effects should become important when

d

becomes less than the de Broglie wave length. Increasing

N

, where should quantum effects show up first, in the non-relativistic or the relativistic system?
d) At what temperature is the de Broglie wavelength comparable to the wave length for photons in the visible part of the spectrum, say 500 nm ? What wave length do photons have if their wave length is equal to the de Brogile wave length at room temperature?
e) Repeat the derivation in a) and b) for a massive relativistic particle working to first non-trivial order in

m c^{2} / k_{B} T

B. 5 Exercises for chapter 6

Problems

Problem B. 21 (First law of thermodynamics). Consider the first law of thermodynamics:

T d S = d E + P d V - μ d N

a) Derive the relations

{\frac{\partial N}{\partial E} |}_{V, S} = \frac{1}{μ}, {\frac{\partial N}{\partial S} |}_{V, E} = - \frac{T}{μ}, {\frac{\partial N}{\partial V} |}_{E, S} = \frac{P}{μ} .

Hint: rewrite the first law in terms of the differentials

d E, d V, d S

.
b) Introduce the free energy by

F = E - T S

, viewed as a function of

T, N, V

. Write the first law in terms of

F

instead of

E

.
c) Write the first law as

d S = \dots

. Applying the exterior differential

d

to the resulting equation and using

d (d S) = 0

, derive the relation

{\frac{\partial T}{\partial V} |}_{E, N} - {P \frac{\partial T}{\partial E} |}_{V, N} + {T \frac{\partial P}{\partial E} |}_{V, N} = 0 .

Hint: Keep in mind that

d E d V = - d V d E

.
Problem B. 22 (Idealized Otto engine). An idealized Otto engine is described by the following cycle:

$I \to I I$ : Adiabatic compression of air: piston moves up.
II $\to I I I$ : Constant volume heat transfer: ignition and burning of fuel.
III $\to I V$ : Adiabatic expansion: power stroke, piston moves down.
IV $\to I$ : Constant volume cooling.
a) Draw the cycle in a $(P, V)$ -diagram. Identify those processes in the diagram where work is performed by/on the system, and where heat is injected/given off by the system.
b) Treating the fluid as an ideal gas, compute the net work $Δ W$ performed by the system in one cycle, and the heat $Δ Q_{in}$ injected into the system. (Give each of these quantities in terms of the temperatures $T_{I}, \dots, T_{I V})$ . Compute the efficiency $η$ of the idealized Otto-cycle in terms of the temperatures.

Problem B. 23 (Cyclic process). Consider the following cyclic process:

$I \to I I$ : Adiabatic (constant $S$ ) expansion
$I I \to I I I$ : Isochoric (constant $V$ ) cooling
III $\to I V$ : Adiabatic (constant $S$ ) compression
$I V \to I$ : Isothermal (constant $T$ ) expansion

Throughout it is assumed that the particle number

N

remains constant (so that

d N = 0

in the entire process), and we assume that the equations of state of an ideal gas hold:

P V = N k_{B} T, E = \frac{3}{2} P V .

a) Show that

P V = c s t

. on isotherms and

P V^{5 / 3} = c s t

. on adiabatics using the equation(s) of state and the first law

T d S = d E + P d V

. (If you cannot do this, carry on with b)-e) assuming these results.)
b) Sketch the process in a (

P, V

)-diagram, and identify where heat is injected/given off by the system.
c) What is the work

Δ W

performed by the system in one cycle?
d) What is the heat

Δ Q_{in}

injected into the system in one cycle?
e) What is the efficiency

η = \frac{Δ W}{Δ Q_{in}}

State your answers in c)-e) in terms of

N, T_{I}, V_{I}, V_{I I} (= V_{I I I}), V_{I V}

.
Problem B. 24 (Gibbs-Duhem relation). Consider a system in equilibrium characterized by a fixed energy

E

, volume

V

, and particle number

N_{i}

for the

i

-th species of particle. We argued in chapter 4 that the entropy

S (E, V, {N_{i}})

of such an equilibrium state is extensive in the sense that, for each

ν

, we have

S (E, V, {N_{i}}) = ν S (E / ν, V / ν, {N_{i} / ν}) .

a) Differentiate this relation to obtain the Gibbs-Duhem relation

E + P V - \sum_{i} μ_{i} N_{i} - T S = 0

(Use the definitions of

T, P, μ_{i}

in terms of

S

given in the lectures.)
b) Write this relation as

H = \sum_{i} μ_{i} N_{i}

in terms of the free enthalpy

H

, and derive the relationship

d P = s d T + \sum_{i} n_{i} d μ_{i}

for the pressure, where

s = S / V

and

n_{i} = N_{i} / V

are the entropy- and number densities. Derive the identities

{\frac{\partial s}{\partial μ_{i}} |}_{P, T} = {\frac{\partial n_{i}}{\partial T} |}_{{μ_{j}}, P}, {\frac{\partial n_{i}}{\partial μ_{j}} |}_{P, T} = {\frac{\partial n_{j}}{\partial μ_{i}} |}_{P, T}

c) Consider two copies of a system characterized by the variables

z^{(1)} = (E^{(1)}, V^{(1)}, N^{(1)})

and

z^{(2)} = (E^{(2)}, V^{(2)}, N^{(2)})

, which are separately in equilibrium but not necessarily with each other. Since the entropy of the composite system is maximal in equilibrium, we should normally have

S (z^{(1)}) + S (z^{(2)} ⩽ S (z^{(1)} + z^{(2)})

Argue that the entropy must be a concave function. [Recall that a function

f (x)

n

variables

x = (x_{1}, \dots, x_{n})

is called concave iff

f (λ x + (1 - λ) y) ⩾ λ f (x) + (1 - λ) f (y)

for all

0 ⩽ λ ⩽ 1

.] In particular, show that

{\frac{\partial T}{\partial E} |}_{V, N} ⩾ 0

Problem B. 25 (Charged gas). We consider a gas of particles of unit charge

\pm q

. The eigenstates of the charge operator

\hat{Q}

and the Hamiltonian

\hat{H}

are

| n_{+}, n_{-} ⟩

with

\hat{H} | n_{+}, n_{-} ⟩ = ϵ_{n_{+}, n_{-}} | n_{+}, n_{-} ⟩, \hat{Q} | n_{+}, n_{-} ⟩ = q (n_{+} - n_{-}) | n_{+}, n_{-} ⟩

where

n_{+}, n_{-} ⩾ 0

are integers that have the interpretation of the number of positively resp. negatively charged particles in the state. We consider a density matrix of the form

ρ = \sum_{n_{+}, n_{-} ⩾ 0} p_{n_{+}, n_{-}} | n_{+}, n_{-} ⟩ ⟨ n_{+}, n_{-} | .

The information entropy is defined by

S (ρ) = - k_{B} tr ρ \log ρ

, the mean energy by

E = ⟨ \hat{H} ⟩

and the mean charge by

Q = ⟨ \hat{Q} ⟩

.
a) Using the method of Lagrange multipliers, show that the density matrix which maximizes

S (ρ)

for fixed

E, Q

is of the form

p_{n_{+}, n_{-}} = \frac{1}{Y} \exp (- β [ϵ_{n_{+}, n_{-}} + Φ q (n_{+} - n_{-})])

or equivalently

ρ = \frac{1}{Y} \exp [- β (\hat{H} + Φ \hat{Q})]

where

Y = tr \exp [- β (\hat{H} + Φ \hat{Q})]

(Here

β

and

Φ

are constants.)
b) Define

G = - k_{B} T \log Y (T, Φ)

, where

β^{- 1} = k_{B} T

. Show that

\begin{matrix} (B.3) & S = - {\frac{\partial G}{\partial T} |}_{Φ}, Q = - {\frac{\partial G}{\partial Φ} |}_{T} \end{matrix}

where

S, Q

are defined as above.
c) For a charged gas at fixed volume, the first law of thermodynamics is

T d S = d E -

Φ d Q

. What is the physical meaning of

Φ

? Show that if we define

G = E - T S - Φ Q

, then

G = G (T, Φ)

satisfies

d G = - S d T - Q d Φ

.
d) Verify the relations (B.3) using

d G = - S d T - Q d Φ

Problem B. 26 (Virial expansion and van der Waals equation of state). The aim of this exercise is to use the linked cluster expansion in order to derive an equation of state for a realistic monoatomic gas. Recall that the cluster expansion for a classical monoatomic non-relativistic gas is

\frac{1}{V} \log Y (μ, V, β) = \frac{1}{λ^{3}} \sum_{l = 1}^{\infty} b_{l} (V, β) z^{l}

where

Y

is the grand canonical partition function,

z = e^{β μ}

is the fugacity, and

λ

is the thermal de Broglie wavelength.
a) Using the Gibbs-Duhem relation and expressing the grand potential

G

in terms of

Y

according to (6.70), show that

\frac{P}{k_{B} T} = \frac{1}{λ^{3}} \sum_{l = 1}^{\infty} b_{l} (V, β) z^{l}

b) Using

N = - {\frac{\partial G}{\partial μ} |}_{T, V}

, demonstrate the relation

λ^{3} n = \sum_{l = 1}^{\infty} l \cdot b_{l} (V, β) z^{l} = z + 2 b_{2} z^{2} + 3 b_{3} z^{3} + \dots

where

n = N / V

is the particle density.
c) We next want to eliminate

z

in favor of

n

in a). For this we write

z = λ^{3} n +

a_{2} {(λ^{3} n)}^{2} + a_{3} {(λ^{3} n)}^{3} + \dots

and substitute this in b) in order to determine

a_{2}, a_{3}

. Show that

a_{2} = - 2 b_{2}

and

a_{3} = 8 b_{2}^{2} - 3 b_{3}

.
d) Using the result obtained in c) in a), derive the virial expansion

\frac{P}{k_{B} T} = n (1 + B_{2} (T) n + B_{3} (T) n^{2} + \dots)

where

B_{2} = - b_{2} λ^{3}, B_{3} = (4 b_{2}^{2} - 3 b_{3}) λ^{6}

.
e) Let us now study the virial coefficient

B_{2}

for a typical gas. We use the following approximation:

v (r) = {\begin{cases} + \infty & for r < r_{0} \\ - u_{0} {(r_{0} / r)}^{6} & for r > r_{0} \end{cases}

Sketch this potential. Show that

2 λ^{3} b_{2} = - \frac{4 π r_{0}^{3}}{3} + 4 π \int_{r_{0}}^{\infty} [e^{u_{0} {(r_{0} / r)}^{6} / (k_{B} T)} - 1] r^{2} d r

Approximating the integrand by

\approx u_{0} {(r_{0} / r)}^{6} / (k_{B} T)

in the high temperature limit

u_{0} / k_{B} T ≪ 1

, show that

B_{2} \approx \frac{V_{a}}{2} [1 - u_{0} / (k_{B} T)]

where

V_{a} = \frac{4 π r_{0}^{3}}{3}

is the effective volume of one atom.
f) Using the results of e), we can write the virial expansion as

\frac{P}{k_{b} T} = n + \frac{V_{a}}{2} [1 - u_{0} / (k_{B} T)] n^{2} + \dots

Show that this can be rearranged into

\frac{1}{k_{B} T} (P + \frac{u_{0} V_{a}}{2} n^{2}) \approx \frac{n}{1 - n V_{a} / 2}

neglecting higher orders than

n^{2}

(i.e., assuming low density). Show that this can be written as

(P + a (N / V)^{2}) (V - b N) \approx N k_{B} T

which is known as the van der Waals equation. Identify the van der Waals parameters

a, b

with the microscopic parameters of the system.
g) Plot the isotherms of the van der Waals equation using a computer programme such as Mathematica. It is sensible to plot

p = P / P_{c}, P_{c} = a / (27 b^{2})

against

v = V / (3 b N)

and several isotherms around

T_{c} = 8 a / (27 b)

in the range

0 < P / P_{c} < 2

and

0 < v < 10

. You should see a distinctive qualitative change of the isotherms above and below

T_{c}

. Compare this to the isotherms of the ideal gas.

Acknowledgements

These lecture notes are based on lectures given by Prof. Dr. Stefan Hollands at the University of Leipzig.

$^{1}$ Of course this theory turned out to be incorrect. Nevertheless, we nowadays know that heat can be radiated away by particles which we call "photons". This shows that, in science, even a wrong idea can contain a germ of truth.
$^{2}$ It seems that Lavoisier's foresight in political matters did not match his superb scientific insight. He became very wealthy owing to his position as a tax collector during the "Ancien Régime" but got in trouble for this lucrative but highly unpopular job during the French Revolution and was eventually sentenced to death by a revolutionary tribunal. After his execution, one onlooker famously remarked: "It takes one second to chop off a head like this, but centuries to grow a similar one."
$^{1}$ This description is not always appropriate, as the example of a rigid body shows. Here the phase space coordinates take values in the co-tangent space of the space of all orthogonal frames describing the configuration of the body, i.e. $Ω ≅ T^{*} S O (3)$ , with $S O (3)$ the group of orientation preserving rotations.
$^{2}$ A general self-adjoint operator on a Hilbert space will have a spectral decomposition $A =$ $\int_{- \infty}^{\infty} a d E_{A} (a)$ . The spectral measure does not have to be atomic, as suggested by the formula (2.58). The corresponding probability measure is in general $d μ (a) = ⟨ Ψ ∣ d E_{A} (a) Ψ ⟩$ .
$^{1}$ This equation can be viewed as a discretized analog of the Boltzmann equation in the present context. See the Appendix for further discussion of this equation.
$^{1}$ The quantity $W^{cl}$ is for this reason often defined by

$\begin{matrix} (4.43) & W^{cl} (E, N) := h^{- 3 N} | Ω_{E, N} | \end{matrix}$

Also, one often includes further combinatorial factors to include the distinction between distinguishable and indistinguishable particles, cf. (4.49).
$^{2}$ For distinguishable particles, this would be $H_{N} = L^{2} (R^{N})$ . However, in real life, quantum mechanical particles are either bosons or fermions, and the corresponding definition of the $N$ -particle Hilbert space has to take this into account, see Ch. 5.
$^{3}$ The proof of the linked cluster theorem is very similar to that of the formula (2.10) for the cumulants ${⟨ x^{n} ⟩}_{c}$ , see section 2.1.
$^{1}$ Here, we make use of the Riemann zeta function, which is defined by
$^{1}$ This is how one could actually mathematically implement the idea of "thermal contact"
$^{2}$ Mathematically, the differentials $d X_{i}$ are the generators of a Grassmann algebra of dimension $N$ .
$^{3}$ Here we quote the formula for indistinguishable particles, which means that we should include the $\frac{1}{N!}$ into the definition of the microcanonical partition function $W (E, N, V)$ for indistinguishable particles, cf. section 4.2.3.
$^{4}$ One also uses the enthalpy defined as $E + P V$ . Its natural variables are $T, P, N$ which is more useful for processes leaving $N$ unchanged.
$^{5}$ Here we assume implicitly that $[H, {\hat{N}}_{1}] = 0$ so that $H$ maps subspaces of $N_{1}$ -particles to itself.
$^{1}$ More precisely, $χ_{i j} = lim \frac{M_{i}}{V \cdot B_{j}}$ is a tensor. Here we only look at the $z z$ -component of this tensor, which is relevant in our situation.

Lecture Notes on Statistical Mechanics and Thermodynamics

Universität Leipzig

Contents

List of Figures

Chapter 1

Introduction and Historical Overview

Timeline

17 th 17 th 17^("th ")17^{\text {th }}17th century:

Chapter 2

Basic Statistical Notions

2.1 Probability Theory and Random Variables

2.2 Entropy

2.3 Examples of probability distributions

2.3.1 Gaussian distribution

2.3.2 Binomial and Poisson distribution

2.3.3 Ising model

2.3.4 Site percolation

2.3.5 Random walk on a lattice

2.4 Ensembles in Classical Mechanics

2.5 Ensembles in Quantum Mechanics (Statistical Operators / Density Matrices)

Chapter 3

Time-evolving ensembles

3.1 Boltzmann Equation in Classical Mechanics

3.2 Boltzmann Equation, Approach to Equilibrium in Quantum Mechanics

Chapter 4

Equilibrium Ensembles

4.1 Generalities

4.2 Micro-Canonical Ensemble

4.2.1 Micro-Canonical Ensemble in Classical Mechanics

Example:

4.2.2 Microcanonical Ensemble in Quantum Mechanics

4.2.3 Mixing entropy of the ideal gas

4.3 Canonical Ensemble

4.3.1 Canonical Ensemble in Quantum Mechanics

4.3.2 Canonical Ensemble in Classical Mechanics

Example:

4.3.3 Equidistribution Law and Virial Theorem in the Canonical Ensemble

Example: estimation of the mass of distant galaxies:

4.4 Grand Canonical Ensemble

4.5 Summary of different equilibrium ensembles

4.6 Approximation methods

4.6.1 The cluster expansion

4.6.2 Peierls contours

Chapter 5

The Ideal Quantum Gas

5.1 Hilbert Spaces, Canonical and Grand Canonical Formulations

Example:

Examples:

5.2 Degeneracy pressure for free fermions

5.3 Spin Degeneracy

5.4 Black Body Radiation

5.5 Degenerate Bose Gas

Chapter 6

The Laws of Thermodynamics

6.1 The Zeroth Law

6.2 The First Law

Differentials ("differential forms")

6.3 The Second Law

6.4 Cyclic processes

6.4.1 The Carnot Engine

6.4.2 General Cyclic Processes

6.4.3 The Diesel Engine

6.5 Thermodynamic potentials

Example: Virial expansion and van der Waals equation of state

6.6 Chemical Equilibrium

6.7 Phase Co-Existence and Clausius-Clapeyron Relation

Examples:

6.8 Osmotic Pressure

Appendix A

Dynamical Systems and Approach to Equilibrium

A. 1 The Master Equation

Example: Time evolution of a population of bacteria

A. 2 Properties of the Master Equation

A. 3 Relaxation time vs. ergodic time

A. 4 Monte Carlo methods and Metropolis algorithm

Metropolis Algorithm

A. 5 Eigenstate thermalization

Appendix B

Exercises

B. 1 Exercises for chapter 2

$17^{th}$ century: