7.3.2.P4

Prerequisites:

• Probability
• Expected values  
• The 2nd Law of Thermodynamics: A Probabilistic Law 

The concept of entropy depends on counting the ways energy can be distributed, and counting microstates and macrostates. Here we will think about these issues in a simpler but analogous context that will help you see what microstates and macrostates correspond to: coin flips.

In this problem, a coin flip can yield heads or tails. “Heads” will be worth +1 points, and “tails” will be worth -1 points.

1. The expected value (or average value) of a randomly-determined quantity is obtained by

multiplying the probability of each of outcome, P_i, with the value of each outcome, S_i, and summing the results over the possible outcomes, i = 1...N.

<S> = Σ S_iP_i. = S₁P₁ + S₂P₂ + S₃P₃ ...

1.1. For a single coin flip, list the possible outcomes, i = 1,...?, the value of each outcome, S_i, and the probability of each outcome P_i. Use the expectation equation to calculate the average value of a single coin flip. 

1.2. For flipping two coins, list the possible outcomes, i = 1,...?, the value of each outcome, S_i, and the probability of each outcome P_i. Use the expectation equation to calculate the expected average value of the two flips.

1.3. How about flipping 10 coins? What is the average total expected number of points?

1.4. Consider flipping N coins. If the total number of points obtained from the N flips labels our macrostates, which macrostate is most likely? 

2. Now let’s make the connection to entropy more explicit. The key is determining how many different microstates correspond to what we choose to call a macrostate. Recall that in the case of the flipping coins, a microstate is a distinct outcome -- a specific arrangement of heads and tails on the flipped coins, and a macrostate is the total value in points. Each microstate has a value (number of heads minus the number of tails) but each macrostate may correspond to many microstates,

2.1 If one flips 2 coins once each, how many different microstates (total distinct outcomes of sets of heads and tails) are there?  How many microstates are there if one flips 3 coins once each? What is the total number of microstates associated with flipping N coins once each? 

2.2 We say that the entropy is proportional to the logarithm of the total number of microstates (time Boltzmann's constant to give us the right units for entropy). For our coin flipping model, select the most plausible formula for the entropy S. 

1. S = k_BN log 2
2. S = k_B log N
3. S = k_B/N²
4. S = k_B2^N
5. S = k_BN²
6. S = k_Be^N

2.3 Entropy is normally extensive. That implies that the entropy of (2 x 10³ coins flipped) is equal to twice the entropy of (10³ coins flipped). Carefully demonstrate whether or not your chosen entropy formula describes an extensive property. 

2.4 For N as large as 10²³, the macrostate with the largest number of microstates is zero. Yet it is also true that the probability of ending up with no points decreases as N increases. This probability P₀ ∝ 1/√𝑁 for large values of N. That gets to be a very small number! Can you make sense of this odd result? Discuss! Be clear!

Bill Dorland with formatting by Joe Redish 1/30/18