Lectures 4+5 homework:

[1 point each] Mathematical exercises. Do any of the exercises &/or problems in chapter 11 of Nielsen & Chuang.
[7 points] Numerical experiment. You will want to use a math tool to help you with the below exercises. I know that this exercise can definitely be easily done in Matlab, but if you can figure out how to do this in Mathematica, Maple, or emacs calc, that's OK too.   Please state what tool you used, and what functions within that tool you used.
    We are going to construct a density matrix representing an uncertain quantum state (mixed state) of a system S having 4 distinguishable basis states b₁, ..., b₄, and figure out its entropy.
    Suppose we prepare this system using a mechanism that initalizes it randomly to be in 1 of 5 quantum states (state vectors) s₁, ..., s₅. (Call these states "preparations.") Assume we don't know which state the mechanism chose.
    Note that each of the below questions depends on the previous ones. If you make a critical error at any point, you will get zero credit for the remaining parts of the problem!
    (a) [1/2 point] Given that the system has only 4 distinguishable states, what is its maximum entropy (total infropy), in bits and in nats?
    (b) [1/2 point] Is it possible for all of the preparations s₁, ..., s₅ to be 100% distinguishable from each other? Explain why or why not.
    (c) [1/2 point] Now, make up an arbitrary probability distribution p₁, ..., p₅ over the 5 preparation states that will be used in the preparation mechanism. Every element of your distribution should be a different real number greater than 0, less than 1, and their sum should be 1. Write down your distribution. What is the value of its Shannon entropy H? Is it possible that you could have chosen a distribution here that had greater entropy than the maximum entropy of system S? So, do you think that the H that you just calculated is likely to be the true entropy of this randomly-prepared initial state?
    (d) [1 point] Now, you need to pick the 5 preparation states s₁, ..., s₅ . Each one of these should be a 4-vector of complex numbers of magnitude 1, giving the amplitudes of the basis states. An easy way to generate such a vector is to write down an arbitrary 4-vector of complex numbers, then "normalize" it to unit magnitude by calculating the magnitude of the vector and dividing each element of the vector by that number. Write down each of these vectors. They should all be different from each other, but they do not all need to be orthogonal to each other(indeed, they cannot all be).
    (e) [1 point] For each of your states s_i, find its equivalent density matrix rho_i. (The definition of density matrices was given in lecture.) Calculate the Shannon entropy of each of these states (relative to the b_i basis), using the diagonal elements of the density matrix. What do you expect the von Neumann entropy of each of these individual states to be?
    (f) [1 point] Calculate the weighted average of the the density matrices rho_i according to the distribution p_i. The resulting density matrix rho is the actual mixed state of the system.
    (g) [2 points] Calculate the von Neumann entropy of the density matrix rho resulting from part f. Remember to use the matrix logarithm (logm in matlab) and matrix multiplication. Is the result greater or less than the Shannon entropy of the distribution p_i from part c? Try to explain qualitatively why this is the case in your example.
    (h) [1/2 point] Of all the above calculations, which gives the true entropy of the system in question, from our point of view (not knowing which p_i was chosen)? Finally, how much known information (according to the definitions in lecture) is in the system, from our point of view, in bits and in nats?
[10 points] Formal proof (race). Prove (one form of) the 2nd law of thermodynamics. Construct a rigorous, formal, and general proof that von Neumann entropy of a system increases whenever the system's density matrix is transformed by a unitary transform U that is not completely certain (i.e., is given by a weighted average of two or more different transforms U_i), except in the case where the initial state already has maximum entropy.
[1-2 points] Conceptual exercise. Consider the slide from lecture 5 showing different distinctions or categorizations of infropy that one may make. (The slide from lecture 5 has more distinctions than the one from lecture 4.) Come up with an example of a scenario involving some system and some entity with a state of knowledge about that system. For the infropy in the system, and each of the properties on the slide, describe to what extent the infropy in the system has that property. Also include "compressibility" and "effective entropy" as additional distinctions. You may use either a computational or physical example. (A computational example being one where only macro-scale bits are involved.) To earn the 2nd point, come up with one computational example and one physical example.
Example of what you could write (don't use this one though): Computational example: Consider a "system" which is a 1-megabyte binary program on my computer hard disk (bits only; we're not including the detailed microstate of the atoms in the magnetic domains on the hard disk in the system). The entity is myself.
- I don't know the exact contents of the program, so most of its infropy is entropy, to me. (I might know what architecture it's written for, but that's about it.) However, if I copy the file to a floppy that I carry around with me, the entropy in the original file is now information, if I consider the floppy's contents to be part of my personal "knowledge." (This might be more intuitive if the copy were on a computer chip implanted in my brain that I could access at will, or if the file were short enough that I could just memorize it.)
- This infropy is accessible. In more detail, I can both read the file (it is measurable) and write the file (it is controllable) unless, of course, I don't have the required access permissions and it's too much trouble to get them.
- No matter how much of the infropy in the file is information, I know that the infropy in the file is reasonably stable against unwanted changes or degradation to entropy, though there is always some chance that a hacker or computer virus will infect my system and alter the program's contents, or that a hard drive head crash or meteor impact will "entropize" the entire program.
- If it's an off-the-shelf program, I know that it is correlated with many other copies around the globe (being identical with them), though I don't know exactly where the other copies are.
- The infropy is wanted because I want to run the program in the future.
- Since it is not a blank area of disk, it is not in a standard state, however, I could convert it to such a state by erasing it. If I have access to another copy, I could even reversibly restore it to a standard state by, say, flipping each bit of the file that is a 1 in the copy.
- If another copy of the program is available, the original program is almost 100% compressible because I could "compress" it by replacing it with a short text file that just says, "to `uncompress' this file, copy it from such-and-such location". Otherwise, the program is probably only moderately compressible (say, by 30-50%).
- The effective entropy of the program is the size of its compressed version, using the best compression method available to me. If no other copies of the program exist, this is probably close to the number of bits in the program.

Remember, if you don't like any of these exercises, you can always do a short paper instead, as described in the grading guidelines, or propose your own project idea.