Down the rabbit hole

_posts/2019-02-14-pset2.md

@@ -9,8 +9,8 @@ title: Problem Set 2
 Derive equation (3.16) (i.e. the Poisson probability density function) from the binomial
 distribution and Stirling’s approximation.
 
-Fix any $$\lambda$$ between 0 and 1. For any $$n$$, the binomial probability mass function for $$n$$
-trials with probability of success $$p = \lambda / n$$ is
+Fix any (positive, real) $$\lambda$$. For any $$n$$, the binomial probability mass function for
+$$n$$ trials with probability of success $$p = \lambda / n$$ is
 
 $$
 \begin{align*}
@@ -90,13 +90,18 @@ N per second. How many must be counted by a photodetector per second to be able
 rate to within 1%? To within 1 part per million? How many watts do these cases correspond to for
 visible light?
 
-We have already found that $$\sigma = \sqrt{\lambda}$$. For large $$\lambda$$, the Poisson
+### The Poisson Approach
+
+Note: in the previous problem I used $$\lambda$$ as the expected number of events of a Poisson process.
+Here I'll use $$N$$ to avoid confusion with wavelength.
+
+We have already found that $$\sigma = \sqrt{N}$$. For large $$N$$, the Poisson
 distribution is very close to the normal distribution. So about two thirds of the probability mass
-lies between $$\lambda - \sigma$$ and $$\lambda + \sigma$$. Thus if $$\sigma \leq 0.01 \lambda$$, in
+lies between $$N - \sigma$$ and $$N + \sigma$$. Thus if $$\sigma \leq 0.01 N$$, in
 any given second it's more likely than not that the number of photons emitted is within one percent
-of the true mean. Thus we'd need $$\sqrt{\lambda} \leq 0.01 \lambda$$, i.e. $$\lambda \geq 10^4$$.
-To have the same probability that the number of observed photons is within $$10^{-6} \lambda$$ of
-the true value, we need $$\lambda \geq 10^{12}$$.
+of the true mean. Thus we'd need $$\sqrt{N} \leq 0.01 N$$, i.e. $$N \geq 10^4$$.
+To have the same probability that the number of observed photons is within $$10^{-6} N$$ of
+the true value, we need $$N \geq 10^{12}$$.
 
 The wavelength of visible light is about $$\num{500e-9} \si{m}$$, so the energy of each photon will
 be
@@ -112,7 +117,7 @@ $$\num{3.8e-7} \si{W}$$.
 
 Some caveats about this answer: the Poisson distribution is strictly greater than zero on its whole
 domain, so one can never be *certain* that they have found the correct value within any error
-bounds. That is, even if the real value of $$\lambda$$ is one, there's still a (vanishingly) slight
+bounds. That is, even if the real value of $$N$$ is one, there's still a (vanishingly) slight
 chance you'll see a million. The best we can do is find an answer that's [probably approximately
 correct](https://en.wikipedia.org/wiki/Probably_approximately_correct_learning).
 
@@ -120,13 +125,104 @@ Second, I only found limits on the probability that the number of emitted photon
 (known) average. It's a more subtle question to ask for the probability that a given estimate is
 within one percent of the true (latent) mean. That could be tackled with confidence intervals,
 p-values, Bayes factors, or direct posterior estimation, though none would be quite so simple. As an
-unapologetic Bayesian I'm required to point out that my confidence I've determined $$\lambda$$ with
-some precision depends on my prior for $$\lambda$$. This would be very real if we were, for
+unapologetic Bayesian I'm required to point out that my confidence I've determined $$N$$ with
+some precision depends on my prior for $$N$$. This would be very real if we were, for
 instance, counting photons in a lab module for this course. I would be more confident in my answer
 if I measured something close to $$\num{1e12}$$ photons, than if I measured something like
 $$\num{3.64e12}$$, since I think it's more likely that the instructor would choose a nice round
 number with lots of zeros.
 
+### The Shot Noise Approach
+
+To verify:
+- expected displacement of random walk with normally distributed steps
+- parseval's theorem without expectations (I expect this is true) -> Yes; Plancherel Theorem
+- parseval's theorem without the square (i.e. just magnitude) -> False, considering that a
+  constant value (infinite area) maps to a delta function (finite)
+- how to show Guassian normalization, and variance inversion under Fourier transform
+
+Let's also look at this from the perspective of shot noise. As derived in the book, the shot noise
+formula isn't exactly what we want. This is because the energy transmitted by a sequence of photons
+is just the sum of the energy carried by each photon; it's not proportional to the square of any
+photon "current". So the power
+makes some assumptions that aren't clearly valid here. First, if each photon carries the
+same amount of energy, then the above Poisson analysis is all we need, since
+same amount of energy
+
+We can measure our "current" of light in photons per second, assigning each photon a "charge" of one
+photon. But Then the root mean square shot noise will be
+
+$$
+\begin{align*}
+\sqrt{\langle I_\text{noise}^2 \rangle}
+&= \sqrt{2 q \langle I \rangle \Delta f} \\
+&= \sqrt{2 \cdot 1 \si{photon} \cdot N \si{photons/s} \cdot \num{1e-8} \si{Hz}}
+\end{align*}
+$$
+
+So we need to solve
+
+$$
+\sqrt{2 \cdot 1 \si{photon} \cdot N \si{photons/s} \cdot \num{1e-8} \si{Hz}} \leq 0.01 N
+$$
+
+Ok let's do this the right way. Instead of electric current (in Amps), we're looking directly at Watts.
+As a function of time, this is
+
+$$
+W(t) = \sum_{n = 1}^N q_n \delta(t - t_n)
+$$
+
+where $$q_n$$ is the energy of the $$n$$th photon, and $$t_n$$ its time of arrival. The Fourier
+transform of this function is
+
+$$
+\begin{align*}
+W(f) &= \int_\mathbb{R} e^{-2 \pi i t f} \sum_{n = 1}^N q_n \delta(t - t_n) \mathrm{d} t \\
+&= \sum_{n = 1}^N q_n e^{-2 \pi i t_n f} \\
+\end{align*}
+$$
+
+Each photon contributes one term to this sum. The phase of its contribution is determined by its
+arrival time (and the frequency of interest), while the magnitude is determined by its energy.
+
+We want to know the expected magnitude of this function. Formally the expectation should be taken
+over sequences of arrival energies and times $$(q_1, t_1), \ldots, (q_n, t_N)$$. The probability of
+such a sequence could be modeled so that every $$q_n$$ is independently drawn from a certain normal
+distribution, and the $$t_n$$ are generated by a Poisson process.
+
+$$
+\begin{align*}
+\langle |W(f)| \rangle &= \langle \left| \sum_{n = 1}^N q_n e^{-2 \pi i t_n f} \right| \rangle \\
+&= \sqrt{N} \langle q \rangle
+\end{align*}
+$$
+
+This is a 2d Guassian random walk. The arrival times determine direction (phase), and the energies determine
+the step lengths. For a given frequency, all are uncorrelated in expectation.
+
+So to
+
+Let's look at the expected squared magnitude as well, so we can compare with the standard shot noise
+formula.
+
+$$
+\begin{align*}
+\langle |W(f)|^2 \rangle &= \langle \sum_{n = 1}^N q_n e^{-2 \pi i t_n f}
+                                    \sum_{m = 1}^N q_m e^{2 \pi i t_m f} \rangle \\
+&= \langle \sum_{n = 1}^N \sum_{m = 1}^N q_n q_m e^{-2 \pi i (t_n - t_m) f} \rangle \\
+&= \langle \sum_{n = 1}^N q_n^2 + \sum_{n < m} 2 q_n q_m
+   \text{Re} \left[ e^{-2 \pi i (t_n - t_m) f} \right] \rangle \\
+&= \sum_{n = 1}^N \langle q_n^2 \rangle + 2 \sum_{n < m} \langle q_n q_m
+   \text{Re} \left[ e^{-2 \pi i (t_n - t_m) f} \right] \rangle \\
+&= N \langle q^2 \rangle
+\end{align*}
+$$
+
+The second term drops out because the arrival times are generated by a Poisson process, so the real
+part of the phase differences averages out to zero.
+
+
 ## (3.3)
 
 Consider an audio amplifier with a 20 kHz bandwidth.
@@ -136,6 +232,25 @@ Consider an audio amplifier with a 20 kHz bandwidth.
 If it is driven by a voltage source at room temperature with a source impedance of 10kΩ how large
 must the input voltage be for the SNR with respect to the source Johnson noise to be 20 dB?
 
+$$
+\begin{align*}
+\langle V_\text{noise}^2 \rangle &= 4 k T R \Delta f \\
+&= 4 \cdot \num{1.38e-23} \si{J/K} \cdot 300 \si{K} \cdot 10^4 \si{\ohm} \cdot \num{20e3} \si{Hz} \\
+&= \num{3.3e-12} \si{V^2}
+\end{align*}
+$$
+
+So for the input signal to be 20dB louder, we need
+
+$$
+10 \log_{10} \left( \frac{V_\text{signal}^2}{\num{3.3e-12}} \right) = 20 \\
+V_\text{signal}^2 = 10^2 \cdot \num{3.3e-12} \si{V^2} = \num{3.3e-10} \si{V^2}
+$$
+
+Note that I use a factor of 10 since $$V^2$$ is proportional to power. This signal strength would be
+achieved, for instance, by a sinusoid that ranges between $$-\num{6.6e-10}$$ and $$\num{6.6e-10}$$
+volts.
+
 ### (b)
 
 What size capacitor has voltage fluctuations that match the magnitude of this Johnson noise?

_posts/2019-02-15-ch2-notes.md

@@ -50,8 +50,8 @@ $$
 (\mathcal{F} f(x' + x_0))(x)
 &= \int_\mathbb{R} f(x' + x_0) e^{-2 \pi i x' x} \mathrm{d} x' \\
 &= \int_\mathbb{R} f(x') e^{-2 \pi i (x' - x_0) x} \mathrm{d} x' \\
-&= e^{2 \pi i x_0} \int_\mathbb{R} f(x') e^{-2 \pi i x' x} \mathrm{d} x' \\
-&= e^{2 \pi i x_0} (\mathcal{F} f(x')))(x)
+&= e^{2 \pi i x_0 x} \int_\mathbb{R} f(x') e^{-2 \pi i x' x} \mathrm{d} x' \\
+&= e^{2 \pi i x_0 x} (\mathcal{F} f(x'))(x)
 \end{align*}
 $$