Ex 1.10 - On mean and variance of two variables (Pattern Recognition and ML by Christopher Bishop)

Question:




Answer:

From the solutions to book

Using ChatGPT

If x and y are two independent continuous variables, can you derive the expression for E[x+y]

Certainly! If $x$ and $y$ are two independent continuous random variables, you can find the expected value of their sum $E[x + y]$ by using the property that for independent random variables, the expected value of the sum is the sum of the expected values. Mathematically:

$E[x + y] = E[x] + E[y]$

So, the expected value of the sum of two independent continuous random variables is simply the sum of their individual expected values.

If you have specific probability density functions (PDFs) for $x$ and $y$, you can also find $E[x + y]$ by integrating over all possible values of $x$ and $y$:

$E[x + y] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} (x + y) \cdot f(x) \cdot g(y) \,dx \,dy$

Here, $f(x)$ and $g(y)$ are the probability density functions of $x$ and $y$ respectively.

If $x$ and $y$ have known probability density functions, you can substitute them into the above expressions to find the expected value of their sum.

Please integrate the last expression and show how for i.i.d. x and y, E[x+y] = E[x] + E[y]

Certainly! Let's integrate the last expression and show that for independent and identically distributed (iid) random variables $x$ and $y$, $E[x + y] = E[x] + E[y]$.

Assuming $f(x)$ and $g(y)$ are the probability density functions (PDFs) of $x$ and $y$ respectively, and $x$ and $y$ are iid, the expression for $E[x + y]$ is:

$E[x + y] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} (x + y) \cdot f(x) \cdot g(y) \,dx \,dy$

Now, let's perform the integration:

$E[x + y] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} (x + y) \cdot f(x) \cdot g(y) \,dx \,dy$

$= \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} x \cdot f(x) \cdot g(y) \,dx \,dy + \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} y \cdot f(x) \cdot g(y) \,dx \,dy$

Now, by the independence of $x$ and $y$, the joint PDF $f(x, y)$ can be expressed as the product of the individual PDFs:

$f(x, y) = f(x) \cdot g(y)$

So, we can simplify the above expression:

$= \int_{-\infty}^{\infty} x \cdot f(x) \left(\int_{-\infty}^{\infty} g(y) \,dy\right) \,dx + \int_{-\infty}^{\infty} y \left(\int_{-\infty}^{\infty} f(x) \,dx\right) \cdot g(y) \,dy$

The integrals inside the parentheses evaluate to 1 because $f(x)$ and $g(y)$ are probability density functions, which integrate to 1 over their respective ranges. Therefore:

$= \int_{-\infty}^{\infty} x \cdot f(x) \,dx + \int_{-\infty}^{\infty} y \cdot g(y) \,dy$

This is equivalent to $E[x] + E[y]$, showing that for iid $x$ and $y$, $E[x + y] = E[x] + E[y]$.