Solution 1.11 - On Gaussian Distribution and it's Log Likelihood (Pattern Recognition and ML by Christopher Bishop)

Can you give me the equation for Gaussian distribution?

Can you give me it's log likelihood function?

Note: This is already given as part of the problem statement.

Can you show the detailed steps how you arrived at it?

Some paperwork

In the next slide we will solve for the below expression:

Some Logarithm Properties we would use:

Can you maximize the log likelihood expression w.r.t. mu and show the detailed steps?

Some paperwork for derivative of:

We will prove it next:

Some ‘Differentiation’ identities we would see next

And now maximize log likelihood w.r.t. sigma-squared and show the detailed steps

Identity:

References

https://chat.openai.com/share/4278db33-9b76-4d8b-9586-4e1e78844433

Tags: Mathematical Foundations for Data Science,Machine Learning,

Question 1.11 - On Gaussian Distribution and it's Log Likelihood (Pattern Recognition and ML by Christopher Bishop)

Pre-read:

(1)


(2)


(3)

From (1.46) and (1.53), the log likelihood function can be written in the form:



Question:

By setting the derivatives of the log likelihood function (1.54) with respect to µ and σ^2 equal to zero, verify the results (1.55) and (1.56).

Link to Solution

Ex 1.10 - On mean and variance of two variables (Pattern Recognition and ML by Christopher Bishop)

Question:




Answer:

From the solutions to book

Using ChatGPT

If x and y are two independent continuous variables, can you derive the expression for E[x+y]

Certainly! If $x$ and $y$ are two independent continuous random variables, you can find the expected value of their sum $E[x + y]$ by using the property that for independent random variables, the expected value of the sum is the sum of the expected values. Mathematically:

$E[x + y] = E[x] + E[y]$

So, the expected value of the sum of two independent continuous random variables is simply the sum of their individual expected values.

If you have specific probability density functions (PDFs) for $x$ and $y$, you can also find $E[x + y]$ by integrating over all possible values of $x$ and $y$:

$E[x + y] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} (x + y) \cdot f(x) \cdot g(y) \,dx \,dy$

Here, $f(x)$ and $g(y)$ are the probability density functions of $x$ and $y$ respectively.

If $x$ and $y$ have known probability density functions, you can substitute them into the above expressions to find the expected value of their sum.

Please integrate the last expression and show how for i.i.d. x and y, E[x+y] = E[x] + E[y]

Certainly! Let's integrate the last expression and show that for independent and identically distributed (iid) random variables $x$ and $y$, $E[x + y] = E[x] + E[y]$.

Assuming $f(x)$ and $g(y)$ are the probability density functions (PDFs) of $x$ and $y$ respectively, and $x$ and $y$ are iid, the expression for $E[x + y]$ is:

$E[x + y] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} (x + y) \cdot f(x) \cdot g(y) \,dx \,dy$

Now, let's perform the integration:

$E[x + y] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} (x + y) \cdot f(x) \cdot g(y) \,dx \,dy$

$= \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} x \cdot f(x) \cdot g(y) \,dx \,dy + \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} y \cdot f(x) \cdot g(y) \,dx \,dy$

Now, by the independence of $x$ and $y$, the joint PDF $f(x, y)$ can be expressed as the product of the individual PDFs:

$f(x, y) = f(x) \cdot g(y)$

So, we can simplify the above expression:

$= \int_{-\infty}^{\infty} x \cdot f(x) \left(\int_{-\infty}^{\infty} g(y) \,dy\right) \,dx + \int_{-\infty}^{\infty} y \left(\int_{-\infty}^{\infty} f(x) \,dx\right) \cdot g(y) \,dy$

The integrals inside the parentheses evaluate to 1 because $f(x)$ and $g(y)$ are probability density functions, which integrate to 1 over their respective ranges. Therefore:

$= \int_{-\infty}^{\infty} x \cdot f(x) \,dx + \int_{-\infty}^{\infty} y \cdot g(y) \,dy$

This is equivalent to $E[x] + E[y]$, showing that for iid $x$ and $y$, $E[x + y] = E[x] + E[y]$.

Statistics. Unit 12: Hypothesis Testing. Ch 1: The idea of significance tests

A: Simple hypothesis testing
Question A.1


Ans:




Question A.2


Ans:




Question A.3



Ans:




Question A.4




Ans:



B: Writing null hypothesis

Question B.1



Ans:



Question B.2




Ans:




C: Estimating P-values from simulations

Question C.1




Ans:



Question C.2:



Ans:



Question C.3:



Question C.4:



D: Using P-values to make conclusions
Q:


A:

Tags: Math,Statistics