Spring 2023/2024: ECE 30200 (Section 2) Homework HINTs

Probabilistic Methods in Electrical and Computer Engineering (Spring 2023 by Prof. Saul Gelfand)

HINTs Author: Bianjiang Yang

Comp1 HINTs

Q1:

You should be familiar with:

the definition of CDF: $F_{X} (x) = P r (X \leq x)$
the properties of inverse function: $F_{X}^{- 1} (F_{X} (x)) = x$ and $F_{X}^{- 1} (F_{X} (x)) = x$ . See https://en.wikipedia.org/wiki/Inverse_function and https://www.math.utah.edu/lectures/math1220/2PostNotes.pdf Page2
the properties of increasing function $a \leq b \Leftrightarrow f (a) \leq f (b)$
the CDF of the continuous Uniform Distribution between 0 and 1: $F_{U} (u) = u$ when $u \in (0,1)$ , $F_{U} (u) = 1$ when $u \geq 1$ , $F_{U} (u) = 0$ when $u \leq 0$ .

Q2:

Solve the CDF by considering two conditions:

(1) $(x < 0) : F_{X} (x) = \int_{- \infty}^{x} f_{X} (x) d x, w h e r e | x | = - x$

(2) $(x > 0) : F_{X} (x) = \int_{- \infty}^{0} f_{X} (x^{'}) d x^{'} + \int_{0}^{x} f_{X} (x) d x, | x^{'} | = - x^{'}, | x | = x$

After solving the CDF of X, solve the inverse function by considering two parts as well (i.e., u less than 1/2 and u greater and equal to 1/2)

Please note that $f_{X} (x) = \frac{1}{\sqrt{2 σ^{2}}} e^{- \sqrt{\frac{2}{σ^{2}}} | x |}$

Q3:

I recommend using MATLAB, the following functions should be helpful:

histogram, rememeber to set 'Normalization' as 'pdf'.
fplot, to define a function f(x) as fx to feed into fplot, use fx = @(x) f(x);
rand, try to use uniform_realizations=rand(N,1); where N is the number of realizations.

For submission, please submit a ZIP that includes a PDF of your derivation in Q1&2 and plots(plot both the pdf function(analytical) and histogram(numerical) in one figure so that they can be compared with each other) in Q3; a MATLAB/Python3 file which includes runnable code for Q3.

HW5 HINTs

Q1:

Take (d)(ii) as an example, $P (Y < X^{2}) = P (Y < X^{2} \cap Y \in [0,1] \cap X \in [0,1]) = P (Y \in [0, X^{2}] \cap X \in [0,1]) = \int_{0}^{1} \int_{0}^{x^{2}} f_{X, Y} (x, y) d y d x$

Alternatively, you can solve it by: $P (Y < X^{2}) = P (Y < X^{2} \cap Y \in [0,1] \cap X \in [0,1]) = P (Y \in [0,1] \cap X \in [\sqrt{Y}, 1]) = \int_{0}^{1} \int_{\sqrt{y}}^{1} f_{X, Y} (x, y) d x d y$

Draw a Graph to determine the low/upper bound of an integral is recommended: check Q1_d(iii) here

Q2:

$C o v (X, Y) = 0 \Rightarrow U n c o r r e l a t e d$

$E (X Y) = 0 \Rightarrow O r t h o g o n a l$

$f_{X, Y} (x, y) = f_{X} (x) \cdot f_{Y} (y) \Rightarrow I n d e p e n d e n t$

Q3:

(b):

$P (Y > X | x) = P (Y > X \cap Y \in (0,1) \cap X \in (0,1) | X = x) = P (Y \in (X, 1) | X = x) = \int_{x}^{1} f_{Y | X} (y | x) d y$

(c):

$P (Y > X) = P (Y > X \cap Y \in (0,1) \cap X \in (0,1)) = P (Y \in (X, 1) \cap X \in (0,1)) = \int_{0}^{1} \int_{x}^{1} f_{X, Y} (x, y) d y d x = \int_{0}^{1} \int_{x}^{1} f_{Y | X} (y | x) f_{X} (x) d y d x = \int_{0}^{1} (\int_{x}^{1} f_{Y | X} (y | x) d y) \cdot f_{X} (x) d x = \int_{0}^{1} (P (Y > X | x)) \cdot f_{X} (x) d x$

HW6 HINTs

Q1:

(b):

Use either density method(with an auxiliary variable) or distribution method. (See Lec27-29)

Please note that fz(z) should have different expressions for z in range (0,1) and (1,2). Check details here

(c):

The convolution can be defined as: $f_{X} * f_{Y} (z) = \int_{- \infty}^{\infty} f_{X} (x) f_{Y} (z - x) d x$

You can first prove the inequality for z in the range of (0,1), after showing that, the proof is already finished.

For z in the range of (0,1), the integral value range of x is determined by:

$0 \leq z \leq 1,0 \leq x \leq 1,0 \leq z - x \leq 1$ because for marginal PDF of x and y, the variable x and y are required to be in the range of (0,1). Your goal is to solve the range of x in terms of z.

Q2:

(a):

Define U:={the time when the packet has arrived on one of the paths}, T1:={the time when the packet has arrived on path1}, T2:={the time when the packet has arrived on path2}, by knowing that T1 and T2 are independent, you can first find CDF of U by:

$F_{U} (u) = P r (U \leq u) = P r (m i n {T_{1}, T_{2}} \leq u) = P r ((T_{1} \leq u) \cup (T_{2} \leq u)) = P r (T_{1} \leq u) + P r (T_{2} \leq u) - P r ((T_{1} \leq u) \cap (T_{2} \leq u)) = P r (T_{1} \leq u) + P r (T_{2} \leq u) - P r (T_{1} \leq u) P r (T_{2} \leq u) = F_{T_{1}} (u) + F_{T_{2}} (u) - F_{T_{1}} (u) F_{T_{2}} (u)$

Then take the derivative of u to get PDF of U.

(b):

Define V:={the time when the packet has arrived on both of the paths}, by knowing that T1 and T2 are independent, you can first find CDF of V by:

$F_{V} (v) = P r (V \leq v) = P r (m a x {T_{1}, T_{2}} \leq v) = P r ((T_{1} \leq v) \cap (T_{2} \leq v)) = P r (T_{1} \leq v) P r (T_{2} \leq v) = F_{T_{1}} (v) F_{T_{2}} (v)$

Then take the derivative of v to get PDF of V.

(c):

By using the PDF of U and V, solve E[V-U]=E[V]-E[U]

Q3

(b):

helpful formulas:

$V a r [a X + b Y] = a^{2} V a r [X] + b^{2} V a r [Y] + 2 a b C o v [X, Y]$

$ρ_{v w} = \frac{C o v (V, W)}{σ_{v} σ_{w}} = \frac{E [(V - V) (W - W)]}{σ_{v} σ_{w}}$

(d) and (e):

Method1: By having the joint PDF of V and W from (c), solve marginal PDF of W, and then solve the conditional PDF of V given W by:

$f_{V | W} (v | w) = \frac{f_{V, W} (v, w)}{f_{W} (w)}$

Method2 (Simpler): By having E[V], E[W], Var[V], Var[W], and correlation coefficient of V and W from (a)(b)(c), you should be able to utilize the formula in Lec30 Page4 to solve the conditional PDF of V given W.

Q4

(b):

$P_{e} = P r (e r r o r) = P r (f a l s e a l a r m) P r (H_{0}) + (1 - P r (D e t e c t i o n)) P r (H_{1})$

Q5

Solve the following:

$E [X], E [Y], σ_{x}, σ_{y}, ρ_{x y}, f_{Y | X} (y | x)$

Then, find the LMMSE, MMSE, MAP for "Y given X" by using formulas in Lec32-33.

Comp2 HINTs

For 1(c), doing a convolution of pdfs for X1, X2 and X3 is recommended.

Alternatively, consider distribution method as shown below.

From 1(a) and (b), we know that: $Z_{3} = 2 (X_{1} + X_{2} + X_{3}) - 3$

We also know that: $S_{3} = X_{1} + X_{2} + X_{3}$

I recommend solving the pdf of $S_{3}$ , then solve the pdf of $Z_{3}$

$F_{S_{3}} (s) = P r (S_{3} \leq s) = P r (X_{1} + X_{2} + X_{3} \leq s)$

where $X_{1}, X_{2}, X_{3} \in (0,1)$

If you sketch the possible values of $(X_{1}, X_{2}, X_{3})$ , you will see they are in a cube.

The probability of $X_{1} + X_{2} + X_{3} \leq s$ is the remaining volume of the cube after cutting by the plane $X_{1} + X_{2} + X_{3} = s$

Some useful geometry facts from stackexchange, please don't simple copy the answer from there, because what we need is a little different.

After solving $F_{S_{3}} (s)$ , take the derivate to have $f_{S_{3}} (s)$ , then solve $f_{Z_{3}} (z)$ by "function of a random variable" using either density method or distribution method. The transformation function is $z = g (s) = 2 s - 3$

For coding, similar to Computer Problem1. Use either MATLAB or Python.

WHAT TO SUBMIT: please submit multiple files to Brightspace(just upload multiple files, do NOT ZIP, this will help me quickly grade):

(1) your derivation for Q1

(2) Figure1 (rememeber to plot (i)(ii)(iii) in one Figure): (i) a plot with the pdf function from Q1(c)(the real pdf) / (ii) histogram(numerical solution, n=3) for Q2 / (iii) a plot of a gaussian function with zero-mean&unit-variance (approximation solution)

(3) Figure2 (rememeber to plot (i)(ii) in one Figure): (i) a plot of histogram (numerical solution, n=30) for Q2 / (ii) a plot of a gaussian function with zero-mean&unit-variance(approximation solution)

(4) a printout result generated from your code for Q3.

(1)(2)(3)(4) can be in a single PDF file

(5) Your code file (*.py or *.m)

Final Review

link

Spring 2024 Midterm1 Lecture Notes Review

link

Spring 2024 Lecture 30 Notes

link