信息安全原理与数学基础(四)

基本信息

授课教师：qz

Lecture 10 概率论

实质内容可以不听回头自己看
有手有脑得个八九十分不是问题
来听就一定能听懂；不来听也能懂
作业 = 没有意义的东西

研究随机事件发生的可能性的应用数学

集合论 Set Theory

• 并集 Union $\cup$
• 交集 Intersection $\cap$
• 补集 Complement $A^{C}$
• 互斥 mutually Exclusive
• 穷尽 Collectively Exhaustive
• 分割 Partition

 

Applying Set Theory to Probability

• 随机实验 Random Experience
• 样本空间 Sample Space
• 事件 Events

 

Probability Axioms

• A1: $P[A]\ge0$
• A2: $P[S]=1$
• A3: mutually exclusive events $P[A_1\cup A_2\cup…]=P[A_1]+P[A_2]+…$
• $P[A\cup B]=P[A]+P[B]-P[A\cap B]$

 

Discrete Sample Space

$S=\lbrace{a_1,a_2,…a_n}\rbrace$

$P[\lbrace a_i\rbrace]=1/n$

 

Conditonal Probability

Defination

$P[A|B]=P[AB]/P[B]$

 

Theorem

• $P[A|B]>0$
• $P[B|B]=1$
• If $A_i$ is the partition of A, then $P[A|B]=P[A_1|B]+P[A_2|B]+…$

 

Partitions & the Law of Total Probability

If the partition is $B = \{B_1,B_2,…,B_n\}$, and $C_i=A\cap B_i$, then $A=C_1\cup C_2\cup …\cup C_n$

Bayel’s Law

$P[B|A]=\displaystyle\frac{P[A|B]P[B]}{P[A]}$

 

Independence

A and B are independent if only if $P(A\cap B)=P(A)P(B)\iff P(A|B)=P(A),P(B|A)=P(B)$

Independence & Mutually Exclusive

independence ans mutually exclusive are not synonyms

only when $P(A)P(B) = 0$, Ind = M.E.

 

Random Variables

$X\in S$

$S_X$: random variable range

map the sample outcomes $s$ to the corresponding value of the random variable $X$

 

Discrete Random Variables

Probablity Mass Function

Defination: $P_X(x)= P[X=x]$

 

Classical Distribution

Name	Meaning	PMF	Expected Value	Variance
Bernoulli(p)	one test, result is 0, or 1	$\begin{cases}1-p&,x=0\\p&,x=1\end{cases}$	$p$	$p(1-p)$
Geometric(p)	the number of tests that result occurs 1 time	$p(1-p)^{x-1},x=1,2,...$	$\displaystyle\frac{1}{p}$	$\displaystyle\frac{1-p}{p^2}$
Binomial(p)	the number of result occurs in n times of tests	$\dbinom{k}{n}p^k(1-p)^{n-k},k=0,1,2,...$	$np$	$np(1-p)$
Pascal(k, p)	the number of tests when the result occurs $k$ times	$\displaystyle\binom{x-1}{k-1}p^k(1-p)^{x-k}$	$\displaystyle\frac{k}{p}$	$\displaystyle\frac{k(1-p)}{p^2}$
Discrete Uniform(k, l)	in range $[k,l+1)$, all events have equal probability	$\displaystyle\frac{1}{(l+1)-k}$	$\displaystyle\frac{(l+1)+k-1}{2}$	$\displaystyle\frac{((l+1)-k-1)((l+1)-k+1)}{12}$
Poisson(a)	the number of events occuring in a fixed interval of time if each occurs with a known average rate $a$ and independently	$\displaystyle\frac{a^xe^{-a}}{x!}$	$a$	$a$

Expected Value: $E[X]=\mu_X=\displaystyle\sum_{x\in S_X} xP_X(x)$
Variance Value: $Var[X]=E[(X-\mu_X)^2]=E(X^2)-\mu_X^2$
Standard Deviation: $\sigma_X=\sqrt{Var[X]}$

Cumulative Distribution Function (CDF)

Defination: $F_X(x)=P[X\le x]=\displaystyle\sum_{x_i\le x} P[X=x_i]$

Derived Random Variable

$Y=g(x),E[Y]=\displaystyle\sum_{x\in S_X} g(x)P_X(x)$

• $E[aX+b]=aE[X]+b$
• $Var[aX+b]=a^2Var[X]$

Continuous Random Variables

CDF: $F_X(x)=P[X\le x]$

• $P[x_1\le X\le x_2]=\displaystyle\int_{x_1}^{x_2}f_X(x)\mathrm{d}x=F_X(x_2)-F_X(x_1)$

PDF: $f_X(x)=\displaystyle\frac{\mathrm{d}F_X(x)}{\mathrm{d}x}$

• $\displaystyle\int_{-\infty}^{+\infty}f_X(x)\mathrm{d}x=1$

Uniform Random Variables

X is a uniform (a, b), PDF: $f_X(x)=\displaystyle\frac{1}{b-a}$

CDF: $F_X(x)=(x-a)/(b-x),x\in(a,b)$

$E[X]=(a+b)/2$

$Var[X]=(b-a)^2/12$

Gaussian / Normal Random Variables

X is a Gaussian, PDF: $f_X(x)=\displaystyle\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}$

CDF: $F_X(x)=\phi(\displaystyle\frac{x-\mu}{\sigma})$

Deifine: $\phi(x)=\displaystyle\frac{1}{\sqrt{2\pi}}\int_{-\infty}^xe^{-\frac{t^2}{2}}\mathrm{d}t$

$E[X]=\mu$

$Var[X]=\sigma^2$

Standard Normal Random Variables

Gaussian Random Variables when $\mu=0,\sigma=1$

X is a Standard Normal, PDF: $f_X(x)=\displaystyle\frac{1}{\sqrt{2\pi}}e^{-\frac{x^2}{2}}$

CDF: $F_X(x)=\phi(x)=\displaystyle\frac{1}{\sqrt{2\pi}}\int_{-\infty}^xe^{-\frac{t^2}{2}}\mathrm{d}t$

$E[X]=0$

$Var[X]=1$

In Gaussian($\mu$, $\sigma$), test $x=x_0$, in Standard Normal, $x^\prime=(x_0-\mu)/\sigma$

• $\phi(z)+\phi(-z)=1$

Binary Random Variables

Joint Probability Mass Function(PMF)

$P_{X,Y}(x,y)=P[X=x,Y=y]$

use table to present P(x, y)

Joint CDF

$F_{X,Y}(x,y)=P[X\le x,Y\le y]$

Joint PDF

$f_{X,Y}(x,y)=\displaystyle\frac{\partial^2F_{X,Y}(x,y)}{\partial x\partial y}$

Marginal PMF

$P_X(x)=\displaystyle\sum_{y\in S_Y}P_{X,Y}(x,y)$

$P_Y(y)=\displaystyle\sum_{x\in S_X}P_{X,Y}(x,y)$

Marginal PDF

$f_X(x)=\displaystyle\int_{-\infty}^{\infty}F_{X,Y}(x,y)\mathrm{d}y$

Covariance

$Cov[X,Y]=E[(X-\mu_X)(Y-\mu_Y)]$

• $Cov[X,Y]=E[X\cdot Y]-\mu_x\mu_y$

If 2 variables tend to show

• similar behaviour, cov is positive
• opposite behaviour, cov is negative
• uncorrelated behaviour, cov is zero

Correlation

$\rho$

Independence