目录
某信源发送端有两个符号： x i , i = 1 , 2 , p ( x 1 ) = a x_i,i=1,2,p(x_1)=a xi,i=1,2,p(x1)=a,每秒发出一个符号。接收端有三种符
号： y j , j = 1 , 2 , 3 y_{j},j=1,2,3 yj,j=1,2,3,转移概率矩阵 P = [ 1 / 2 1 / 2 0 1 / 2 1 / 4 1 / 4 ] P=\begin{bmatrix}1/2&1/2&0\\1/2&1/4&1/4\end{bmatrix} P=[1/21/21/21/401/4]。
(1) 计算接收端的平均不确定性；
(2)计算由于噪声产生的不确定性 H(Y/X);
(3)计算信道容量。
(1)
p ( x 1 ) = a , p ( x 2 ) = 1 − a , P ( Y ∣ X ) = [ 1 2 1 2 0 1 2 1 4 1 4 ] p(x_1)=a,p(x_2)=1-a,P(Y\mid X)=\begin{bmatrix}\dfrac12&\dfrac12&0\\\dfrac12&\dfrac14&\dfrac14\end{bmatrix} p(x1)=a,p(x2)=1−a,P(Y∣X)= 21212141041
P ( X Y ) = [ 1 2 a 1 2 a 0 1 2 ( 1 − a ) 1 4 ( 1 − a ) 1 4 ( 1 − a ) ] P(XY)=\begin{bmatrix}\dfrac12a&\dfrac12a&0\\\dfrac12(1-a)&\dfrac14(1-a)&\dfrac14(1-a)\end{bmatrix} P(XY)= 21a21(1−a)21a41(1−a)041(1−a)
p ( y 1 ) = 1 2 a + 1 2 ( 1 − a ) = 1 2 , p ( y 2 ) = 1 2 a + 1 4 ( 1 − a ) = 1 4 + 1 4 a , p ( y 3 ) = 1 4 ( 1 − a ) p(y_1)=\frac12a+\frac12(1-a)=\frac12,p(y_2)=\frac12a+\frac14(1-a)=\frac14+\frac14a,p(y_3)=\frac14(1-a) p(y1)=21a+21(1−a)=21,p(y2)=21a+41(1−a)=41+41a,p(y3)=41(1−a)
接收端的平均不确定性为：
H ( Y ) = 1 2 log ( 2 ) − ( 1 4 + 1 4 a ) log ( 1 4 + 1 4 a ) − ( 1 4 − 1 4 a ) log ( 1 4 − 1 4 a ) H(Y)=\frac{1}{2}\log(2)-(\frac{1}{4}+\frac{1}{4}a)\log(\frac{1}{4}+\frac{1}{4}a)-(\frac{1}{4}-\frac{1}{4}a)\log(\frac{1}{4}-\frac{1}{4}a) H(Y)=21log(2)−(41+41a)log(41+41a)−(41−41a)log(41−41a)
= 3 2 − 1 + a 4 log ( 1 + a ) − 1 − a 4 log ( 1 − a ) =\frac32-\frac{1+a}4\log(1+a)-\frac{1-a}4\log(1-a) =23−41+alog(1+a)−41−alog(1−a) (bit/symbol)
(2)
H ( Y ∣ X ) = ∑ X , Y p ( x y ) log 1 p ( y ∣ x ) = 1 2 a log ( 2 ) + 1 2 a log ( 2 ) + 0 + 1 2 ( 1 − a ) log ( 2 ) + 1 4 ( 1 − a ) log ( 4 ) + 1 4 ( 1 − a ) log ( 4 ) = 3 2 − 1 2 a ( bit/symbol ) \begin{aligned}&H(Y\mid X)=\sum_{X,Y}p(xy)\log\begin{array}{c}1\\p(y\mid x)\end{array}\\&=\frac12a\log(2)+\frac12a\log(2)+0+\frac12(1-a)\log(2)+\frac14(1-a)\log(4)+\frac14(1-a)\log(4)\\&=\frac32-\frac12a(\text{bit/symbol})\end{aligned} H(Y∣X)=X,Y∑p(xy)log1p(y∣x)=21alog(2)+21alog(2)+0+21(1−a)log(2)+41(1−a)log(4)+41(1−a)log(4)=23−21a(bit/symbol)
(3)
I ( X ; Y ) = H ( Y ) − H ( Y ∣ X ) = [ 3 2 − 1 + a 4 log ( 1 + a ) − 1 − a 4 log ( 1 − a ) ] − ( 3 2 − 1 2 a ) I\left(X;Y\right)=H\left(Y\right)-H\left(Y|X\right)=[\frac32-\frac{1+a}4\log(1+a)-\frac{1-a}4\log(1-a)]-(\frac32-\frac12a) I(X;Y)=H(Y)−H(Y∣X)=[23−41+alog(1+a)−41−alog(1−a)]−(23−21a)
C = max p ( x i ) { I ( X ; Y ) } C=\max_{p(x_i)}\{I(X;Y)\} C=p(xi)max{I(X;Y)}
C ( a ) = [ 3 2 − 1 + a 4 log ( 1 + a ) log ( 2 ) − 1 − a 4 log ( 1 − a ) log ( 2 ) ] − ( 3 2 − 1 2 a ) C(a)=[\frac{3}{2}-\frac{1+a}{4}\frac{\log(1+a)}{\log(2)}-\frac{1-a}{4}\frac{\log(1-a)}{\log(2)}]-(\frac{3}{2}-\frac{1}{2}a) C(a)=[23−41+alog(2)log(1+a)−41−alog(2)log(1−a)]−(23−21a)
d d a C ( a ) = 0 \frac d{da}C(a)=0 dadC(a)=0
得到
1 4 ⋅ − ln ( 1 + a ) + ln ( 1 − a ) + 2 ln ( 2 ) ln ( 2 ) = 0 \frac14\cdot\frac{-\ln(1+a)+\ln(1-a)+2\ln(2)}{\ln(2)}=0 41⋅ln(2)−ln(1+a)+ln(1−a)+2ln(2)=0
解得 a = 3 5 a=\frac{3}{5} a=53
所以 C = max { I ( X ; Y ) } = C ( 3 5 ) = 0.161 ( b i t / s y m b o l ) C=\max\{I(X;Y)\}=C(\frac{3}{5})=0.161(bit/symbol) C=max{I(X;Y)}=C(53)=0.161(bit/symbol)