TN 208 - Lecture 2

8/4/2019 TN 208 - Lecture 2

1/37

TN 208: STOCHASTIC SIGNALS

AND SYSTEMS

Module 1:

Probability and Random Variables

8/4/2019 TN 208 - Lecture 2

2/37

Random Experiments,

Sample Space and Sample Point,

Events, Mutually Exclusive Events,Independent Events.

Probability definition and theorems,

Random variable definition.

Classification of random variables.

To be Covered

8/4/2019 TN 208 - Lecture 2

3/37

To be Covered

Cumulative Distribution Function

(cdf).

Probability Density Function (pdf).

Statistical Averages.

Common Probability Distributionfunctions.

Gaussian random variables.

8/4/2019 TN 208 - Lecture 2

4/37

Probability

Probability implies random experiments.

A random experiment can have many possibleoutcomes; each outcome known as a sample point(a.k.a. elementary event) has some probability assigned.This assignment may be based on measured data orguestmates (equally likely is a convenient and oftenmade assumption).

Sample Space S : a set of all possible outcomes(elementary events) of a random experiment. Finite (e.g., if statement execution; two outcomes) Countable (e.g., number of times a while statement is

executed; countable number of outcomes)

Continuous (e.g., time to failure of a component or signal)

8/4/2019 TN 208 - Lecture 2

5/37

5

Probability

Definition

A probabilistic experiment, or randomexperiment, or simply an experiment, isthe process by which an observation is

made. In probability theory, any action or process that

leads to an observation is referred to as anexperiment.

Examples include: Tossing a pair of fair coins.

Throwing a balanced die.

8/4/2019 TN 208 - Lecture 2

6/37

6

Probability

Definition

The sample space associated with aprobabilistic experiment is the set

consisting of all possible outcomes of theexperiment and is denoted by S. The elements of the sample space are

referred to as sample points.

A discrete sample space is one that containseither a finite or a countable number ofdistinct sample points.

8/4/2019 TN 208 - Lecture 2

7/37

7

Probability

Definition

An event in a discrete sample space Sis acollection of sample points, i.e., any subset of S.In other words, an event is a set consisting of

possible outcomes of the experiment. Definition

A simple event is an event that cannot bedecomposed. Each simple event corresponds to

one and only one sample point. Any event thatcan be decomposed into more than one simpleevent is called a compound event.

8/4/2019 TN 208 - Lecture 2

8/37

8

Probability

Definition

Let A be an event connected with a probabilisticexperiment Eand let Sbe the sample space of E.

The event

Bof nonoccurrence of

Ais called thecomplementary event of A.

This means that the subset Bis the complement Aof A in S.

In an experiment, two or more events are said to beequally likely if, after taking into consideration allrelevant evidences, none can be expected inreference to another.

8/4/2019 TN 208 - Lecture 2

9/379

Probability

8/4/2019 TN 208 - Lecture 2

10/37

10

Probability

Axiomatic Approach

Analyzing the concept of equally likely probability, wesee that three conditions must hold.

1. The probability of occurrence of any event must

be greater than or equal to 0.

2. The probability of the whole sample space mustbe 1.

3. If two events are mutually exclusive, the

probability of their union is the sum of their

respective probabilities. These three fundamental concepts form the basis of

the definition of probability.

8/4/2019 TN 208 - Lecture 2

11/37

11

Probability

8/4/2019 TN 208 - Lecture 2

12/3712

Probability

Let A, B and C be events in the sample space, S, then

8/4/2019 TN 208 - Lecture 2

13/37

MUTUALLY EXCLUSIVE EVENTS

8/4/2019 TN 208 - Lecture 2

14/37

MUTUALLY EXCLUSIVE EVENTS

Events are mutually exclusive if they cannot

happen at the same time.

For example, if we toss a coin, either heads or

tails might turn up, but not heads and tails at

the same time.

Similarly, in a single throw of a die, we can

only have one number shown at the top face.

The numbers on the face are mutually

exclusive events

8/4/2019 TN 208 - Lecture 2

15/37

MUTUALLY EXCLUSIVE EVENTS cont..

IfA and B are mutually exclusive

events then the probability ofA

happening OR the probability ofBhappening is P(A) + P(B).

P(A or B) = P(A) + P(B)

8/4/2019 TN 208 - Lecture 2

16/37

Example 1 What is the probability of a die showing a 2 or

a 5?


8/4/2019 TN 208 - Lecture 2

17/37

Practice The probabilities of three teams A, B and C

winning a badminton competition are

Calculate the probability that

a) either A or B will win

b) either A or B or C will win

c) none of these teams will win

d) neither A nor B will win


8/4/2019 TN 208 - Lecture 2

18/37

Solution/s

c) P(none will win) = 1 P(A or B or C will win)

d) P(neither A nor B will win) = 1 P(either A or B will win)

8/4/2019 TN 208 - Lecture 2

19/37

Independent Events

Events are independent if the outcome of oneevent does not affect the outcome of another.

For example, if you throw a die and a coin, the

number on the die does not affect whether the

result you get on the coin.

IfA and B are independent events, then the

probability ofA happening AND the probability

ofB happening is P(A) P(B).

P(A and B) = P(A) P(B)

8/4/2019 TN 208 - Lecture 2

20/37

Example 1

If a dice is thrown twice, find the probability

of getting two 5s.

8/4/2019 TN 208 - Lecture 2

21/37

Two sets of cards with a letter on each card as

follows are placed into separate bags.

Sara randomly picked one card from each bag.

Find the probability that:

a) She picked the letters J and R.b) Both letters are L.

c) Both letters are vowels.

8/4/2019 TN 208 - Lecture 2

22/37

Solution for no. 2

a) Probability that she picked J and R =

b) Probability that both letters are L =

c) Probability that both letters are vowels =

8/4/2019 TN 208 - Lecture 2

23/37

Example 3

Two fair dice, one colored white and

one colored red, are thrown. Find

the probability that: a) the score on the red die is 2 and

white die is 5.

b) the score on the white die is 1 and

red die is even

8/4/2019 TN 208 - Lecture 2

24/37

Solution for No. 3

a) Probability the red die shows 2

and white die 5 =

b) Probability the white die shows 1

and red die shows an evennumber =

8/4/2019 TN 208 - Lecture 2

25/37

DEPENDENT EVENTS

Events are dependent if the outcome ofone event affects the outcome ofanother. For example, if you draw two

colored balls from a bag and the firstball is not replaced before you draw thesecond ball then the outcome of the

second draw will be affected by theoutcome of the first draw.

8/4/2019 TN 208 - Lecture 2

26/37

DEPENDENT EVENTS cont..

IfA and B are dependent events, then

the probability ofA happening AND the

probability ofB happening, givenA, is

P(A) P(B afterA).

P(A and B) = P(A) P(B afterA)

P(B afterA) can also be written as P(B |A)

then P(A and B) = P(A) P(B |A)

8/4/2019 TN 208 - Lecture 2

27/37

Example 1

A purse contains four P50 bills, five P100bills and three P20 bills. Two bills areselected without the first selection being

replaced.

Find P(P50, then P50)

8/4/2019 TN 208 - Lecture 2

28/37

Solution

There are four P50 bills.

There are a total of twelve bills.

P(P50) = 4/12

The result of the first draw affected the

probability of the second draw.

There are three P50 bills left. There are a total of eleven bills left.

P(P50 after P50) = 3/11

8/4/2019 TN 208 - Lecture 2

29/37

P(P50, then P50) = P(P50) P(P50

after P50) = (4/12)x(3/11)=12/132

The probability of drawing a P50bill and then a P50bill is

8/4/2019 TN 208 - Lecture 2

30/37

Dependent: Practice

A bag contains 6 red, 5 blue and 4

yellow marbles. Two marbles are

drawn, but the first marble drawnis not replaced.

a) Find P(red, then blue). b) Find P(blue, then blue)

8/4/2019 TN 208 - Lecture 2

31/37

Independent Events: Practice

Two fair dice, one colored white and

one colored red, are thrown. Find

the probability that: a) the score on the red die is 2 and

white die is 5.

b) the score on the white die is 1 and

red die is even

8/4/2019 TN 208 - Lecture 2

32/37

Mutually Exclusive Events: Practice

The probabilities of three teams A, B and C

winning a badminton competition are

Calculate the probability that

a) either A or B will win

b) either A or B or C will win c) none of these teams will win

d) neither A nor B will win

8/4/2019 TN 208 - Lecture 2

33/37

Summary

For mutually exclusive eventsPr(A or B) = Pr(AB) = Pr(A)+Pr(B)

For independent eventsPr(A and B)=Pr(A B) = Pr(A)Pr(B)

In general,Pr(A B) = Pr(A)+Pr(B)-Pr(A B)Pr(A B) = Pr(A)+Pr(B)-Pr(A B)

Pr(A B)=Pr(B|A)Pr(A)=Pr(A|B)Pr(B)

8/4/2019 TN 208 - Lecture 2

34/37

Sample Space Worked Examples

Problem 1:Count the number of voice packets containing only

silence produced from a group of N speakers in a 10-ms period.

Solution: Denote sample space by S then,

S = { 0, 1, 2, , N }

Problem 2:A block is transmitted repeatedly over a noisy

channel until an error-free block arrives at the receiver. Count

the number of transmission required.


S = { 1, 2, 3, , }

8/4/2019 TN 208 - Lecture 2

35/37

Sample Space Worked Examples

Problem 3:Measure the time between two message arrivals at a

message center.


S = { t: t 0} = [ 0, )

where t denotes time.

Problem 4:Measure the lifetime of a given computer memory

chip in a specified environment.


S = { t: t 0} = [ 0, )

where t denotes time.

8/4/2019 TN 208 - Lecture 2

36/37

Events Worked examples

Problem 1:Write the values of events for problems in case

study of sample space for following events:

1. No active packets are produced

2. Fewer than 10 transmission are required

3. Less than t0 seconds elapse between message arrivals

4. The chip lasts for more than 1000 hours but fewer than 5000

hour

Solution :

1. No active packets are produced, then

A = { 0 }

8/4/2019 TN 208 - Lecture 2

37/37

Events Worked examples cont..

2.Fewer than 10 transmission are required

A = { 1, 2, , 9 }

3.Less than t0 seconds elapse between message arrivalsA = { t : 0 t < t0 } = [ 0, t0 )

4. The chip lasts for more than 1000 hours but fewer than5000 hour

A = { t : 1000 < t < 5000 } = (1000, 5000 )

Documents

TN 208 - Lecture 2