SI140: Probability and Statistics

Description

The goal of this course is to provide a carefully motivated, accessible, and interesting introduction to probability, random process, and statistics for electrical and computer engineers. The complexity of the systems encountered in engineering practice calls for an understanding of probability concepts and a facility in the use of probability, for which we shall teach both the basic theoretical concepts and techniques for solving problems that arise in practice.

Textbooks and Optional References

Textbooks:

• Probability, Statistics and Random Processes for Electrical Engineering, by Alberto Leon-Garcia, Pearson, Third Edition, 2008.

References:

• High-Dimensional Probability: An Introduction with Applications in Data Science, by Roman Vershynin, Cambridge University Press, 2018.

• Topics in Random Matrix Theory, by Terence Tao, American Mathematical Society, 2012.

• High-Dimensional Statistics: A Non-Asymptotic Viewpoint, by Martin Wainwright, Cambridge University Press, 2019.

Lectures

1. Basic Probability Theory

   1. Course Introduction, Probability Models

   2. Random Experiments, Set Operations, and Probability Axioms

   3. Probabilities and Counting

   4. Conditional Probability, Independence

   5. Sequential Experiments

2. Single Random Variable

   1. Discrete Random Variables, Probability Mass Function

   2. Expected Value and Moments

   3. Conditional Probability Mass Function

   4. Important Discrete RVs

   5. Continuous RVs, CDF, PDF

   6. Expected value, Important Continuous RVs

   7. Functions of a RV

3. Pairs of Random Variables

   1. Pairs of discrete RV’s

   2. Pairs of continuous RV’s

   3. Independence, Joint Moments

   4. Conditional Probability, Conditional Expectation

   5. Functions of Two RV’s

   6. Multiple Random Variables and Expectation

   7. Jointly Gaussian RV’s

4. Sum of Random Variables and Random Process

   1. Sums of Random Variables; Laws of Large Numbers

   2. Transform methods, Central Limit Theorem

   3. Definition of a Random Process, Specifying a Random Process

   4. Mean and Autocorrelation of Random Processes

   5. Discrete Time Random Processes

   6. Continuous Time Random Processes

   7. Stationary Random Processes

5. Statistics

   1. Samples and Sampling Distributions

   2. Parameter Estimation, Maximum Likelihood Estimation

   3. Confidence Intervals

   4. Hypothesis Testing

   5. Bayesian Decision Method