To provide a good understanding of the key concepts of stochastic processes in various settings: discrete time and finite state space; discrete time and countable state space; continuous time and countable state space.

Two lectures per week for 10 weeks and fortnightly tutorials.

STATS4024 Stochastic Processes

120-minute, end-of-course examination (100%)

To provide a good understanding of the key concepts of stochastic processes in various settings: discrete time and finite state space; discrete time and countable state space; continuous time and countable state space.

By the end of this course students should be able to:

■ Explain the concept of a homogeneous Markov chain;

■ Define what it means for a matrix to be stochastic and discuss the consequences this has for the eigenvalues;

■ Explain what it means for a state to be absorbing, periodic, persistent, transient, or ergodic;

■ Explain the difference between a stationary and a limiting distribution;

■ Decide whether a Markov chain has a unique limiting distribution;

■ Describe the gambler's ruin problem in terms of a discrete-time Markov chain;

■ Calculate the probability of ruin and the expected duration of a game in the gambler's ruin problem;

■ Explain the difference between a homogeneous and an inhomogeneous linear difference equation and the standard procedure to solve them;

■ Derive the probability distribution of a random walk;

■ Calculate the first return of a symmetric random walk;

■ Define the concept of a homogeneous Poisson process, and derive the form of the distribution of the inter-arrival times;

■ Calculate the expected length and waiting time for a queue in which arrivals form a Poisson process;

■ Define the concepts of a reliability function and a hazard function;

■ Calculate the expected number of renewals in a renewal process;

■ Explain the difference between a discrete-time and a continuous-time Markov chain and explain the concept of a rate matrix;

■ Decide whether a birth-death process has a stationary distribution;

■ Read further into one topic related to the course and answer questions on this topic.

None