To provide a good understanding of the key concepts of stochastic processes in various settings: discrete time and finite state space; discrete time and countable state space; continuous time and countable state space.

Two lectures per week for 10 weeks and fortnightly tutorials.

STATS5026 - Stochastic Processes (M)

Courses prescribed in the Honours or Master's programme to which the student has been admitted.

90-minute, end-of-course examination (100%)

Reassessments are normally available for all courses, except those which contribute to the Honours classification. For non Honours courses, students are offered reassessment in all or any of the components of assessment if the satisfactory (threshold) grade for the overall course is not achieved at the first attempt. This is normally grade D3 for undergraduate students and grade C3 for postgraduate students. Exceptionally it may not be possible to offer reassessment of some coursework items, in which case the mark achieved at the first attempt will be counted towards the final course grade. Any such exceptions for this course are described below. 

The aims of this course are to provide a good understanding of the key concepts of stochastic processes in various settings: discrete time and finite state space; discrete time and countable state space; continuous time and countable state space.

By the end of this course students should be able to:

■ Explain the concept of a homogeneous Markov chain;

■ Define what it means for a matrix to be stochastic and discuss the consequences this has for the eigenvalues;

■ Explain what it means for a state to be absorbing, periodic, persistent, transient, or ergodic;

■ Explain the difference between a stationary and a limiting distribution;

■ Decide whether a Markov chain has a unique limiting distribution;

■ Describe the gambler's ruin problem in terms of a discrete-time Markov chain;

■ Calculate the probability of ruin and the expected duration of a game in the gambler's ruin problem;

■ Explain the difference between a homogeneous and an inhomogeneous linear difference equation and the standard procedure to solve them;

■ Derive the probability distribution of a random walk;

■ Calculate the first return of a symmetric random walk;

■ Define the concept of a homogeneous Poisson process, and derive the form of the distribution of the inter-arrival times;

■ Calculate the expected length and waiting time for a queue in which arrivals form a Poisson process;

■ Define the concepts of a reliability function and a hazard function;

■ Calculate the expected number of renewals in a renewal process;

■ Explain the difference between a discrete-time and a continuous-time Markov chain and explain the concept of a rate matrix;

■ Decide whether a birth-death process has a stationary distribution. 

N/A