(Generally) To learn to formulate and analyse relatively simple dynamic probabilistic models. (Especially) To be acquainted with some models of this type which have proved practically usable.

• Differentiate between different types of stochastic processes, and determine which model class that is most relevant for a certain dynamic phenomenon
• Simulate realizations of a Markov- or renewal process
• Classify states of a Markov process and the process itself
• Determine invariant distributions in Markov processes
• Determine simple time varying transition probabilities in Markov processes
• Formulate and solve equations for time to absorption or expected time to absorption in Markov chains.
• Formulate discrete time Markov processes, which arise from different sampling techniques in continuous time processes
• Identify and analyze important special cases of Markov processes, e.g. birth and death processes and fundamental queueing systems

The topics are: Markov chains in discrete and continous time, renewal and Markov renewal processes.

The course is a good supplement for courses in statistics, operations research, time series analysis, image analysis, and control theory.

Please contact the teacher for information on whether this course gives the student the opportunity to prepare a project that may participate in DTU´s Study Conference on sustainability, climate technology, and the environment (GRØN DYST). More information http://www.groendyst.dtu.dk/kursustilmelding.aspx