This module focuses on using Bayesian Networks as tools for probabilistic reasoning and decision-making under uncertainty. Students will learn how to interpret a given network, compute probabilities, and perform inference—both exact and approximate—using techniques such as direct sampling and Gibbs sampling. Emphasis is placed on applying Bayes Nets to answer queries, update beliefs with evidence, and reason efficiently in complex domains.

This module introduces temporal probabilistic models, focusing on how AI systems reason about hidden states that evolve over time. Students will learn to apply inference techniques such as filtering, prediction, smoothing, and the Viterbi algorithm to update beliefs and infer the most likely state sequences from observations. Emphasis is placed on using Hidden Markov Models to perform calculations and interpret how evidence shapes reasoning in dynamic, uncertain environments.

This module introduces how AI agents make optimal decisions in uncertainty environments over time using the framework of Markov Decision Processes. Students will learn how to represent sequential decision problems with states, actions, rewards, and policies, and how to compute optimal behavior using value iteration, policy iteration, and the Bellman equation. Emphasis is placed on selecting actions that maximize expected utility in uncertain, sequential environments.