Examples

This section contains examples of how to use POMDPs.jl. For specific informaiton about the interface and functions used in the examples, please reference the correpsonding area in the documenation or the API Documentation.

The examples are organized by topic. The exmaples are designed to build through each step. First, we have to define a POMDP. Then we need to solve the POMDP to get a policy. Finally, we can simulate the policy to see how it performs. The examples are designed to be exeucted in order. For example, the examples in Simulations Examples assume that the POMDPs defined in the Defining a POMDP section have been defined and we have a policy we would like to simulate that we computed in the Using Different Solvers section.

The GridWorld MDP Tutorial section is a standalone example that does not require any of the other examples.

Outline

• Defining a POMDP
• • CryingBaby Problem Definition
  • QuickPOMDP Interface
  • Explicit Interface
  • Generative Interface
  • Probability Tables
• Using Different Solvers
• • Checking Requirements
  • Offline (SARSOP)
  • Online (POMCP)
  • Heuristic Policy
• Simulations Examples
• • Stepthrough
  • Rollout Simulations
  • Recording Histories
  • Parallel Simulations
• GridWorld MDP Tutorial
• • Dependencies
  • Problem Overview
  • Defining the Grid World MDP Type
  • Grid World State Space
  • Grid World Action Space
  • Grid World Transition Function
  • Grid World Reward Function
  • Grid World Remaining Functions
  • Solving the Grid World MDP (Value Iteration)
  • Solving the Grid World MDP (MCTS)
  • Visualizing the Value Iteration Policy
  • Seeing a Policy In Action