POMCP.jl

The Partially Observable Monte Carlo Planning (POMCP) online solver for POMDPs.jl.

Described in

Silver, D., & Veness, J. (2010). Monte-Carlo Planning in Large POMDPs. In Advances in neural information processing systems (pp. 2164–2172). Retrieved from http://discovery.ucl.ac.uk/1347369/

Installation

See README.md on the Github repo.

Documentation

Basics

This implementation of the POMCP solver may be used to solve POMDPs defined according to the POMDPs.jl interface. Note that this is an online solver, so the computation is carried out as the simulation is running (simulations take a long time, but solve takes no time).

For a usage example, see the Basic Usage notebook. For some more (poorly documented) examples, see the Sanity Checks notebook.

Behavior is controlled through two mechanisms: solver options and method specializations.

Belief Updates

By default, POMCP uses an unweighted particle filter for belief updates as discussed in the original paper describing it. However, this implementation can use any Updater to keep track of the belief. A notebook describing the various belief updater options and features can be viewed here: http://nbviewer.jupyter.org/github/JuliaPOMDP/POMCP.jl/blob/master/notebooks/Belief_and_Particle_Filter_Options.ipynb

Tree Visualization

Interactive visualization of the POMCP tree is available in Jupyter notebooks. Run the Display Tree notebook in the notebooks folder for a demonstration, or view it here: http://nbviewer.jupyter.org/github/JuliaPOMDP/POMCP.jl/blob/master/notebooks/Display_Tree.ipynb

In order to display a tree, create a POMCPTreeVisualizer with any BeliefNode. If the last line of a cell in a Jupyter notebook returns a POMCPTreeVisualizer, the output cell will be populated with html and javascript that display the tree. See the documentation for MCTS.jl for more detailed information about the tree and instructions describing how to customize its appearance. In particular, the node_tag and tooltip_tag functions can be overridden to customize how actions and observations are displayed.