Spaces and Distributions

Two important components of the definitions of MDPs and POMDPs are spaces, which specify the possible states, actions, and observations in a problem and distributions, which define probability distributions. In order to provide for maximum flexibility spaces and distributions may be of any type (i.e. there are no abstract base types). Solvers and simulators will interact with space and distribution types using the functions defined below.

Spaces

A space object should contain the information needed to define the set of all possible states, actions or observations. The implementation will depend on the attributes of the elements. For example, if the space is continuous, the space object may only contain the limits of the continuous range. In the case of a discrete problem, a vector containing all states is appropriate for representing a space.

The following functions may be called on a space object (Click on a function to read its documentation):

• rand
• length
• dimensions

Distributions

A distribution object represents a probability distribution.

The following functions may be called on a distribution object (Click on a function to read its documentation):

• rand(rng, d) ^[1]
• support
• pdf
• mode
• mean

You can find some useful pre-made distribution objects in Distributions.jl or POMDPModelTools.jl.

^[1] rand(rng::AbstractRNG, d) where d is a distribution object is the only method of rand that is officially part of the POMDPs.jl interface, so it is the only required method for new distributions. However, users may wish to hook into the official julia rand interface to enable more flexible rand calls.