API Documentation

Documentation for the POMDPs.jl user interface. You can get help for any type or function in the module by typing ? in the Julia REPL followed by the name of type or function. For example:

julia> using POMDPs
julia> ?
help?> reward
search: reward

  reward{S,A,O}(pomdp::POMDP{S,A,O}, state::S, action::A, statep::S)

  Returns the immediate reward for the s-a-s triple

  reward{S,A,O}(pomdp::POMDP{S,A,O}, state::S, action::A)

  Returns the immediate reward for the s-a pair

Index

Types

# POMDPs.POMDP — Type.

Abstract base type for a partially observable Markov decision process.

S: state type
A: action type
O: observation type

source

# POMDPs.MDP — Type.

Abstract base type for a fully observable Markov decision process.

S: state type
A: action type

source

# POMDPs.Solver — Type.

Base type for an MDP/POMDP solver

source

# POMDPs.Policy — Type.

Base type for a policy (a map from every possible belief, or more abstract policy state, to an optimal or suboptimal action)

B: a belief (or policy state) that represents the knowledge an agent has about the state of the system

source

# POMDPs.Updater — Type.

Abstract type for an object that defines how the belief should be updated

B: belief type that parametrizes the updater

A belief is a general construct that represents the knowledge an agent has about the state of the system. This can be a probability distribution, an action observation history or a more general representation.

source

Model Functions

Explicit

These functions return distributions.

# POMDPs.transition — Function.

transition{S,A,O}(problem::POMDP{S,A,O}, state::S, action::A)
transition{S,A}(problem::MDP{S,A}, state::S, action::A)

Return the transition distribution from the current state-action pair

source

# POMDPs.observation — Function.

observation{S,A,O}(problem::POMDP{S,A,O}, action::A, statep::S)

Return the observation distribution for the a-s' tuple (action and next state) and returns it

source

# POMDPs.initial_state_distribution — Function.

initial_state_distribution(pomdp::POMDP)

Returns an initial belief for the pomdp.

source

Generative

These functions should return states, observations, and rewards.

# POMDPs.generate_s — Function.

generate_s{S,A}(p::Union{POMDP{S,A},MDP{S,A}}, s::S, a::A, rng::AbstractRNG)

Return the next state given current state s and action taken a.

source

# POMDPs.generate_o — Function.

generate_o{S,A,O}(p::POMDP{S,A,O}, s::S, a::A, sp::S, rng::AbstractRNG)

Return the next observation given current state s, action taken a and next state sp.

Usually the observation would only depend on the next state sp.

generate_o{S,A,O}(p::POMDP{S,A,O}, s::S, rng::AbstractRNG)

Return the observation from the current state. This should be used to generate initial observations.

source

# POMDPs.generate_sr — Function.

generate_sr{S}(p::Union{POMDP{S},MDP{S}}, s, a, rng::AbstractRNG)

Return the next state sp and reward for taking action a in current state s.

source

# POMDPs.generate_so — Function.

generate_so{S,A,O}(p::POMDP{S,A,O}, s::S, a::A, rng::AbstractRNG)

Return the next state sp and observation o.

source

# POMDPs.generate_or — Function.

generate_or{S,A,O}(p::POMDP{S,A,O}, s::S, a::A, sp::S, rng::AbstractRNG)

Return the observation o and reward for taking action a in current state s reaching state sp.

source

# POMDPs.generate_sor — Function.

generate_sor{S,A,O}(p::POMDP{S,A,O}, s::S, a::A, rng::AbstractRNG)

Return the next state sp, observation o and reward for taking action a in current state s.

source

# POMDPs.initial_state — Function.

initial_state{S}(p::Union{POMDP{S},MDP{S}}, rng::AbstractRNG)

Return the initial state for the problem p.

Usually the initial state is sampled from an initial state distribution.

source

Common

# POMDPs.states — Function.

states(problem::POMDP)
states(problem::MDP)

Returns the complete state space of a POMDP.

source

# POMDPs.actions — Function.

actions(problem::POMDP)
actions(problem::MDP)

Returns the entire action space of a POMDP.

source

# POMDPs.observations — Function.

observations(problem::POMDP)

Return the entire observation space.

source

# POMDPs.reward — Function.

reward{S,A,O}(problem::POMDP{S,A,O}, state::S, action::A)
reward{S,A}(problem::MDP{S,A}, state::S, action::A)

Return the immediate reward for the s-a pair

source

# POMDPs.isterminal — Function.

isterminal{S,A,O}(problem::POMDP{S,A,O}, state::S)
isterminal{S,A}(problem::MDP{S,A}, state::S)

Check if state s is terminal

source

# POMDPs.isterminal_obs — Function.

isterminal_obs{S,A,O}(problem::POMDP{S,A,O}, observation::O)

Check if an observation is terminal.

source

# POMDPs.discount — Function.

discount(problem::POMDP)
discount(problem::MDP)

Return the discount factor for the problem.

source

# POMDPs.n_states — Function.

n_states(problem::POMDP)
n_states(problem::MDP)

Return the number of states in problem. Used for discrete models only.

source

# POMDPs.n_actions — Function.

n_actions(problem::POMDP)
n_actions(problem::MDP)

Return the number of actions in problem. Used for discrete models only.

source

# POMDPs.n_observations — Function.

n_observations(problem::POMDP)

Return the number of actions in problem. Used for discrete models only.

source

# POMDPs.state_index — Function.

state_index{S,A,O}(problem::POMDP{S,A,O}, s::S)
state_index{S,A}(problem::MDP{S,A}, s::S)

Return the integer index of state s. Used for discrete models only.

source

# POMDPs.action_index — Function.

action_index{S,A,O}(problem::POMDP{S,A,O}, a::A)
action_index{S,A}(problem::MDP{S,A}, a::A)

Return the integer index of action a. Used for discrete models only.

source

# POMDPs.obs_index — Function.

obs_index{S,A,O}(problem::POMDP{S,A,O}, o::O)

Return the integer index of observation o. Used for discrete models only.

source

Distribution/Space Functions

# Base.Random.rand — Function.

rand{T}(rng::AbstractRNG, d::Any)

Return a random element from distribution or space d. The sample can be a state, action or observation.

source

# Distributions.pdf — Function.

pdf(d::Any, x::Any)

Value of probability distribution d function at sample x.

source

# StatsBase.mode — Function.

mode(d::Any)

Return the most likely value in a distribution d.

source

# Base.mean — Function.

mean(d::Any)

Return the mean of a distribution d.

source

# POMDPs.dimensions — Function.

dimensions(s::Any)

Returns the number of dimensions in space s.

source

# POMDPs.iterator — Function.

iterator(d::Any)

Return an iterable type (array or custom iterator) that iterates over possible values of d. Values with zero probability may be skipped.

source

Belief Functions

# POMDPs.update — Function.

update{B,A,O}(updater::Updater, belief_old::B, action::A, obs::O)

Returns a new instance of an updated belief given belief_old and the latest action and observation.

source

# POMDPs.initialize_belief — Function.

initialize_belief{B}(updater::Updater{B}, 
                     state_distribution::Any)
initialize_belief{B}(updater::Updater{B}, belief::Any)

Returns a belief that can be updated using updater that has similar distribution to state_distribution or belief.

The conversion may be lossy. This function is also idempotent, i.e. there is a default implementation that passes the belief through when it is already the correct type: initialize_belief{B}(updater::Updater{B}, belief::B) = belief

source

Policy and Solver Functions

# POMDPs.solve — Function.

solve(solver::Solver, problem::POMDP)

Solves the POMDP using method associated with solver, and returns a policy.

source

# POMDPs.updater — Function.

updater(policy::Policy)

Returns a default Updater appropriate for a belief type that policy p can use

source

# POMDPs.action — Function.

action{B}(policy::Policy, x::B)

Fills and returns action based on the current state or belief, given the policy. B is a generalized information state - can be a state in an MDP, a distribution in POMDP, or any other representation needed to make a decision using the given policy.

source

# POMDPs.value — Function.

value{B}(p::Policy, x::B)

Returns the utility value from policy p given the state

source

Simulator

# POMDPs.Simulator — Type.

Base type for an object defining how simulations should be carried out.

source

# POMDPs.simulate — Function.

simulate{S,A,O,B}(simulator::Simulator, problem::POMDP{S,A,O}, policy::Policy{B}, updater::Updater{B}, initial_belief::B) 
simulate{S,A}(simulator::Simulator, problem::MDP{S,A}, policy::Policy, initial_state::S)

Run a simulation using the specified policy.

The return type is flexible and depends on the simulator. For example implementations, see the POMDPToolbox package.

source

Other

The following functions are not part of the API for specifying and solving POMDPs, but are included in the package.

Type Inference

# POMDPs.state_type — Function.

state_type(t::Type)
state_type(p::Union{POMDP,MDP})

Return the state type for a problem type (the S in POMDP{S,A,O}).

type A <: POMDP{Int, Bool, Bool} end

state_type(A) # returns Int

source

# POMDPs.action_type — Function.

action_type(t::Type)
action_type(p::Union{POMDP,MDP})

Return the state type for a problem type (the S in POMDP{S,A,O}).

type A <: POMDP{Bool, Int, Bool} end

action_type(A) # returns Int

source

# POMDPs.obs_type — Function.

obs_type(t::Type)

Return the state type for a problem type (the S in POMDP{S,A,O}).

type A <: POMDP{Bool, Bool, Int} end

obs_type(A) # returns Int

source

Requirements Specification

# POMDPs.check_requirements — Function.

check_requirements(r::AbstractRequirementSet)

Check whether the methods in r have implementations with implemented(). Return true if all methods have implementations.

source

# POMDPs.show_requirements — Function.

show_requirements(r::AbstractRequirementSet)

Check whether the methods in r have implementations with implemented() and print out a formatted list showing which are missing. Return true if all methods have implementations.

source

# POMDPs.get_requirements — Function.

get_requirements(f::Function, args::Tuple)

Return a RequirementSet for the function f and arguments args.

source

# POMDPs.requirements_info — Function.

requirements_info(s::Solver, p::Union{POMDP,MDP}, ...)

Print information about the requirement for solver s.

source

# POMDPs.@POMDP_require — Macro.

@POMDP_require solve(s::CoolSolver, p::POMDP) begin
    PType = typeof(p)
    @req states(::PType)
    @req actions(::PType)
    @req transition(::PType, ::S, ::A)
    s = first(states(p))
    a = first(actions(p))
    t_dist = transition(p, s, a)
    @req rand(::AbstractRNG, ::typeof(t_dist))
end

Create a get_requirements implementation for the function signature and the requirements block.

source

# POMDPs.@POMDP_requirements — Macro.

reqs = @POMDP_requirements CoolSolver begin
    PType = typeof(p)
    @req states(::PType)
    @req actions(::PType)
    @req transition(::PType, ::S, ::A)
    s = first(states(p))
    a = first(actions(p))
    t_dist = transition(p, s, a)
    @req rand(::AbstractRNG, ::typeof(t_dist))
end

Create a RequirementSet object.

source

# POMDPs.@requirements_info — Macro.

@requirements_info ASolver() [YourPOMDP()]

Print information about the requirements for a solver.

source

# POMDPs.@get_requirements — Macro.

@get_requirements f(arg1, arg2)

Call get_requirements(f, (arg1,arg2)).

source

# POMDPs.@show_requirements — Macro.

@show_requirements solve(solver, problem)

Print a a list of requirements for a function call.

source

# POMDPs.@warn_requirements — Macro.

@warn_requirements solve(solver, problem)

Print a warning if there are unmet requirements.

source

# POMDPs.@req — Macro.

@req f( ::T1, ::T2)

Convert a f( ::T1, ::T2) expression to a (f, Tuple{T1,T2}) for pushing to a RequirementSet.

If in a @POMDP_requirements or @POMDP_require block, marks the requirement for including in the set of requirements.

source

# POMDPs.@subreq — Macro.

@subreq f(arg1, arg2)

In a @POMDP_requirements or @POMDP_require block, include the requirements for `f(arg1, arg2) as a child argument set.

source

# POMDPs.implemented — Function.

implemented(function, Tuple{Arg1Type, Arg2Type})

Check whether there is an implementation available that will return a suitable value.

source

Utility Tools

# POMDPs.add — Function.

add(solver_name::AbstractString, v::Bool=true)

Downloads and installs a registered solver with name solver_name. This is a light wrapper around Pkg.clone(), and it does nothing special or different other than looking up the URL.

v is a verbose flag, when set to true, function will notify the user if solver is already installed. This function is not exported, and must be called:

julia> using POMDPs
julia> POMDPs.add("MCTS")

source

# POMDPs.add_all — Function.

add_all()

Downloads and installs all the packages supported by JuliaPOMDP

source

# POMDPs.test_all — Function.

test_all()

Tests all the JuliaPOMDP packages installed on your current machine.

source

# POMDPs.available — Function.

available()

Prints all the available packages in JuliaPOMDP

source

API Documentation

Contents

Index

Types

Model Functions

Explicit

Generative

Common

Distribution/Space Functions

Belief Functions

Policy and Solver Functions

Simulator

Other

Type Inference

Requirements Specification

Utility Tools