Interface Extensions
POMDPModelTools contains several interface extensions that provide shortcuts and standardized ways of dealing with extra data.
Programmers should use these functions whenever possible in case optimized implementations are available, but all of the functions have default implementations based on the core POMDPs.jl interface. Thus, if the core interface is implemented, all of these functions will also be available.
Weighted Iteration
Many solution techniques, for example value iteration, require iteration through the support of a distribution and evaluating the probability mass for each value. In some cases, looking up the probability mass is expensive, so it is more efficient to iterate through value => probability pairs. weighted_iterator provides a standard interface for this.
POMDPModelTools.weighted_iterator — Functionweighted_iterator(d)Return an iterator through pairs of the values and probabilities in distribution d.
This is designed to speed up value iteration. Distributions are encouraged to provide a custom optimized implementation if possible.
Example
julia> d = BoolDistribution(0.7)
BoolDistribution(0.7)
julia> collect(weighted_iterator(d))
2-element Array{Pair{Bool,Float64},1}:
true => 0.7
false => 0.3Observation Weight
Sometimes, e.g. in particle filtering, the relative likelihood of an observation is required in addition to a generative model, and it is often tedious to implement a custom observation distribution type. For this case, the shortcut function obs_weight is provided.
POMDPModelTools.obs_weight — Functionobs_weight(pomdp, s, a, sp, o)Return a weight proportional to the likelihood of receiving observation o from state sp (and a and s if they are present).
This is a useful shortcut for particle filtering so that the observation distribution does not have to be represented.
Ordered Spaces
It is often useful to have a list of states, actions, or observations ordered consistently with the respective _index function from POMDPs.jl. Since the POMDPs.jl interface does not demand that spaces be ordered consistently with _index, the states, actions, and observations functions are not sufficient. Thus POMDPModelTools provides ordered_actions, ordered_states, and ordered_observations to provide this capability.
POMDPModelTools.ordered_actions — Functionordered_actions(mdp)Return an AbstractVector of actions ordered according to actionindex(mdp, a).
ordered_actions(mdp) will always return an AbstractVector{A} v containing all of the actions in actions(mdp) in the order such that actionindex(mdp, v[i]) == i. You may wish to override this for your problem for efficiency.
POMDPModelTools.ordered_states — Functionordered_states(mdp)Return an AbstractVector of states ordered according to stateindex(mdp, a).
ordered_states(mdp) will always return a AbstractVector{A} v containing all of the states in states(mdp) in the order such that stateindex(mdp, v[i]) == i. You may wish to override this for your problem for efficiency.
POMDPModelTools.ordered_observations — Functionordered_observations(pomdp)Return an AbstractVector of observations ordered according to obsindex(pomdp, a).
ordered_observations(mdp) will always return a AbstractVector{A} v containing all of the observations in observations(pomdp) in the order such that obsindex(pomdp, v[i]) == i. You may wish to override this for your problem for efficiency.
Info Interface
It is often the case that useful information besides the belief, state, action, etc is generated by a function in POMDPs.jl. This information can be useful for debugging or understanding the behavior of a solver, updater, or problem. The info interface provides a standard way for problems, policies, solvers or updaters to output this information. The recording simulators from POMDPSimulators.jl automatically record this information.
To specify info for a problem (in POMDPs v0.8 and above), one should modify the problem's DDN with the add_infonode function, then return the info in gen. There is an example of this pattern in the docstring below:
Missing docstring for add_infonode. Check Documenter's build log for details.
To specify info from policies, solvers, or updaters, implement the following functions:
POMDPModelTools.action_info — Functiona, ai = action_info(policy, x)Return a tuple containing the action determined by policy 'p' at state or belief 'x' and information (usually a NamedTuple, Dict or nothing) from the calculation of that action.
By default, returns nothing as info.
POMDPModelTools.solve_info — Functionpolicy, si = solve_info(solver, problem)Return a tuple containing the policy determined by a solver and information (usually a NamedTuple, Dict or nothing) from the calculation of that policy.
By default, returns nothing as info.
POMDPModelTools.update_info — Functionbp, i = update_info(updater, b, a, o)Return a tuple containing the new belief and information (usually a NamedTuple, Dict or nothing) from the belief update.
By default, returns nothing as info.