Distributions

POMDPModelTools contains several utility distributions to be used in the POMDPs transition and observation functions. These implement the appropriate methods of the functions in the distributions interface.

This package also supplies showdistribution for pretty printing distributions as unicode bar graphs to the terminal.

Sparse Categorical (`SparseCat`)

SparseCat is a sparse categorical distribution which is specified by simply providing a list of possible values (states or observations) and the probabilities corresponding to those particular objects.

Example: SparseCat([1,2,3], [0.1,0.2,0.7]) is a categorical distribution that assigns probability 0.1 to 1, 0.2 to 2, 0.7 to 3, and 0 to all other values.

POMDPModelTools.SparseCat — Type

SparseCat(values, probabilities)

Create a sparse categorical distribution.

values is an iterable object containing the possible values (can be of any type) in the distribution that have nonzero probability. probabilities is an iterable object that contains the associated probabilities.

This is optimized for value iteration with a fast implementation of weighted_iterator. Both pdf and rand are order n.

source

Implicit

In situations where a distribution object is required, but the pdf is difficult to specify and only samples are required, ImplicitDistribution provides a convenient way to package a sampling function.

POMDPModelTools.ImplicitDistribution — Type

ImplicitDistribution(sample_function, args...)

Define a distribution that can only be sampled from using rand, but has no explicit pdf.

Each time rand(rng, d::ImplicitDistribution) is called,

sample_function(args..., rng)

will be called to generate a new sample.

ImplicitDistribution is designed to be used with anonymous functions or the do syntax as follows:

Examples

ImplicitDistribution(rng->rand(rng)^2)

struct MyMDP <: MDP{Float64, Int} end

function POMDPs.transition(m::MyMDP, s, a)
    ImplicitDistribution(s, a) do s, a, rng
        return s + a + 0.001*randn(rng)
    end
end

td = transition(MyMDP(), 1.0, 1)
rand(td) # will return a number near 2

source

Bool Distribution

POMDPModelTools.BoolDistribution — Type

BoolDistribution(p_true)

Create a distribution over Boolean values (true or false).

p_true is the probability of the true outcome; the probability of false is 1-p_true.

source

Deterministic

POMDPModelTools.Deterministic — Type

Deterministic(value)

Create a deterministic distribution over only one value.

This is intended to be used when a distribution is required, but the outcome is deterministic. It is equivalent to a Kronecker Delta distribution.

source

Uniform

POMDPModelTools.Uniform — Type

Uniform(collection)

Create a uniform categorical distribution over a collection of objects.

The objects in the collection must be unique (this is tested on construction), and will be stored in a Set. To avoid this overhead, use UnsafeUniform.

source

POMDPModelTools.UnsafeUniform — Type