Sim

The sim function provides a convenient way to interact with a POMDP or MDP environment. The first argument is a function that is called at every time step and takes a state (in the case of an MDP) or an observation (in the case of a POMDP) as the argument and then returns an action. The second argument is a pomdp or mdp. It is intended to be used with Julia's do syntax as follows:

pomdp = TigerPOMDP()
history = sim(pomdp, max_steps=10) do obs
println("Observation was $obs.")
return TIGER_OPEN_LEFT
end

This allows a flexible and general way to interact with a POMDP environment without creating new Policy types.

Note: by default, since there is no observation before the first action, on the first call to the do block, obs is nothing.

POMDPSimulators.sim — Function.

sim(polfunc::Function, mdp::MDP)
sim(polfunc::Function, pomdp::POMDP)

Alternative way of running a simulation with a function specifying how to calculate the action at each timestep.

The intended usage is

sim(mdp) do s
    # code that calculates action `a` based on `s` - this is the policy
    # you can also do other things like display something
    return a
end

for an MDP or

sim(pomdp) do o
    # code that does belief updates with observation `o` and calculates `a`
    # you can also do other things like display something
    return a
end

for a POMDP.

Use the simulator keyword argument to specify any simulator to run the simulation. If nothing is specified for the simulator, a HistoryRecorder will be used as the simulator, with all keyword arguments forwarded to it, e.g.

sim(mdp, max_steps=100) do s
    # ...
end

will limit the simulation to 100 steps

source