Parallel
POMDPSimulators contains a utility for running many Monte Carlo simulations in parallel to evaluate performance. The basic workflow involves the following steps:
- Create a vector of
Sim
objects, each specifying how a single simulation should be run. - Use the
run_parallel
orrun
function to run the simulations. - Analyze the results of the simulations contained in the
DataFrame
returned byrun_parallel
.
Example
An example can be found in the POMDPExamples Package.
Sim objects
Each simulation should be specified by a Sim
object which contains all the information needed to run a simulation, including the Simulator
, POMDP
or MDP
, Policy
, Updater
, and any other ingredients.
POMDPSimulators.Sim
— TypeSim(m::MDP, p::Policy[, initialstate]; kwargs...)
Sim(m::POMDP, p::Policy[, updater[, initial_belief[, initialstate]]]; kwargs...)
Create a Sim object that contains everything needed to run and record a single simulation, including model, initial conditions, and metadata.
A vector of Sim
objects can be executed with run
or run_parallel
.
Keyword Arguments
rng::AbstractRNG=Random.GLOBAL_RNG
max_steps::Int=typemax(Int)
simulator::Simulator=HistoryRecorder(rng=rng, max_steps=max_steps)
metadata::NamedTuple a named tuple (or dictionary) of metadata for the sim that will be recorded, e.g.
(solver_iterations=500,)`.
Running simulations
The simulations are actually carried out by the run
and run_parallel
functions.
POMDPSimulators.run_parallel
— Functionrun_parallel(queue::Vector{Sim})
run_parallel(f::Function, queue::Vector{Sim})
Run Sim
objects in queue
in parallel and return results as a DataFrame
.
By default, the DataFrame
will contain the reward for each simulation and the metadata provided to the sim.
Arguments
queue
: List ofSim
objects to be executedf
: Function to process the results of each simulation
This function should take two arguments, (1) the Sim
that was executed and (2) the result of the simulation, by default a SimHistory
. It should return a named tuple that will appear in the dataframe. See Examples below.
Keyword Arguments
show_progress::Bool
: whether or not to show a progress meterprogress::ProgressMeter.Progress
: determines how the progress meter is displayed
Examples
run_parallel(queue) do sim, hist
return (n_steps=n_steps(hist), reward=discounted_reward(hist))
end
will return a dataframe with with the number of steps and the reward in it.
The run
function is also provided to run simulations in serial (this is often useful for debugging). Note that the documentation below also contains a section for the builtin julia run
function, even though it is not relevant here.
Base.run
— Functionrun(queue::Vector{Sim})
run(f::Function, queue::Vector{Sim})
Run the Sim
objects in queue
on a single process and return the results as a dataframe.
See run_parallel
for more information.
Specifying information to be recorded
By default, only the discounted rewards from each simulation are recorded, but arbitrary information can be recorded.
The run_parallel
and run
functions accept a function (normally specified via the do
syntax) that takes the Sim
object and history of the simulation and extracts relevant statistics as a named tuple. For example, if the desired characteristics are the number of steps in the simulation and the reward, run_parallel
would be invoked as follows:
df = run_parallel(queue) do sim::Sim, hist::SimHistory
return (n_steps=n_steps(hist), reward=discounted_reward(hist))
end
These statistics are combined into a DataFrame
, with each line representing a single simulation, allowing for statistical analysis. For example,
mean(df[:reward]./df[:n_steps])
would compute the average reward per step with each simulation weighted equally regardless of length.