Vanilla

The "vanilla" solver is the most basic version of MCTS. It works well with small discrete state and action spaces.

The solver fields are used to specify solver parameters. All of them can be specified as keyword arguments to the solver constructor.

# MCTS.MCTSSolver — Type.

MCTS solver type

Fields:

n_iterations::Int64
    Number of iterations during each action() call.
    default: 100

depth::Int64:
    Maximum rollout horizon and tree depth.
    default: 10

exploration_constant::Float64:
    Specifies how much the solver should explore.
    In the UCB equation, Q + c*sqrt(log(t/N)), c is the exploration constant.
    default: 1.0

rng::AbstractRNG:
    Random number generator

estimate_value::Any (rollout policy)
    Function, object, or number used to estimate the value at the leaf nodes.
    If this is a function `f`, `f(mdp, s, depth)` will be called to estimate the value.
    If this is an object `o`, `estimate_value(o, mdp, s, depth)` will be called.
    If this is a number, the value will be set to that number
    default: RolloutEstimator(RandomSolver(rng))

init_Q::Any
    Function, object, or number used to set the initial Q(s,a) value at a new node.
    If this is a function `f`, `f(mdp, s, a)` will be called to set the value.
    If this is an object `o`, `init_Q(o, mdp, s, a)` will be called.
    If this is a number, Q will be set to that number
    default: 0.0

init_N::Any
    Function, object, or number used to set the initial N(s,a) value at a new node.
    If this is a function `f`, `f(mdp, s, a)` will be called to set the value.
    If this is an object `o`, `init_N(o, mdp, s, a)` will be called.
    If this is a number, N will be set to that number
    default: 0

reuse_tree::Bool:
    If this is true, the tree information is re-used for calculating the next plan.
    Of course, clear_tree! can always be called to override this.
    default: false

enable_tree_vis::Bool:
    If this is true, extra information needed for tree visualization will
    be recorded. If it is false, the tree cannot be visualized.
    default: false

source