Metrics

Metrics are included in Phantom as a tool for recording properties of agents and environments during the running of training or evaluation episodes, without cluttering up the code in Agent and Environment classes.

Phantom includes three implementations of the base Metric class: SimpleAgentMetric, SimpleEnvMetric and AggregatedAgentMetric. These will likely cover most use-cases, if not custom metric classes can be created.

Phantom provides functionality to record and store metrics through three methods:

When training or evaluating policies using RLlib with the Phantom helper functions, a dictionary of metrics can be passed to the functions. For training the metrics will be logged along with RLlib training metrics to Tensorboard. For evaluation these will be stored in the returned Rollout objects.
When debugging or evaluating environments manually by calling env.step() in a loop (see Environment Execution) the logging of metrics, to the terminal and/or to a log file can be enabled with the TelemetryLogger class configuration methods.
When using a custom Phantom Trainer class, metrics can be provided to the train() method.

Metric

class phantom.metrics.Metric(fsm_stages=None, description=None)[source]

Class for extracting metrics from a phantom.PhantomEnv instance.

Parameters:

fsm_stages (Optional[Sequence[FSMStage]]) – Optional list of FSM stages to filter metric recording on. If None is given metrics will be recorded on all stages when used with an FSM Env. If a list of FSM stages is given, the metric will only be recorded when the Env is in these stages, otherwise a None value will be recorded.
description (Optional[str]) – Optional description string for use in data exploration tools.

abstract extract(env)[source]

Extract and return the current metric value from env.

Parameters:: env (PhantomEnv) – The environment instance.
Return type:: TypeVar(MetricValue)

reduce(values, mode)[source]

Reduce a set of observations into a single representative value.

The default implementation is to return the latest observation.

Parameters:

values (Sequence[TypeVar(MetricValue)]) – Set of observations to reduce.
mode (Literal['train', 'evaluate']) – Whether the metric is being recorded during training or evaluation.

Return type:

TypeVar(MetricValue)

Simple Agent Metric

class phantom.metrics.SimpleAgentMetric(agent_id, agent_property, train_reduce_action='mean', eval_reduce_action='none', fsm_stages=None, description=None)[source]

Simple helper class for extracting single ints or floats from the state of a given agent.

Three options are available for summarizing the values at the end of each episode during training or evaluation:

‘last’ - takes the value from the last step

‘mean’ - takes the mean of all the per-step values

‘sum’ - takes the sum of all the per-step values

During evaluation there is also the option for no value summarizing by using ‘none’.

Parameters:

agent_id (str) – The ID of the agent to record the metric for.
agent_property (str) – The property existing on the agent to record, can be nested (e.g. Agent.property.sub_property).
train_reduce_action (Literal['last', 'mean', 'sum']) – The operation to perform on all the per-step recorded values at the end of the episode (‘last’, ‘mean’ or ‘sum’).
eval_reduce_action (Literal['last', 'mean', 'sum', 'none']) – The operation to perform on all the per-step recorded values at the end of the episode (‘last’, ‘mean’ or ‘sum’, ‘none’).
description (Optional[str]) – Optional description string for use in data exploration tools.

extract(env)[source]

Extract and return the current metric value from env.

Parameters:: env (PhantomEnv) – The environment instance.
Return type:: TypeVar(SimpleMetricValue, int, float)

reduce(values, mode)

Reduce a set of observations into a single representative value.

The default implementation is to return the latest observation.

Parameters:

values (Sequence[TypeVar(SimpleMetricValue, int, float)]) – Set of observations to reduce.
mode (Literal['train', 'evaluate']) – Whether the metric is being recorded during training or evaluation.

Return type:

TypeVar(SimpleMetricValue, int, float)

Simple Env Metric

class phantom.metrics.SimpleEnvMetric(env_property, train_reduce_action='mean', eval_reduce_action='none', fsm_stages=None, description=None)[source]

Simple helper class for extracting single ints or floats from the state of the env.

Three options are available for summarizing the values at the end of each episode during training or evaluation:

‘last’ - takes the value from the last step

‘mean’ - takes the mean of all the per-step values

‘sum’ - takes the sum of all the per-step values

During evaluation there is also the option for no value summarizing by using ‘none’.

Parameters:

env_property (str) – The property existing on the environment to record, can be nested (e.g. Agent.property.sub_property).
train_reduce_action (Literal['last', 'mean', 'sum']) – The operation to perform on all the per-step recorded values at the end of the episode (‘last’, ‘mean’ or ‘sum’).
eval_reduce_action (Literal['last', 'mean', 'sum', 'none']) – The operation to perform on all the per-step recorded values at the end of the episode (‘last’, ‘mean’ or ‘sum’, ‘none’).
description (Optional[str]) – Optional description string for use in data exploration tools.

extract(env)[source]

Extract and return the current metric value from env.

Parameters:: env (PhantomEnv) – The environment instance.
Return type:: TypeVar(SimpleMetricValue, int, float)

reduce(values, mode)

Reduce a set of observations into a single representative value.

The default implementation is to return the latest observation.

Parameters:

values (Sequence[TypeVar(SimpleMetricValue, int, float)]) – Set of observations to reduce.
mode (Literal['train', 'evaluate']) – Whether the metric is being recorded during training or evaluation.

Return type:

TypeVar(SimpleMetricValue, int, float)

Aggregated Agent Metric

class phantom.metrics.AggregatedAgentMetric(agent_ids, agent_property, group_reduce_action='mean', train_reduce_action='mean', eval_reduce_action='none', fsm_stages=None, description=None)[source]

Simple helper class for extracting single ints or floats from the states of a group of agents and performing a reduction operation on the values.

Three options are available for reducing the values from the group of agents:

‘min’ - takes the mean of all the per-step values

‘max’ - takes the mean of all the per-step values

‘mean’ - takes the mean of all the per-step values

‘sum’ - takes the sum of all the per-step values

Three options are available for summarizing the values at the end of each episode during training or evaluation:

‘last’ - takes the value from the last step

‘mean’ - takes the mean of all the per-step values

‘sum’ - takes the sum of all the per-step values

During evaluation there is also the option for no value summarizing by using ‘none’.

Parameters:

agent_ids (Iterable[str]) – The ID’s of the agents to record the metric for.
agent_property (str) – The property existing on the agent to record, can be nested (e.g. Agent.property.sub_property).
train_reduce_action (Literal['last', 'mean', 'sum']) – The operation to perform on all the per-step recorded values at the end of the episode (‘last’, ‘mean’ or ‘sum’).
eval_reduce_action (Literal['last', 'mean', 'sum', 'none']) – The operation to perform on all the per-step recorded values at the end of the episode (‘last’, ‘mean’ or ‘sum’, ‘none’).
description (Optional[str]) – Optional description string for use in data exploration tools.

extract(env)[source]

Extract and return the current metric value from env.

Parameters:: env (PhantomEnv) – The environment instance.
Return type:: TypeVar(SimpleMetricValue, int, float)

reduce(values, mode)

Reduce a set of observations into a single representative value.

The default implementation is to return the latest observation.

Parameters:

values (Sequence[TypeVar(SimpleMetricValue, int, float)]) – Set of observations to reduce.
mode (Literal['train', 'evaluate']) – Whether the metric is being recorded during training or evaluation.

Return type:

TypeVar(SimpleMetricValue, int, float)