Network

At the core of any Phantom environment is the network. This defines the relationships between all actors and agents, controls who can send messages to who and handles the way in which messages are sent and resolved.

Each actor and agent, or ‘entity’, in the network is identified with a unique ID. Usually a string is used for this purpose. Any entity can be connected to any number of other actors and agents. When two entities are connected, bi-directional communication is allowed. Connected entities also have read-only access to each others View’s allowing the publishing of information to other entities without having to send messages.

Consider a simple environment where we have some agents learn to play a card game. We have several ‘PlayerAgent’s and a single ‘DealerAgent’.

Creating the Network

The Network class is always initialised in the __init__() method of our environments. First we need to gather all our actors and agents into a single list:

players = [PlayerAgent("p1"), PlayerAgent("p2"), PlayerAgent("p3")]

dealer = DealerAgent("d1")

agents = players + [dealer]

Next we create our network with the list of agents. By default a BatchResolver is used as the message resolver. Alternatively a custom Resolver class instance can be provided.

network = ph.Network(agents)

Our agents are now in the network and we must now define the connections. We want to connect all our players to the dealer. In this environment players do not communicate to each other. We can manually add each connection one by one:

network.add_connection("p1", "d1")
network.add_connection("p2", "d1")
network.add_connection("p3", "d1")

Or we can use one of the convenience methods of the Network class. The following examples all acheive the same outcome. Use whichever one works best for your situation.

network.add_connections_from([
    ("d1", "p1"),
    ("d1", "p2"),
    ("d1", "p3"),
])

network.add_connections_between(["d1"], ["p1", "p2", "p3"])

network.add_connections_with_adjmat(
    ["d1", "p1", "p2", "p3"],
    np.array([
        [0, 1, 1, 1],
        [1, 0, 0, 0],
        [1, 0, 0, 0],
        [1, 0, 0, 0],
    ])
)

Accessing the Network

The easiest way to retrieve a single actor/agent from the Network is to use the subscript operator:

dealer = network["d1"]

The Network class also provides three methods for retrieving multiple actors/agents at once:

players = network.get_actors_with_type(PlayerAgent)
dealer = network.get_actors_without_type(PlayerAgent)

odd_players = network.get_actors_where(lambda a: a.id in ["p1", "p3"])

StochasticNetwork

Phantom has a StochasticNetwork class that implements connection sampling once top of the standard Network class where each connection has a strength 0.0 <= x <= 1.0. Every time the network’s reset() method is called connections are created or destroyed randomly, weighted by the connection’s strength.

agents = [
    Agent("a1"),
    Agent("a2"),
    Agent("a3"),
]

network = StochasticNetwork(agents)

# Agent a1 will be connected to agent a2 in 50% of episodes:
network.add_connection("a1", "a2", 0.5)

# Agent a1 will always be connected to agent a3:
network.add_connection("a1", "a3", 1.0)

# This is equivalent as the default connectivity is 1.0:
network.add_connection("a1", "a3")

# Agent a2 will never be connected to agent a3. There is no real reason to do this:
network.add_connection("a2", "a3", 0.0)

# This samples the connectivities, creating new combinations of connections:
network.reset()

# This has a 50% chance of returning True:
print(network.has_edge("a1", "a2"))

Message Resolution

The process of resolving messages is configurable by the user by choosing or implementing a Resolver class. The default provided resolver class is the BatchResolver and should adequately cover most typical use-cases. It works as follows:

1. Agents send messages and the network checks that the message is being sent along a valid connection before passing the message to the resolver:

2. The BatchResolver gathers messages sent from all agents into batches based on the message’s recipients. The BatchResolver can optionally shuffle the ordering of the messages within each batch using the shuffle_batches argument.

3. The agents receive their messages in batches via the Agent class handle_batch() method. By default, these are then automatically distributed to and handled with the handle_message() method.

4. Any further sent messages are then resolved again and delivered.

Each step of collecting messages from the agents, batching the messages and then delivering the messages is known as a ‘round’. By default the BatchResolver will continue processing infinite rounds until no messages are left to be sent. Alternatively this can be limited using the round_limit argument.