Anti-Poaching Game

This is the environment for the Anti-Poaching game. This is a muli-agent competitive game between poachers and rangers. This is currently on v2, and implements the model as described in the formal model description.

Setting up the Virtual Environment

To have a ready-to-go environment created for you, execute the following line of code

$ source init.sh

This uses python 3.8 (by default) and requires virtualenv to create it. To install the environment with a GPU-enabled version of pytorch enabled, you can supply the full option as follows:

$ source init.sh full

In case you want to install it yourself, either locally or if you want to manage the environment yourself, the following command should do it:

$ pip  install -e .[code,gpu] # For GPU-enabled torch
$ pip  install -e .[code,cpu] # For CPU-only torch

Running tests

Tests can be run from the root folder using pytest as follows:

$ pytest [test/]

Furthermore, tools like coverage can be used to generate more extensive data like branch coverage as followS:

$ coverage --branch -m pytest
$ coverage html 

Using the PettingZoo Environment

The main environment is implemented in anti_poaching_game.py, following the PettingZoo API. The init.sh script will automatically install the package anti_poaching for you (as a pip-editable package), so the following code should run:

from anti_poaching.anti_poaching_v0 import anti_poaching
  
cg = anti_poaching.parallel_env(render_mode="rgb")
done, observations, terminations, truncations = False, None, None, None
action_mask = {
    agent: cg.grid.permitted_movements(agent) for agent in cg.agents
}

while not done:
    # sample the actions for each agent randomly
    actions = {
        agent: cg.action_space(agent).sample(mask=action_mask[agent])
        for agent in cg.agents
    }
    observations, _, terminations, truncations, _ = cg.step(actions)
    action_mask = {
        agent: observations[agent]["action_mask"] for agent in cg.agents
    }
    done = all(
        x or y for x, y in zip(terminations.values(), truncations.values())
    )
    cg.render()

Alternatively, try running the examples from manual_policies.

Examples + RLlib

A few examples are found in the examples folder.

Manual policies

The fixed_policy.py and the random_policy.py show how to use the game using hand-coded policies, or just to show the basic RL loop.

Rllib examples

The examples run MARL algorithms (Policy Gradients, PPO, QMIX) on the developed model using RLlib. All experiments can be launched the central script main.py. This runs an RLlib algorithm (PPO) in Multi-Agent Independent Learning mode for an AntiPoachingGame instance by default. All examples have parameters that can be specified via command line (use --help to see all options); everything is wrapped to provide compatibility with RLlib.

# from the repository root
$ cd examples/rllib_examples
$ python main.py

To see all the configuration options possible, run

$ python main.py --help