Introduction

Gym-WiPE (Gym Wireless Plant Environment) is an OpenAI Gym environment for the application of reinforcement learning in the simulation of wireless networked feedback control loops. It is written in Python.

Why Gym-WiPE?

Networked control systems often put high requirements on the underlying networks. Off-the-shelf wireless network solutions, however, may not fulfill their needs without further improvements. Reinforcement learning may help to find appropriate policies for radio resource management in control systems for which optimal static resource management algorithms can not easily be determined. This is where Gym-WiPE comes in: It provides simulation tools for the creation of OpenAI Gym reinforcement learning environments that simulate wireless networked feedback control loops.

What’s included?

Gym-WiPE features an all-Python wireless network simulator based on SimPy. The Open Dynamics Engine (ODE), more specifically its Python wrapper Py3ODE is integrated for plant simulation. Two Gym environments have been implemented for frequency band assignments yet: A simplistic network-only example and a (yet untested) environment for frequency band assignments to a sensor and a controller of an inverted pendulum. The development of further environments may concern frequency band assignments but is not limited to these as the entire simulation model is accessible from within Python and may be used for arbitrary Gym wireless networked control environments. Control algorithm implementations may profit from the python-control project.

Getting started

Environment Setup

Gym-WiPE uses pipenv. To install it, run

pip install pipenv

With pipenv installed, you may clone the repository like

git clone https://github.com/bjoluc/gymwipe.git
cd gymwipe

and invoke pipenv to set up a virtual environment and install the dependencies into it:

pipenv install

Optionally, the development dependencies may be installed via

pipenv install --dev

If ODE is used for plant Simulation, it has to be downloaded and built. After that, make ode will install Py3ODE and pygame for plant visualizations.

Running the tests

The pytest testsuite can be executed via make test.

Further steps

This project lacks tutorials. For now, you can have a look at the API documentation at https://gymwipe.readthedocs.io/en/latest/api/index.html. An example agent implementation for a Gym-WiPE environment is provided in the agents directory.