Introduction¶
Gym-WiPE (Gym Wireless Plant Environment) is an OpenAI Gym environment for the application of reinforcement learning in the simulation of wireless networked feedback control loops. It is written in Python.
Why Gym-WiPE?¶
Networked control systems often put high requirements on the underlying networks. Off-the-shelf wireless network solutions, however, may not fulfill their needs without further improvements. Reinforcement learning may help to find appropriate policies for radio resource management in control systems for which optimal static resource management algorithms can not easily be determined. This is where Gym-WiPE comes in: It provides simulation tools for the creation of OpenAI Gym reinforcement learning environments that simulate wireless networked feedback control loops.
What’s included?¶
Gym-WiPE features an all-Python wireless network simulator based on SimPy. The Open Dynamics Engine (ODE), more specifically its Python wrapper Py3ODE is integrated for plant simulation. Two Gym environments have been implemented for frequency band assignments yet: A simplistic network-only example and a (yet untested) environment for frequency band assignments to a sensor and a controller of an inverted pendulum. The development of further environments may concern frequency band assignments but is not limited to these as the entire simulation model is accessible from within Python and may be used for arbitrary Gym wireless networked control environments. Control algorithm implementations may profit from the python-control project.
Getting started¶
Environment Setup¶
Gym-WiPE uses pipenv. To install it, run
pip install pipenv
With pipenv installed, you may clone the repository like
git clone https://github.com/bjoluc/gymwipe.git
cd gymwipe
and invoke pipenv to set up a virtual environment and install the dependencies into it:
pipenv install
Optionally, the development dependencies may be installed via
pipenv install --dev
If ODE is used for plant Simulation, it has to be downloaded and built. After
that, make ode
will install Py3ODE and pygame for plant visualizations.
Running the tests¶
The pytest testsuite can be executed via make test
.
Further steps¶
This project lacks tutorials. For now, you can have a look at the API documentation at https://gymwipe.readthedocs.io/en/latest/api/index.html. An example agent implementation for a Gym-WiPE environment is provided in the agents directory.