Getting Started

This user guide covers all of OpenModelica Microgrids Gym (OMG) toolbox features by topic area. Each of the following steps is introduced in a more detailed way in the different chapters of this users guide.

First step is to create the microgrid, which is the environment for training reinforcement learning agents in power electronic-driven microgrids. In addition, the OMG toolbox can be used for pure simulation and classical control purpose using OpenModelica models with a Python interface.

Each microgrid model is built in the open source software OpenModelica and can be easily adapted.

For the transfer to Python, it needs to get exported as a Functional Mock-up Unit (FMU).

The creation process of the FMU is shown here. It is used to build a gym environment like in the examples from OpenAI Gym. In OMG, the gym environment is defined for example in (examples/two_inverter_static_droop_control.py).

After creating the environment, the network can be simulated in Python. On the one hand, it is possible to test predefined, static controller designs like described here.

However, the main function of this toolbox is to apply reinforcement learning approaches by utilizing the OMG interface for optimal microgrid control as shown in this example.

Basic Examples

To get an idea how the toolbox works and how to set-up an environment or an agent, two very basic examples are shown below. Both use the network presented above, but only the first inverter will be used.

Creating an environment

Following is a minimal example how to set-up an run an environment. Necessary is the definition of model inputs, in this case the three phases of the first inverter. The model outputs will be shown as simulation results, and the model path is the relative location of the FMU file, which contains the network. For any other simulation parameters, for example the step-size, default values will be used.

For the initialisation, the environment needs to be reseted, and env.render will define the output plots. The simulation will perform 1000 steps. A different random number will be provided to every of the three previously defined model_inputs. Afterwards, the inductor currents of the LC-filter “lc1”shown in the figure above will be plotted, which should result in three increasing and due to the random function noisy lines.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
import gym

if __name__ == '__main__':
    env = gym.make('openmodelica_microgrid_gym:ModelicaEnv-v1',
                   max_episode_steps=None,
                   net='../net/net.yaml',
                   model_path='../omg_grid/grid.network.fmu')

    env.reset()
    for _ in range(1000):
        env.render()
        env.step(env.action_space.sample())  # take a random action
    env.close()

Creating an agent and a runner

Additionally to the environment, an an agent will be created and a runner will be used. The runner class will take care of initializing and termination of agents and environments, as well as the execution of multiple episodes. The class will handle all information exchange between agent and environment like presented in the high level code architecture shown below:

Since the inputs are used for both the agent and the environment, they are defined in advance. Although the Agent gets information of the environment, in this small example, its action is still a random number.

The environment is the same as above. Afterwards, the agent and the runner get defined, and the runner runs for one episode.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
import gym
import numpy as np
import pandas as pd

from openmodelica_microgrid_gym import Agent, Runner


class RndAgent(Agent):
    def act(self, obs: pd.Series) -> np.ndarray:
        return self.env.action_space.sample()


if __name__ == '__main__':
    env = gym.make('openmodelica_microgrid_gym:ModelicaEnv-v1',
                   net='../net/net.yaml',
                   model_path='../omg_grid/grid.network.fmu')

    agent = RndAgent()
    runner = Runner(agent, env)

    runner.run(1)