Python Postprocessing Tool Structure

Each plugin should implement at least the following Python classes.

  1. A data reader class responsible for loading the data from the simulation directory

  2. A visualizer class that outputs a matplotlib plot

  3. A jupyter-widget class that exposes the parameters of the matplotlib visualizer to the user via other widgets.

The repository directory for PIConGPU Python modules for plugins is lib/python/picongpu/plugins/.

Data Reader

The data readers should reside in the lib/python/picongpu/plugins/data directory. There is a base class in base_reader.py defining the interface of a reader. Each reader class should derive from this class and implement the interface functions not implemented in this class.

class picongpu.plugins.data.base_reader.DataReader(run_directory)

Base class that all data readers should inherit from.

__init__(run_directory)
Parameters

run_directory (string) – path to the run directory of PIConGPU (the path before simOutput/)

_get_for_iteration(iteration, **kwargs)

Get the data for a given iteration.

Returns

  • The data for the requested parameters in a plugin

  • dependent format and type.

get(**kwargs)
Parameters

  • 'iteration' or 'time' should be present in the kwargs. (Either) –

  • both are given, the 'time' argument is converted to (If) –

  • iteration and data for the iteration matching the time (an) –

  • returned. (is) –

  • time (float or np.array of float or None.) – If None, data for all available times is returned.

  • iteration (int or np.array of int or None.) – If None, data for all available iterations is returned.

Returns

  • The data for the requested parameters in a plugin

  • dependent format and type.

get_data_path(**kwargs)
Returns

Return type

A string with the path to the underlying data file.

get_dt()

Return the timestep for the chosen simulation.

get_iterations(**kwargs)
Returns

  • An array with unsigned integers of iterations for which

  • data is available.

get_times(**kwargs)
Returns

  • An array of floats of simulation time steps for which

  • data is available

To shorten the import statements for the readers, please also add an entry in the __init__.py file of the data directory.

Matplotlib visualizer

The visualizers should reside in the lib/python/picongpu/plugins/plot_mpl/ directory. The module names should end on _visualizer.py and the class name should only be Visualizer.

To shorten the import statements for the visualizers, please also add an entry in the __init__.py file of the plot_mpl directory with an alias that ends on “MPL”.

There is a base class for visualization found in base_visualizer.py which already handles the plotting logic. It uses (possibly multiple) instances of the data reader classes for accessing the data. Visualizing data simultaneously for more than one scan is supported by creating as many readers and plot objects as there are simulations for visualization. After getting the data, it ensures that (for performance reasons) a matplotlib artist is created only for the first plot and later only gets updated with fresh data.

class picongpu.plugins.plot_mpl.base_visualizer.Visualizer(reader_cls, run_directories=None, ax=None)

Abstract base class for matplotlib visualizers that implements the visualization logic. Classes that derive from this class need to write their own implementations for the following functions in order to work:

_create_data_reader(self, run_directory) _create_plt_obj(self, ax) _update_plt_obj(self)

Note: When using classes derived from this within jupyter notebooks, use %matplotlib notebook mode.

__init__(reader_cls, run_directories=None, ax=None)

Initialize the reader and data as member parameters.

Parameters

  • run_directories (list of tuples of length 2) – or single tuple of length 2 or list of strings or string. If tuples are specified, they have to be of the following form (sim_label, sim_path) and consist of strings with ‘sim_label’ being a short string used e.g. in plot legends and ‘sim_path’ leading to the run directory of PIConGPU (the path before simOutput/). If only a string or list of strings is given, labels are generated by enumeration. If None, the user is responsible for providing run_directories later on via set_run_directories() before calling visualize().

  • reader_cls (class) – handle of a PIConGPU Data reader class (not string!) which inherited from BaseReader in plugins.data.base_reader.

  • ax (matplotlib.axes) –

_check_and_fix_run_dirs(run_directories)

Check variable type for the run_directories and change to list of tuples if necessary. This can be overridden in derived classes to e.g. restrict to single simulation visualization.

Returns

  • a list of tuples, each of the form

  • (simulation_label, path_to_simulation).

_clean_ax()

clear the ax and if available also clear possible colorbars that are still left from previous plots

_create_plt_obj(idx)

Sets ‘self.plt_obj’ to an instance of a matplotlib.artist.Artist object (or derived classes) created by using ‘self.ax’ which can later be updated by feeding new data into it. Only called on the first call for visualization.

_remove_plt_obj(idx)

Removes the plt_obj at position idx from the current plot and sets it to None so that in a subsequent visualization call the plt_obj is created again.

_update_plt_obj(idx)

Take the ‘self.data’ member, interpret it and feed it into the ‘self.plt_obj’.

adjust_plot(**kwargs)

Executed after the plotting is done for adjusting legends etc…

clear_cbar()

Clear colorbars if present. Should be implemented in derived classes that use colorbars.

set_run_directories(run_directories)

Prepare everything for a fresh start with new run_directories

visualize(**kwargs)

  1. gathers the data for the selected kwargs

  2. removes plot elements for sources which have no data

3. plot the data 3.a Creates the ‘plt_obj’ if it does not exist 3.b Updates the ‘plt_obj’ with the new data. 4. adjusts the plot

All new plugins should derive from this class.

When implementing a new visualizer you have to perform the following steps:

  1. Let your visualizer class inherit from the Visualizer class in base visualizer.py and call the base class constructor with the correct data reader class.

2. Implement the _create_plt_obj(self, idx) function. This function needs to access the plotting data from the self.data member (this is the data structure as returned by the data readers .get(...) function, create some kind of matplotlib artist by storing it in the self.plt_obj member variable at the correct index specified by the idx variable (which corresponds to the data of the simulation at position idx that is passed in construction.

3. Implement the _update_plt_obj(self, idx) function. This is called only after a valid self.plt_obj was created. It updates the matplotlib artist with new data. Therefore it again needs to access the plotting data from the self.data member and call the data update API for the matplotlib artist (normally via .set_data(...).

Jupyter Widget

The widget is essentially only a wrapper around the matplotlib visualizer that allows dynamical adjustment of the parameters the visualizer accepts for plotting. This allows to adjust e.g. species, filter and other plugin-dependent options without having to write new lines of Python code.

The widgets should reside in the lib/python/picongpu/plugins/jupyter_widgets/ directory. The module names should end on _widget.py.

To shorten the import statements for the widgets, please also add an entry in the __init__.py file of the jupyter_widget directory.

There is a base class for visualization found in base_widget.py which already handles most of the widget logic.

class picongpu.plugins.jupyter_widgets.base_widget.BaseWidget(plot_mpl_cls, run_dir_options=None, fig=None, output_widget=None, **kwargs)

Basic widget class that wraps a corresponding plot_mpl visualizer. It handles selection of scans, simulations and iterations. It also allows to expose the parameters of the corresponding plot_mpl visualizer via jupyter widgets to the user. Only classes derived from this base class should be used!

Note: In order to work, those objects should be used in %matplotlib widget mode and interactive plotting should be switched off (by using plt.ioff()).

__init__(plot_mpl_cls, run_dir_options=None, fig=None, output_widget=None, **kwargs)
Parameters

  • run_dir_options (list) – list of tuples with label and filepath

  • plot_mpl_cls (a valid plot_mpl class handle (not string!)) – Specifies the underlying plot_mpl visualizer that will be used.

  • fig (a matplotlib figure instance.(optional)) – If no figure is provided, the widget will create its own figure. Otherwise the widget will draw into the provided figure and will not create its own.

  • output_widget (None or instance of ipywidgets.Output) – used for capturing messages from the visualizers. If None, a new output widget is created and displayed as a child. If not None, it is not displayed but only used for capturing output and the owner of the output is responsible for displaying it.

  • kwargs (options for plotting, passed on to matplotlib commands.) –

_create_sim_dropdown(options)

Provide the widget for selection of simulations. Can be overridden in derived classes if some of those widgets are not necessary. Note: Make sure that no value of the widget is selected initially since otherwise initial plotting after creation of the widget might not work (since the constructor sets the value to the first available which leads to the visualization callback being triggered.)

Returns

Return type

a jupyter widget that allows selection of value(s)

_create_widgets_for_vis_args()

Provide additional UI widget elements that expose the parameters of the underlying plot_mpl visualizer instance that the user should be able to modify.

Returns

  • a dict mapping keyword argument names of the PIC visualizer

  • to the widgets that are used for adjusting those values.

  • Those widgets should be created in this function as well.

  • Note (no callback for plotting needs to be registered, this is done)

  • automatically during construction.

_get_widget_args()
Returns

  • a dict mapping keyword argument names of the PIC visualizer

  • to the values of the corresponding widget elements.

_handle_run_dir_selection_callback(change)

Callback function when user selects a subset of the available simulations.

_init_fig_and_ax(fig, **kwargs)

Creates the figure and the ax as members.

_make_drop_val_compatible(val)

Depending on the type of self.sim_drop we have to assign a single value or a tuple to the self.sim_drop.value. This function converts the ‘val’ in a way to be compatible with the expected type.

_show_run_dir_options_in_dropdown()

Make the labels of the run_dir_options lookup table available for selection as options for the dropdown.

_update_available_sim_times()

Computes the intersection of simulation times that are present in all simulations currently selected. It automatically plots the iteration step that best matches the specified simulation time.

_update_plot_mpl_run_dir(selected_sims)

Passes the selected simulations to the visualizer instance.

Parameters

selected_sims (list) – list of simulation labels which will be translated to their path.

_visualize_callback(change)

Callback that is executed when one of the extra_ui_elements changes or the iteration changes.

capture_output()

Used as decorator for capturing output of member functions.

It allows to switch between visualizations for different simulation times (iterations) and different simulations.

When implementing a new widget you have to perform the following steps:

  1. Let the widget class inherit from the BaseWidget class in base_widget.py and call the base class constructor with the correct matplotlib visualizer class.

from .base_widget import BaseWidget

class NewPluginWidget(BaseWidget):

  1. In the constructor, call the base class constructor with the matplotlib visualizer class as plot_mpl_cls keyword.

    The base class will then create an instance of the visualizer class and delegate the plotting job to it.

# taken from lib/python/picongpu/plugins/jupyter_widgets/energy_histogram_widget.py
from .base_widget import BaseWidget
from picongpu.plugins.plot_mpl import EnergyHistogramMPL

class EnergyHistogramWidget(BaseWidget):
    def __init__(self, run_dir_options, fig=None, **kwargs):

         BaseWidget.__init__(self,
                             EnergyHistogramMPL,
                             run_dir_options,
                             fig,
                             **kwargs)

  1. implement the _create_widgets_for_vis_args(self) function.

    This function has to define jupyter widgets as member variables of the class to allow interactive manipulation of parameters the underlying matplotlib visualizer is capable of handling. It needs to return a dictionary using the parameter names the matplotlib visualizer accepts as keys and the widget members that correspond to these parameters as values.

# taken from lib/python/picongpu/plugins/jupyter_widgets/energy_histogram_widget.py
 def _create_widgets_for_vis_args(self):
     # widgets for the input parameters
     self.species = widgets.Dropdown(description="Species",
                                     options=["e"],
                                     value='e')
     self.species_filter = widgets.Dropdown(description="Species_filter",
                                            options=['all'],
                                            value="all")

     return {'species': self.species,
             'species_filter': self.species_filter}