Dumping Metadata

Starting your simulation with

${EXECUTABLE} ${ARGS} --dump-metadata "${OPTIONAL_FILENAME}" --no-start-simulation

will dump a json respresentation of some metadata to ${OPTIONAL_FILENAME}. If no ${OPTIONAL_FILENAME} is given, the default value

"picongpu-metadata.json"

is used. This feature might in a future revision default to being active.

You might want to dump metadata without actually running a simulation. In this case, you can add the –no-start-simulation flag which will make the code skip the actual simulation. If your intention is instead to also run the simulation afterwards, just leave it out and the program will proceed as normal.

The dumping happens after all initialisation work is done immediately before the simulation starts (or is skipped). This implies that

No dynamic information about the simulation can be included (e.g. information about the state at time step 10).

The dumped information will represent the actual parameters used. This implies that the parameters reported by this feature will differ from the ones provided on the commandline if, e.g., automatic adjustment of the grid size (see autoAdjustGrid) kicks in.

Note

Since we are still performing all initialisation work in order to get to the actual parameter values that affect the simulation, it might be necessary to run this feature in a (very short) full batch submission with all resources (like GPUs, etc.) available as for the full simulation run even when running with –no-start-simulation.

The content of the output is a summary of the physical content of the simulated conditions and the format is described below. It is important to note that the structure of the content is aligned with its categorisation in the physical context and not (enforced to be) aligned with the internal code structure.

Note

The scope of this feature is to provide a human- and machine-readable summary of the physical content of the simulated conditions. The envisioned use cases are:

a theoretician quickly getting an overview over their simulation data,

an experimentalist comparing with simulation data or

a database using such information for tagging, filtering and searching.

The following related aspects are out of scope (for the PIConGPU development team):

Reproducibility: The only faithful, feature-complete representation of the input necessary to reproduce a PIConGPU simulation is the complete input directory. If a more standardised and human-readable repesentation is desired, PICMI provides access to a small subset of features.

Completeness: This feature is intended to be fed with well-structured information considered important by the researchers. It is customisable but the design does not allow to ensure any form of completeness with appropriate maintenance effort. We therefore do not aim to describe simulations exhaustively.

(De-)Serialisation: We do not provide infrastructure to fully or partially reconstruct C++ objects from the dumped information.

Standardisation or generalisation of the format: The format and content are the result of our best effort to be useful. Any form of standardisation or generalisation beyond this scope requires a resource commitment from outside the PIConGPU development team. We are happy to implement any such result.

The Format

The created file is a human-readable text file containing valid json the content of which is partially customisable. We do not enforce a particular format but suggest that you stick as closely as possible to the naming conventions from PyPIConGPU and PICMI. For example, the LaserWakefield example dumps the following metadata which might be supplemented with further details as appropriate for the described elements of the simulation:

{
  "incidentField": {
    "XMax": [
      {}
    ],
    "XMin": [
      {}
    ],
    "YMax": [
      {}
    ],
    "YMin": [
      {
        "polarisation": {
          "direction": [
            1.0,
            0.0,
            0.0
          ],
          "type": "circular"
        }
      }
    ],
    "ZMax": [
      {}
    ],
    "ZMin": [
      {}
    ]
  },
  "simulation": {
    "steps": 0
  }
}

Customisation

Content Creation

The main customisation point for adding and adjusting the output related to some typename TObject, say a Laser or the Simulation object itself, is providing a specialisation for picongpu::traits::GetMetadata defaulting to

template<typename TObject>
struct GetMetadata<TObject, std::enable_if_t<providesMetadataAtRT<TObject>>>
{
    // Holds a constant reference to the RT instance it's supposed to report about.
    // Omit this for the CT specialisation!
    TObject const& obj;

    nlohmann::json description() const
    {
        return obj.metadata();
    }
};

template<typename TObject>
struct GetMetadata<TObject, std::enable_if_t<providesMetadataAtCT<TObject>>>
{
    // CT version has no members. Apart from that, the interface is identical to the RT version.

    nlohmann::json description() const
    {
        return TObject::metadata();
    }
};

For example, customising the metadata for MyClass with some runtime (RT) as well as some compiletime (CT) information could look something like this

template<>
struct picongpu::traits::GetMetadata<MyClass>
{
    MyClass const& obj;

    json description() const
    {
        json result = json::object(); // always use objects and not arrays as root
        result["my"]["cool"]["runtimeValue"] = obj.runtimeValue;
        result["my"]["cool"]["compiletimeValue"] = MyClass::MyCompileTimeInformation::value;
        result["somethingElseThatSeemedImportant"] = "not necessarily derived from obj or MyClass";
        return result;
    }
};

This can be put anywhere in the code where MyClass is known, e.g., in a pertinent .param file or directly below the declaration of MyClass itself.

The json object returned from description() is related to the final output via a merge_patch operation but we do not guarantee any particular order in which these are merged. So it is effectively the responsibility of the programmer to make sure that no metadata entries overwrite each other.

Tackling Acess Restrictions

These external classes might run into access restrictions when attempting to dump private or protected members. These can be circumvented in three ways:

If MyClass already implements a .metadata() method, it might already provide the necessary information through that interface, e.g.

template<>
struct picongpu::traits::GetMetadata<SomethingWithPrivateInfo>
{
    SomethingWithPrivateInfo const& obj;

    json description() const
    {
        auto result = obj.metadata();
        // could also depend on the (publicly accessible members of) `obj`:
        result["customisedInfo"] = "Some customised string.";
        return result;
    }
};

This is the preferred way of handling this situation (if applicable). The default implementation of picongpu::traits::GetMetadata forwards to such .metadata() methods anyway.

Declare picongpu::traits::GetMetadata<MyClass> a friend of MyClass, e.g.

struct SomethingWithoutUsefulMetadata : SomethingWithPrivateInfo
{
    friend picongpu::traits::GetMetadata<SomethingWithoutUsefulMetadata>;
    // ...
This way is minimally invasive and preferred if your change is only applicable to your personal situation and is not intended to land into mainline.

Implement/adjust the .metadata() member function of MyClass, e.g.

struct SomethingWithRTInfo
{
    int info = 0;

    json metadata() const
    {
        auto result = json::object();
        result["info"] = info;
        return result;
    }
};
This method is preferred if your change is general enough to make it into the mainline. If so, you are invited to open a pull request (see CONTRIBUTING.md for more details). It is also the approach used to provide you with default implementations to build upon.

Content Registration

If you are not only adjusting existing output but instead you are adding metadata to a class that did not report any in the past, this class must register itself before the simulation starts. Anything that experiences some form of initialisation at runtime, e.g., plugins should register themselves after their initialisation. To stick with the example, a plugin could add

addMetadataOf(*this);

at the end of its pluginLoad() method (see the Simulation class or an example).

Classes that only affect compiletime aspects of the program need to be registered in include/picongpu/param/metadata.param by extending the compiletime list MetadataRegisteredAtCT. Remember: Their specialisation of picongpu::traits::GetMetadata does not hold a reference.

Classes that get instantiated within a running simulation (and not in the initialisation phase) cannot be included (because they are dynamic information, see above) unless their exact state could be forseen at compile time in which case they can be handled exactly as compiletime-only classes.

Metadata Handling Via Policies

It is sometimes convenient to have different strategies for handling metadata at hand which can be applied to independent of the exact content. Despite them not being formal entities in the code, an approach via policies can come in handy. For example, the AllowMissingMetadata policy can wrap any CT type in order to handle cases when no metadata is available. This is its implementation:

template<typename TObject>
struct AllowMissingMetadata
{
    // it's probably a nice touch to provide this, so people don't need a lot of
    // template metaprogramming to get `TObject`
    using type = TObject;
};

template<typename TObject>
struct GetMetadata<AllowMissingMetadata<TObject>> : GetMetadata<TObject>
{
    nlohmann::json description() const
    {
        return handle(GetMetadata<TObject>::description());
    }

    static nlohmann::json handle(nlohmann::json const& result)
    {
        // okay, we've found metadata, so we return it
        return result;
    }

    static nlohmann::json handle(detail::ReturnTypeFromDefault<TObject> const& result)
    {
        // also okay, we couldn't find metadata, so we'll return an empty object
        return nlohmann::json::object();
    }
};

Another example is the categorisation of different incident fields which – by themselves – cannot report from which direction they are incident (see the GetMetadata trait). The IncidentFieldPolicy provides a strategy to gather all pertinent metadata and assemble the incidentField subtree of the output by providing the necessary context.

Handling Custom Types

The nlohmann-json library in use allows to serialise arbitrary types as described here. As an example, we have implemented a serialisation for pmacc::math::Vector in GetMetadata.