Misc
This is a catch-all section for general notes on PyPIConGPU.
Python Concepts
Some common concepts recurr throughout the code and are not explained in detail when used. This aims to give a non-exhaustive list of such patterns.
They are as close to “standard python” as possible, so using a search engine/asking a python expert should also help you understand them.
Type Checking with Typeguard
Use as defined in PEP 484 “Type Hints”. Briefly:
def greeting(name: str) -> str:
return 'Hello ' + name
Note that these type annotations are not checked by python on their own.
Typechecks are enabled through
typeguard, mostly using the
annotation @typeguard.typechecked
for classes.
This does not check attribute type annotations, see next section.
Typesafe Class Attributes
Class attributes can use type annotations according to PEP 484, but this is not enforced by typeguard.
The solution is to use properties.
In the code you will see:
class SomeObject:
attr = util.build_typesafe_property(int)
Now the following hold:
Accessing
attr
before it has been set will raise an exceptionAssigning
attr
anything other than anint
will raise aTypeError
Note: instead of types you can use specifications provided by
import typing
, e.g.typing.Optional[int]
. Optional vars still have no default, so you have to set them toNone
explicitly if you don’t want to provide them.
Internally the declaration above is expanded to the equivalent of:
@typeguard.typechecked
def getter(self) -> int:
if not hasattr(self, "actual_name_for_var__attr"):
raise AttributeError("variable is not initialized")
return getattr(self, "actual_name_for_var__attr")
@typeguard.typechecked
def setter(self, value: int) -> None:
setattr(self, "actual_name_for_var__attr", value)
class SomeObject:
attr = property(getter, setter)
Map-Filter-Reduce
map()
, filter()
, and reduce()
are “higher-order functions”
and a basis of the functional programming paradigm (as opposed to loops
for iterative processing). You can find an an introduction
here (but any
other intro is probably fine too).
Boils down to:
[2, 4, 6] == list(map(lambda x: 2*x,
[1, 2, 3]))
map()
does not return a list directly and must be cast again. (Python, why?)
map()
does not work on dictionaries, to for this the dictionary is
cast to a set of key-value tuples:
{"a": 3, "b": 0} == dict(map(
lambda kv_pair: (kv_pair[0], len(kv_pair[1])),
{"a": [0, 0, 0], "b": []}.items()))
Note: Do not feel obliged to follow this pattern. It is commonly used, because it allows a concise notation, yet it is very much not mandatory.
Tool Support
This is a short list of tools to aid you with development.
Formatting
PEP 8 (formatting guidelines) compliance:
run from repo root flake8 lib/python/ test/python/
Note: Please obey the line length from PEP 8 even if that is annoying at times, this makes viewing files side-by-side much simpler.
Run Tests
go into test/python/picongpu
,
execute python -m quick
(for tests with compilation python -m compiling
)
Test Coverage
install coverage.py:
pip install coverage
go to tests:
cd test/python/picongpu
execute tests, record coverage:
coverage run --branch -m quick
view reports
for PyPIConGPU:
find ../../../lib/python/picongpu/pypicongpu/ -name '*.py' | xargs coverage report -m
for PICMI:
find ../../../lib/python/picongpu/picmi/ -name '*.py' | xargs coverage report -m
Goal is 100% coverage (missing sections are printed)
For further options see the coverage.py doc.
API Documentation
Document the API using docstrings, which will be rendered by Sphinx AutoAPI automatically. The result is available from the TOC, although the name is generated to be the python module name: picongpu.pypicongpu.
The configuration is placed in Sphinx’s conf.py
,
all classes are traversed by using __all__
defined in every __init__.py
.
Additional checks (completeness etc.) are NOT employed (and not even possible). So pay attention that you document everything you need.
As everything is passed to Sphinx, docstring should be valid reStructuredText. (reStructuredText quick guide from Sphinx)
You may use Sphinx-exclusive directives too,
e.g. :ref:`MARK`
.
To document parameters, return values etc. make them compatible with Sphinx’s autodoc. For details please refer to the respective documentation section.
Note
If there is an reStructuredText syntax error during generation (printed as warning after sphinx invocation make html
) to debug:
visit the respective page
click View page source at the top of the page
(This might be only availble locally, the online version displays Edit on GitHub instead)
An example can be found in the official documentation. The types are implictly given by the type hints, so these can be omitted:
import typeguard
import typing
@typeguard.typechecked
def my_func(a_char: str, num: int) -> typing.List[str]:
"""
build list with "triangle" of chars
Accepts a single char and builds a list where the n-th element contains a
string consisting of n times the given char.
The list has the given total length.
This is an example for a section of the PyPIConGPU doc,
see: :ref:`pypicongpu-misc-apidoc`.
to show that all Sphinx directives are valid.
Because this is reStructuredText, (nested) lists require separate
paragraphs:
- an item
- another item
- nested 1
- nested 2
- continuation of top level
:param a_char: character to use, e.g. "."
:param num: max, e.g. "3"
:raises AssertionError: on a_char being not a single character
:raises AssertionError: if num < 0
:return: [".", "..", "..."]
"""
assert 1 == len(a_char)
assert 0 <= num
if 0 == num:
return []
return my_func(a_char, num - 1) + [num * a_char]
Note that :raises TypeError:
is omitted too,
because all functions are checked for types anyways.
This produces the following code section:
- doc_example.my_func(a_char: str, num: int) List[str]
build list with “triangle” of chars
Accepts a single char and builds a list where the n-th element contains a string consisting of n times the given char. The list has the given total length.
This is an example for a section of the PyPIConGPU doc, see: API Documentation. to show that all Sphinx directives are valid. Because this is reStructuredText, (nested) lists require separate paragraphs:
an item
another item
nested 1
nested 2
continuation of top level
- Parameters:
a_char – character to use, e.g. “.”
num – max, e.g. “3”
- Raises:
AssertionError – on a_char being not a single character
AssertionError – if num < 0
- Returns:
[“.”, “..”, “…”]