Python Guide

Python Code Guidelines

No rules here are hard, but they are strong recommendations. The golden rule is to make the code easy to understand and difficult to break.

We use the ruff code formatter and mypy type checker with custom settings. You don't need to check rules enforced by these tools like e.g. line length.

Code Style

Imports

• For functions, import the containing module and call the function using dot notation
• Import classes directly
• Rationale: Using dot notation makes it obvious in the code when a function is external and leads to a more succinct import code. Classes are an exception, they are often used as type hints so less verbosity is preferred.

from mypackage import mymodule
from mypackage.mymodule import MyClass

mymodule.foo()
c = MyClass()

Exceptions from the guidelines:

• We allow direct function import from typing, dataclasses, abc, sqlmodel, sqlalchemy

File layout

• Put important functions at the top
  • Especially, put public functions before private ones
  • Classes should come before global functions
• Rationale: The file should be easy to understand when read top-to-bottom. Important information should be towards the top of the file.

from x import y

class MyClass:
    def __init__(self):
        ...

    def foo(self):
        ...

    def _helper_method(self):
        ...

def bar(...):
    ...

def _helper_func(...):
    ...

Protocols vs ABCs

• It’s ok to use ABCs when appropriate (consider nominal vs structural subtyping).
• Prefer composition over inheritance.
  • Inheritance can introduce bi-directional flow of information between parent and child class.
  • Higher flexibility

TODOs

Use the following format: # TODO({name}, {mm}/{yyyy}): Blah blah

# TODO(Michal, 08/2023): Blah blah

Comments

Use ASD-STE100 Simplified Technical English, avoid bloat. Describe the current state, not the change. Prefer properly formatted comments with a leading capital and punctuation. Final full stop may be omitted for a single sentence.

# This is a proper comment. It spans multiple sentences.
...

# Single sentences can have the final full stop omitted
...

# we don't do this <-
...

Assertions

• Avoid using assertions:
  • Don’t use assertions for user errors, raise an exception instead.
  • Assertions should never fail. If they do it is a bad internal error.
  • Don’t assert that a variable follows its typehint.

They can be used:

• for invariants that are easy to check locally, like a minimum length
• for typing (e.g. non-nullity)
• To document what we expect a particular value to be. Here, expect means that there would need to be some logical error somewhere for the value to be different.
• in a private function when the calling code makes the check

It’s ok to not test assertions as they’re not the intended behavior of a function.

Example:

def get_metrics(values: list[float]) -> Something:
    ...
    if len(values) < 2:
        std = 0 # Or an Error is raised.
    else:
        std = _get_std(values)
    ...

def _get_std(sequence: Sequence[float]) -> float:
    """Gets the standard deviation of a sequence

    Args:
        sequence:
            The sequence to get the std of.
            Must have a length >= 2.

    Returns:
            The standard deviation of a sequence.

    """

    assert len(sequence) >= 2
    return np.std(sequence)

Positional vs. Keyword Arguments

Call functions using keyword arguments, whether or not the arguments are declared keyword-only with *. Do not declare arguments as keyword-only with * in our code.

def fn(hello: str, person: str) -> None:
    ...

fn(hello="Grüezi", person="Bob")

The exception of using positional arguments is allowed for

• if the keyword arguments are not known, e.g. because the function is only given through typing: transform: Callable[[Tensor], Tensor] must be called as transformed = transform(tensor)
• for common standard library functions like e.g. print, math.exp, zip, isinstance, dict.get, etc.
• For common functions from core frameworks such as datetime, pytest, numpy, pytorch, logging, etc.
• For SQL ORM functions like select, col, etc.

__init__.py files

Don't put logic in __init__.py files. They should be preferrably empty, or just expose submodule symbols for convenience.

Exceptions may apply occasionally. Such logic should be accompanied by a comment explaining why it is done.

Docstrings

Function and method-level Docstrings

We use Google style docstrings.

def foo(bar: int) -> str:
    """Summary of function here.

    Longer function information...

    Args:
        bar: Description of the argument `bar`.

    Returns:
        Description of the return value.

    Raises:
        ValueError: Description of the error that may be raised.
    """

Class-level Docstrings

Document the functionality of the class and its public attributes if they are not already documented on the class level.

class SampleClass:
    """Summary of class here.

    Longer class information...

    Attributes:
        likes_spam: A boolean indicating if we like SPAM or not.
        eggs: An integer count of the eggs we have laid.
    """

    name: str
    """An example docstring."""

Tensor Shapes in Docstrings

• The tensor shapes in the Args and Returns sections should be documented through capital letters in parentheses.
• The explanation of these letters happens above in the method/function description.
• Tensor dtypes must be documented unless the dtype is torch.float32 which is assumed to be the default.

    def forward(self, pointclouds: Tensor, index: Tensor) -> Tensor:
        """Propagates a batch of 3D point clouds through the model.
        
        B corresponds to the batch size, N is the number of points in each (padded)
        pointcloud and D is the output dimension of the extracted features.
        
        Args:
            pointclouds: Tensor of shape (B, N, 3).
            index: Binary index for the padded pointcloud of shape (B, N) of dtype
                `torch.bool`.
            
        Returns:
            Reduced point clouds of the shape (B, D).
        """
        self.likes_spam = likes_spam
        self.eggs = 0