Membrane Framework

Membrane Framework

Package: membrane_core ~> 1.3 | Docs: https://hexdocs.pm/membrane_core/ | Module index: https://hexdocs.pm/membrane_core/llms.txt | Demos: https://github.com/membraneframework/membrane_demo | All packages: packages_list.md

How to Approach Tasks

• New component — before writing a new element, check packages_list.md to see if it already exists in an existing plugin; if not, identify subtype (Source/Filter/Sink/Endpoint/Bin), define pads, implement required callbacks (handle_buffer/4 for filters/sinks, handle_demand/5 for manual-flow sources)
• Generating boilerplate — use mix membrane.gen.filter MyApp.MyFilter, mix membrane.gen.source, mix membrane.gen.sink, mix membrane.gen.endpoint, mix membrane.gen.bin, mix membrane.gen.pipeline instead of writing component skeletons by hand
• Choosing element subtype — prefer Filter for transformations (has sensible defaults for stream_format forwarding); use Endpoint only when output is unrelated to input (e.g. a UDP Endpoint); use Source/Sink for pure producers/consumers
• Flow control — default to :auto on all pads; only use :manual when you need fine-grained backpressure control; Almost the only use case of :push are output pads of Sources/Endpoints that cannot control when they produce data, e.g. UDP Source/Endpoint.
• Pipeline topology — use the ChildrenSpec DSL (child/2, get_child/1, via_in/2, via_out/2) (more info: Membrane.ChildrenSpec)
• Static vs dynamic topology — return spec: from handle_init/2 for static pipelines; return additional spec: actions from any callback (e.g. handle_child_notification/4) to grow the topology at runtime
• Naming children — use atoms (:source) for singletons, tuples ({:decoder, track_id}) for multi-instance children of the same type
• Detecting pipeline completion — implement handle_element_end_of_stream/4 in the pipeline to know when a sink's input pad received EOS; then return {[terminate: :normal], state} (doesn't work if sink is a Membrane.Bin - then expect a custom message from the bin in handle_child_notification callback instead, if the bin sends it)
• Dynamic tracks (demuxers, variable inputs) — use the Dynamic Pads Pattern below
• Crash isolation — group children with {spec, group: <name>, crash_group_mode: :temporary}; handle recovery in handle_crash_group_down/3; see Crash Groups guide
• Inserting debug probes — add child(:probe, %Membrane.Debug.Filter{handle_buffer: &IO.inspect(&1, label: :buffer)}) between any two elements to log buffers without changing pipeline logic. You can use different logging functions than IO.inspect/2. More info: Membrane.Debug.Filter.
• Linking children - linked children pads accepted formats must have non-empty intersections
• Debugging — check pad accepted_format compatibility
• Callback context — every callback receives ctx; key fields: ctx.children, ctx.pads, ctx.playback; crash callbacks also have ctx.crash_initiator, ctx.exit_reason, ctx.group_name; see Pipeline.CallbackContext, Bin.CallbackContext, Element.CallbackContext
• Logging — utilize Membrane.Logger instead of Logger in Membrane components; it prepends component path and name to log messages. Requires require Membrane.Logger in the module before calling any logging functions.
• Use mix hex.info <plugin name> when you need to check the newest version of a plugin
• Search for appropriate plugins in packages_list.md before writing one
• Check input and output pad definitions of elements in deps/ (use cat <filename> | grep def_input_pad and cat <filename> | grep def_output pad) to make sure output pad's accepted_stream_format is compatible with accepted_stream_format of the input pad which it is linked to.
• If the accepted_stream_format doesn't match, search for an element which can act as an adapter
• When constructing Membrane Pipeline, lean towards using most powerful Membrane Components, which are Boombox.Bin and Membrane.Transcoder, instead of using many smaller plugins

Common Pitfalls

• Modifying code in deps/ directory - never do that
• Using child/2 for an already-spawned child — child/2 always spawns a new process; use get_child/1 to reference an existing one; duplicating a name raises an error
• Linking static pads outside the spawning spec — static pads must be linked in the same spec that spawns the component; linking them later raises a LinkError
• Not wiring a dynamic bin pad in handle_pad_added/3 — a dynamic bin input pad must be connected to an internal child within 5 seconds or a LinkError is raised
• Heavy work in handle_init/2 — handle_init is synchronous and blocks the parent; move file I/O, network connections, etc. to handle_setup/2
• Producing data before handle_playing/2 — pads are not ready until :playing; don't send buffers from handle_setup/2
• Returning non-spec actions from handle_init/2 — we recommend to return only :spec action from this callback.

Architecture

Pipeline
├── Element (Source/Filter/Sink/Endpoint)  ← leaf, processes data
├── Bin                                     ← dual role: parent (has children) + child (has pads)
│   ├── Element
│   └── Bin                                ← bins nest arbitrarily deep
└── ...

Element subtypes: Source (output only) · Filter (in + out, output is transformed input) · Sink (input only) · Endpoint (in + out, but output might be not related to input)

Pads

Defined on Elements and Bins (not Pipelines) using def_input_pad/2 (Membrane.Element.WithInputPads) and def_output_pad/2 (Membrane.Element.WithOutputPads):

def_input_pad :input, accepted_format: _any
def_output_pad :output, accepted_format: Membrane.RawAudio, flow_control: :auto

• Availability: :always (static, one instance, referenced by atom) or :on_request (dynamic, reference via Pad.ref(:name, id))
• Flow control: :auto (framework manages demand — preferred), :manual (explicit via :demand/:redemand), :push (no demand, risk of overflow)
• One input pad ↔ one output pad only; pads must have compatible accepted_format
• Default pad names :input/:output allow omitting via_in/via_out in specs
• accepted_format matching syntax: _any (accept anything) · Membrane.RawAudio (any struct of that type) · %Membrane.RawAudio{channels: 2} (match specific fields) · %Membrane.RemoteStream{} (unknown/unparsed stream). any_of(pattern1, pattern2, ...) matches if any pattern matches.
• Full pads guide (static vs dynamic, bin pads, lifecycle): Everything about pads