ecu-tests/docs/19_frame_io_and_alm_helpers.md
Hosam-Eldin Mostafa 8fa4cf0be1 refactor(tests): layer fixtures by adapter type (mum/psu/babylin)
Restructures tests/hardware/ so that fixture access is controlled by
directory layout — pytest only walks upward through conftest.py files,
so a PSU test physically cannot request fio/alm/nad.

Layout:
- tests/hardware/conftest.py           (unchanged: PSU fixtures)
- tests/hardware/mum/conftest.py       NEW: _require_mum (session autouse),
                                       fio (session), nad (session),
                                       alm (session), _reset_to_off
                                       (function autouse)
- tests/hardware/mum/**                MUM tests + swe5/ + swe6/
- tests/hardware/psu/**                PSU-only tests
- tests/hardware/babylin/**            deprecated BabyLIN E2E

What this removes (was duplicated before):
- 7 verbatim copies of the `fio` fixture
- 6 copies of the `alm` fixture
- 6 copies of the `_reset_to_off` autouse
- 9 inline `if config.interface.type != "mum": pytest.skip(...)` gates

What this changes by design:
- fio / alm / nad scope: module → session. NAD discovery happens once
  per run instead of once per module. The helpers are immutable beyond
  their constructor args, so sharing them is safe; per-test state is
  reset by the autouse `_reset_to_off`.
- test_overvolt.py: `_park_at_nominal` is now `_reset_to_off`, which
  cleanly overrides the conftest's LED-only version (PSU + LED reset).
- test_mum_alm_animation_generated.py keeps a local `_reset_to_off` +
  `_force_off` so its "no AlmTester anywhere" demonstration is preserved
  via fixture override; the local `nad` is also retained because it
  uses the typed `AlmStatus.receive` API.

Docs:
- docs/24_test_wiring.md NEW — describes the three-layer fixture
  topology, lifecycle sequence diagram, helper class wiring, and the
  playbook for adding a new framework component.
- docs/05_architecture_overview.md: add MCF (mum conftest) node to the
  Mermaid diagram + mention it in the components list.
- docs/19_frame_io_and_alm_helpers.md: replace the per-module
  fixture-wiring example with a request-fixtures-by-name snippet plus
  the override pattern.
- Path references swept across docs/02, docs/14, docs/18, docs/20,
  docs/README to point at the new locations.

Verified: pytest --collect-only collects 93 tests with no errors;
30 unit tests and 10 mock-only smoke tests pass; fixture-per-test
output shows PSU tests cannot see fio/alm/nad.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 19:43:09 +02:00

15 KiB

Hardware Test Helpers — FrameIO and AlmTester

Hardware tests under tests/hardware/ use two helper modules to keep test bodies focused on intent rather than bus mechanics:

Module Scope What it gives you
tests/hardware/frame_io.py Generic LDF I/O FrameIO class — send/receive any LDF-defined frame by name, plus pack/unpack and raw-bus escape hatches. Knows nothing about ALM.
tests/hardware/alm_helpers.py ALM_Node domain AlmTester class + constants + pure utilities. Encodes the test patterns specific to the ALM_Req_A / ALM_Status / PWM_Frame / PWM_wo_Comp / Tj_Frame / ConfigFrame set. Built on FrameIO.

The split lets the same FrameIO class be reused by future test suites for other ECUs while keeping ALM-specific knowledge in one place.


1. Three layers of access

FrameIO exposes the same bus three ways. A test picks whichever layer matches its intent.

1.1 High level — by frame and signal name

This is the default for almost every test. The LDF carries the frame ID, length, and signal layout, so the test code never mentions any of those.

fio.send(
    "ALM_Req_A",
    AmbLightColourRed=255, AmbLightColourGreen=0, AmbLightColourBlue=0,
    AmbLightIntensity=255,
    AmbLightUpdate=0, AmbLightMode=0, AmbLightDuration=10,
    AmbLightLIDFrom=alm.nad, AmbLightLIDTo=alm.nad,
)

decoded = fio.receive("ALM_Status")           # full dict of decoded signals
nad     = fio.read_signal("ALM_Status", "ALMNadNo")   # one signal

1.2 Mid level — pack / unpack without I/O

Use this when you want to build a payload, inspect or modify it, and then send it (often via the low-level path).

data = bytearray(fio.pack("ALM_Req_A", AmbLightColourRed=255, ...))
data[7] |= 0x80                                    # tweak a bit by hand
fio.send_raw(fio.frame_id("ALM_Req_A"), bytes(data))

# Decode raw bytes you already have:
decoded = fio.unpack("PWM_Frame", b"\x12\x34..." )

1.3 Low level — raw bus, bypass the LDF

For cases the LDF doesn't describe, or when you need full control.

fio.send_raw(0x12, bytes([0x00] * 8))
rx = fio.receive_raw(0x11, timeout=0.5)            # returns LinFrame | None

1.4 Introspection

fio.frame_id("PWM_Frame")        # 0x12
fio.frame_length("PWM_Frame")    # 8
fio.frame("PWM_Frame")           # raw ldfparser Frame object (cached)
fio.lin                          # underlying LinInterface
fio.ldf                          # LdfDatabase

2. FrameIO API reference

class FrameIO:
    def __init__(self, lin: LinInterface, ldf): ...

    # high level
    def send(self, frame_name: str, **signals) -> None
    def receive(self, frame_name: str, timeout: float = 1.0) -> dict | None
    def read_signal(self, frame_name: str, signal_name: str, *,
                    timeout: float = 1.0, default=None) -> Any

    # mid level
    def pack(self, frame_name: str, **signals) -> bytes
    def unpack(self, frame_name: str, data: bytes) -> dict

    # low level
    def send_raw(self, frame_id: int, data: bytes) -> None
    def receive_raw(self, frame_id: int, timeout: float = 1.0) -> LinFrame | None

    # introspection
    def frame(self, name: str)
    def frame_id(self, name: str) -> int
    def frame_length(self, name: str) -> int

    # injected refs
    @property
    def lin(self) -> LinInterface
    @property
    def ldf(self)

Notes:

  • send() / pack() require every signal in the frame; ldfparser raises if one is missing. Use receive() first if you want to merge a change into the current state.
  • receive() returns None on timeout (rather than raising), so polling loops stay simple.
  • All frame lookups are cached per FrameIO instance — repeated calls to send/receive/frame for the same name don't re-walk the LDF.

3. AlmTester API reference

AlmTester bundles a FrameIO and a NAD, and exposes ALM-specific test patterns. Build it once in a fixture and pass it into tests.

class AlmTester:
    def __init__(self, fio: FrameIO, nad: int): ...

    @property
    def fio(self) -> FrameIO            # the underlying FrameIO
    @property
    def nad(self) -> int                # bound node NAD

    # ALM_Status polling
    def read_led_state(self, timeout: float = STATE_RECEIVE_TIMEOUT) -> int
    def wait_for_state(self, target: int, timeout: float
                       ) -> tuple[bool, float, list[int]]
    def measure_animating_window(self, max_wait: float
                                 ) -> tuple[float | None, list[int]]

    # LED control
    def force_off(self) -> None         # drives mode=0, intensity=0; sleeps to settle

    # PWM assertions (use rgb_to_pwm.compute_pwm() under the hood)
    def assert_pwm_matches_rgb(self, rp, r, g, b, *, label: str = "") -> None
    def assert_pwm_wo_comp_matches_rgb(self, rp, r, g, b, *, label: str = "") -> None

The assert_pwm_* helpers:

  • Read Tj_Frame_NTC (Kelvin), convert to °C, and pass it to compute_pwm so temperature compensation matches what the ECU is applying.
  • Sleep PWM_SETTLE_SECONDS (10 LIN frame periods) before reading PWM frames so the slave's TX buffer has time to refresh.
  • Record both expected and actual values as report properties via the rp(...) helper from tests/conftest.py. The optional label parameter lets you append a suffix when you assert PWM more than once in the same test.

4. Constants and utilities (in alm_helpers)

# ALMLEDState (from LDF Signal_encoding_types: LED_State)
LED_STATE_OFF        = 0
LED_STATE_ANIMATING  = 1
LED_STATE_ON         = 2

# Test pacing — chosen against the 10 ms LIN frame periodicity
STATE_POLL_INTERVAL       = 0.05    # 50 ms between polls (5 LIN periods)
STATE_RECEIVE_TIMEOUT     = 0.2     # per-poll receive timeout
STATE_TIMEOUT_DEFAULT     = 1.0     # default wait_for_state ceiling
PWM_SETTLE_SECONDS        = 0.1     # let the slave refresh PWM_Frame TX buffer
DURATION_LSB_SECONDS      = 0.2     # AmbLightDuration scale: 1 LSB = 200 ms
FORCE_OFF_SETTLE_SECONDS  = 0.4     # pause after the OFF command

# PWM tolerances
KELVIN_TO_CELSIUS_OFFSET = 273.15
PWM_ABS_TOL = 3277      # ±5% of 16-bit full scale
PWM_REL_TOL = 0.05      # ±5% of expected, whichever is larger

# Pure utilities
def ntc_kelvin_to_celsius(ntc_raw: int) -> float
def pwm_within_tol(actual: int, expected: int) -> bool

5. Fixture wiring

fio, alm, nad, and the autouse _reset_to_off are provided by tests/hardware/mum/conftest.py — session-scoped (except _reset_to_off, which must be function-scoped) and shared by every MUM test. A new MUM test just lists them in its signature:

def test_red_at_full(fio, alm, rp):
    fio.send("ALM_Req_A", ...)
    alm.assert_pwm_matches_rgb(rp, 255, 0, 0)

The MUM gate (if config.interface.type != "mum": pytest.skip(...)) is a session-scoped autouse _require_mum in the same conftest — no per-test opt-in needed.

The lin, ldf, and config fixtures are provided globally by tests/conftest.py; see 24_test_wiring.md for the full three-layer fixture topology and the rationale behind the access control.

Overriding the autouse reset

A module that needs a richer baseline (e.g. tests/hardware/mum/test_overvolt.py restores the PSU rail in addition to the LED) overrides _reset_to_off locally — the local definition shadows the conftest's:

@pytest.fixture(autouse=True)
def _reset_to_off(psu, alm):
    apply_voltage_and_settle(psu, NOMINAL_VOLTAGE, validation_time=0.2)
    alm.force_off()
    yield
    apply_voltage_and_settle(psu, NOMINAL_VOLTAGE, validation_time=0.2)
    alm.force_off()

6. Cookbook

Drive the LED to a color and verify both PWM frames

def test_red_at_full(fio, alm, rp):
    r, g, b = 255, 0, 0
    fio.send("ALM_Req_A",
             AmbLightColourRed=r, AmbLightColourGreen=g, AmbLightColourBlue=b,
             AmbLightIntensity=255,
             AmbLightUpdate=0, AmbLightMode=0, AmbLightDuration=10,
             AmbLightLIDFrom=alm.nad, AmbLightLIDTo=alm.nad)

    reached, _, history = alm.wait_for_state(LED_STATE_ON, timeout=1.0)
    assert reached, history
    alm.assert_pwm_matches_rgb(rp, r, g, b)
    alm.assert_pwm_wo_comp_matches_rgb(rp, r, g, b)

Toggle a single ConfigFrame bit and restore it

def test_with_compensation_off(fio, alm, rp):
    try:
        fio.send("ConfigFrame",
                 ConfigFrame_Calibration=0,
                 ConfigFrame_EnableDerating=1,
                 ConfigFrame_EnableCompensation=0,
                 ConfigFrame_MaxLM=3840)
        time.sleep(0.2)
        # ... drive the LED, observe non-compensated PWM ...
    finally:
        fio.send("ConfigFrame",
                 ConfigFrame_Calibration=0,
                 ConfigFrame_EnableDerating=1,
                 ConfigFrame_EnableCompensation=1,
                 ConfigFrame_MaxLM=3840)
        time.sleep(0.2)

Read one signal periodically

nad = fio.read_signal("ALM_Status", "ALMNadNo", timeout=0.5, default=None)
if nad is None:
    pytest.skip("ECU silent")

Build a malformed payload and send it raw

data = bytearray(fio.pack("ALM_Req_A",
                          AmbLightColourRed=0, AmbLightColourGreen=0,
                          AmbLightColourBlue=0, AmbLightIntensity=0,
                          AmbLightUpdate=0, AmbLightMode=0, AmbLightDuration=0,
                          AmbLightLIDFrom=0, AmbLightLIDTo=0))
data[2] = 0xFF                              # corrupt one byte
fio.send_raw(fio.frame_id("ALM_Req_A"), bytes(data))

7. Writing a new test

7.1 Starting point

A heavily-annotated, copyable template lives at tests/hardware/_test_case_template.py. The leading underscore stops pytest from collecting it, so the example bodies don't run on the bench.

Copy it to a new file named test_<feature>.py under tests/hardware/ and edit. The template includes:

  • The standard imports for frame_io and alm_helpers
  • The three module-level fixtures (fio, alm, _reset_to_off) with inline explanations of fixture scope, autouse, and yield
  • Three skeleton bodies (one per common shape — see §7.3)
  • An appendix listing the most-reached-for patterns

7.2 The four-phase test pattern

Every hardware test that mutates ECU state beyond just the LED should follow a SETUP / PROCEDURE / ASSERT / TEARDOWN structure with a try/finally so the teardown runs even when an assertion fails.

def test_xyz(fio, alm, rp):
    """..."""
    # ── SETUP ──────────────────────────────────────
    # Bring the ECU to the exact state THIS test needs, beyond what the
    # autouse reset already gave us. Anything you change here MUST be
    # undone in TEARDOWN below.
    fio.send("ConfigFrame", ConfigFrame_EnableCompensation=0, ...)
    time.sleep(0.2)

    try:
        # ── PROCEDURE ──────────────────────────────
        # The actions whose effects you are validating.
        fio.send("ALM_Req_A", ...)
        reached, _, history = alm.wait_for_state(LED_STATE_ON, timeout=1.0)

        # ── ASSERT ─────────────────────────────────
        # Bus-observable expectations. Use `rp("key", value)` to attach
        # diagnostics to the report, then assert.
        rp("led_state_history", history)
        assert reached, history
        alm.assert_pwm_wo_comp_matches_rgb(rp, r, g, b)

    finally:
        # ── TEARDOWN ───────────────────────────────
        # Always runs. Restores anything SETUP perturbed.
        fio.send("ConfigFrame", ConfigFrame_EnableCompensation=1, ...)
        time.sleep(0.2)

Why this gives you test independence

Pytest runs tests in a deterministic order (the order they appear in the file). Without strict teardown, a failure midway through one test can leave the ECU in a non-default state that breaks every subsequent test — turning a single bug into a cascade. The four-phase pattern prevents that with two layers:

Layer What it covers Where it lives
Common baseline LED → OFF autouse _reset_to_off fixture
Per-test specifics ConfigFrame, schedules, mode flags, anything else the test's own try/finally

The autouse fixture handles the universal baseline so individual tests don't have to think about it; the per-test try/finally handles whatever that specific test mutated.

When you can skip the four phases

If your test only sends a frame and observes the LED state (i.e. the only mutable state involved is something the autouse reset already restores), the explicit SETUP/TEARDOWN sections are dead weight — just write the procedure straight through. Flavor A in the template illustrates this minimal shape.

7.3 Three flavors in the template

Flavor When to use it
A — minimal Test only drives the LED and asserts on PWM/state. The autouse reset is enough.
B — with isolation Test changes any persistent ECU state (ConfigFrame, schedules, NAD, …). Use the try/finally pattern.
C — single-signal probe "Ask the ECU one thing and check the answer." Uses fio.read_signal(...), no state mutation.

Pick the closest one, delete the others, rename the function and fill in the docstring.

7.4 Tests that drive the PSU and observe the LIN bus

For combined PSU + LIN scenarios (overvoltage / undervoltage tolerance, brown-out behaviour, supply transients) there is a dedicated template at tests/hardware/_test_case_template_psu_lin.py. It adds a psu fixture (cross-platform port resolution + safe-off on close), an autouse _park_at_nominal fixture, a wait_for_voltage_status polling helper, and three flavors:

Flavor Demonstrates
A — overvoltage Drive PSU above the OV threshold, expect ALMVoltageStatus = 0x02, restore.
B — undervoltage Symmetric for UV (0x01).
C — sweep Parametrized walk over (V, expected_status) tuples.

For the settling time characterization that feeds these tests' detect timeouts, see tests/hardware/psu/test_psu_voltage_settling.py (opt-in via pytest -m psu_settling).

See docs/14_power_supply.md §6 and §5 (session-managed power) for the full reference and the constants to tune for your firmware.