Documents the new layers introduced over the past several commits.
- docs/19_frame_io_and_alm_helpers.md (new): full reference for the
FrameIO and AlmTester helpers — three access levels (high/mid/low),
full API tables, fixture wiring, cookbook patterns, and §7
describing the four-phase SETUP/PROCEDURE/ASSERT/TEARDOWN test
pattern with the three template flavors plus a §7.4 link to the
PSU+LIN template.
- docs/14_power_supply.md: rewritten and expanded.
§3 cross-platform port resolution (Windows / WSL1 / WSL2 +
usbipd-win / Linux native compatibility table)
§4 auto-detection via idn_substr
§5 session-managed power: contract for tests, must-not list,
what changed in the existing tests
§6 the settle-then-validate pattern: two-delays table (PSU
bench-dependent vs ECU firmware-dependent), copy-paste
example, tuning guidance for ECU_VALIDATION_TIME_S
§6 PSU settling characterization (-m psu_settling)
§7 library API reference table + safe_off_on_close
§9 troubleshooting expanded with WSL2 usbipd-win + dialout
- docs/18_test_catalog.md: voltage-tolerance section refreshed for
the settle-then-validate shape, new "Hardware – PSU settling
(opt-in)" category, new §8 "Hardware-test infrastructure"
documenting conftest.py, frame_io.py, alm_helpers.py,
psu_helpers.py, and both templates.
- docs/05_architecture_overview.md: components list split into
framework core / hardware test layer / artifacts. Mermaid diagram
gained a Hardware-test helpers subgraph showing FrameIO,
AlmTester, rgb_to_pwm, and the templates. Data/control flow
summary describes the session-managed PSU and the helper layer.
- docs/15_report_properties_cheatsheet.md: PSU section split into
per-test (function-scoped rp) and module-scoped (testsuite
property) blocks; added psu_resolved_port, psu_resolved_idn,
psu_settled_s, validation_time_s.
- docs/README.md: links to the new doc 19.
- README.md, TESTING_FRAMEWORK_GUIDE.md: project-structure trees
expanded to show the full current layout — every file and
directory under tests/hardware/ (conftest, helpers, templates,
tests), tests/unit/, config/, docs/, scripts/, and vendor/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
423 lines
15 KiB
Markdown
423 lines
15 KiB
Markdown
# Hardware Test Helpers — `FrameIO` and `AlmTester`
|
|
|
|
Hardware tests under `tests/hardware/` use two helper modules to keep test
|
|
bodies focused on intent rather than bus mechanics:
|
|
|
|
| Module | Scope | What it gives you |
|
|
| --- | --- | --- |
|
|
| [`tests/hardware/frame_io.py`](../tests/hardware/frame_io.py) | **Generic LDF I/O** | `FrameIO` class — send/receive any LDF-defined frame by name, plus pack/unpack and raw-bus escape hatches. Knows nothing about ALM. |
|
|
| [`tests/hardware/alm_helpers.py`](../tests/hardware/alm_helpers.py) | **ALM_Node domain** | `AlmTester` class + constants + pure utilities. Encodes the test patterns specific to the ALM_Req_A / ALM_Status / PWM_Frame / PWM_wo_Comp / Tj_Frame / ConfigFrame set. Built on `FrameIO`. |
|
|
|
|
The split lets the same `FrameIO` class be reused by future test suites for
|
|
other ECUs while keeping ALM-specific knowledge in one place.
|
|
|
|
---
|
|
|
|
## 1. Three layers of access
|
|
|
|
`FrameIO` exposes the same bus three ways. A test picks whichever layer
|
|
matches its intent.
|
|
|
|
### 1.1 High level — by frame and signal name
|
|
|
|
This is the default for almost every test. The LDF carries the frame ID,
|
|
length, and signal layout, so the test code never mentions any of those.
|
|
|
|
```python
|
|
fio.send(
|
|
"ALM_Req_A",
|
|
AmbLightColourRed=255, AmbLightColourGreen=0, AmbLightColourBlue=0,
|
|
AmbLightIntensity=255,
|
|
AmbLightUpdate=0, AmbLightMode=0, AmbLightDuration=10,
|
|
AmbLightLIDFrom=alm.nad, AmbLightLIDTo=alm.nad,
|
|
)
|
|
|
|
decoded = fio.receive("ALM_Status") # full dict of decoded signals
|
|
nad = fio.read_signal("ALM_Status", "ALMNadNo") # one signal
|
|
```
|
|
|
|
### 1.2 Mid level — pack / unpack without I/O
|
|
|
|
Use this when you want to build a payload, inspect or modify it, and then
|
|
send it (often via the low-level path).
|
|
|
|
```python
|
|
data = bytearray(fio.pack("ALM_Req_A", AmbLightColourRed=255, ...))
|
|
data[7] |= 0x80 # tweak a bit by hand
|
|
fio.send_raw(fio.frame_id("ALM_Req_A"), bytes(data))
|
|
|
|
# Decode raw bytes you already have:
|
|
decoded = fio.unpack("PWM_Frame", b"\x12\x34..." )
|
|
```
|
|
|
|
### 1.3 Low level — raw bus, bypass the LDF
|
|
|
|
For cases the LDF doesn't describe, or when you need full control.
|
|
|
|
```python
|
|
fio.send_raw(0x12, bytes([0x00] * 8))
|
|
rx = fio.receive_raw(0x11, timeout=0.5) # returns LinFrame | None
|
|
```
|
|
|
|
### 1.4 Introspection
|
|
|
|
```python
|
|
fio.frame_id("PWM_Frame") # 0x12
|
|
fio.frame_length("PWM_Frame") # 8
|
|
fio.frame("PWM_Frame") # raw ldfparser Frame object (cached)
|
|
fio.lin # underlying LinInterface
|
|
fio.ldf # LdfDatabase
|
|
```
|
|
|
|
---
|
|
|
|
## 2. `FrameIO` API reference
|
|
|
|
```python
|
|
class FrameIO:
|
|
def __init__(self, lin: LinInterface, ldf): ...
|
|
|
|
# high level
|
|
def send(self, frame_name: str, **signals) -> None
|
|
def receive(self, frame_name: str, timeout: float = 1.0) -> dict | None
|
|
def read_signal(self, frame_name: str, signal_name: str, *,
|
|
timeout: float = 1.0, default=None) -> Any
|
|
|
|
# mid level
|
|
def pack(self, frame_name: str, **signals) -> bytes
|
|
def unpack(self, frame_name: str, data: bytes) -> dict
|
|
|
|
# low level
|
|
def send_raw(self, frame_id: int, data: bytes) -> None
|
|
def receive_raw(self, frame_id: int, timeout: float = 1.0) -> LinFrame | None
|
|
|
|
# introspection
|
|
def frame(self, name: str)
|
|
def frame_id(self, name: str) -> int
|
|
def frame_length(self, name: str) -> int
|
|
|
|
# injected refs
|
|
@property
|
|
def lin(self) -> LinInterface
|
|
@property
|
|
def ldf(self)
|
|
```
|
|
|
|
Notes:
|
|
|
|
- `send()` / `pack()` require **every** signal in the frame; ldfparser
|
|
raises if one is missing. Use `receive()` first if you want to merge a
|
|
change into the current state.
|
|
- `receive()` returns `None` on timeout (rather than raising), so polling
|
|
loops stay simple.
|
|
- All frame lookups are cached per `FrameIO` instance — repeated calls to
|
|
`send`/`receive`/`frame` for the same name don't re-walk the LDF.
|
|
|
|
---
|
|
|
|
## 3. `AlmTester` API reference
|
|
|
|
`AlmTester` bundles a `FrameIO` and a NAD, and exposes ALM-specific test
|
|
patterns. Build it once in a fixture and pass it into tests.
|
|
|
|
```python
|
|
class AlmTester:
|
|
def __init__(self, fio: FrameIO, nad: int): ...
|
|
|
|
@property
|
|
def fio(self) -> FrameIO # the underlying FrameIO
|
|
@property
|
|
def nad(self) -> int # bound node NAD
|
|
|
|
# ALM_Status polling
|
|
def read_led_state(self, timeout: float = STATE_RECEIVE_TIMEOUT) -> int
|
|
def wait_for_state(self, target: int, timeout: float
|
|
) -> tuple[bool, float, list[int]]
|
|
def measure_animating_window(self, max_wait: float
|
|
) -> tuple[float | None, list[int]]
|
|
|
|
# LED control
|
|
def force_off(self) -> None # drives mode=0, intensity=0; sleeps to settle
|
|
|
|
# PWM assertions (use rgb_to_pwm.compute_pwm() under the hood)
|
|
def assert_pwm_matches_rgb(self, rp, r, g, b, *, label: str = "") -> None
|
|
def assert_pwm_wo_comp_matches_rgb(self, rp, r, g, b, *, label: str = "") -> None
|
|
```
|
|
|
|
The `assert_pwm_*` helpers:
|
|
|
|
- Read `Tj_Frame_NTC` (Kelvin), convert to °C, and pass it to `compute_pwm`
|
|
so temperature compensation matches what the ECU is applying.
|
|
- Sleep `PWM_SETTLE_SECONDS` (10 LIN frame periods) before reading PWM
|
|
frames so the slave's TX buffer has time to refresh.
|
|
- Record both expected and actual values as report properties via the
|
|
`rp(...)` helper from `tests/conftest.py`. The optional `label`
|
|
parameter lets you append a suffix when you assert PWM more than once
|
|
in the same test.
|
|
|
|
---
|
|
|
|
## 4. Constants and utilities (in `alm_helpers`)
|
|
|
|
```python
|
|
# ALMLEDState (from LDF Signal_encoding_types: LED_State)
|
|
LED_STATE_OFF = 0
|
|
LED_STATE_ANIMATING = 1
|
|
LED_STATE_ON = 2
|
|
|
|
# Test pacing — chosen against the 10 ms LIN frame periodicity
|
|
STATE_POLL_INTERVAL = 0.05 # 50 ms between polls (5 LIN periods)
|
|
STATE_RECEIVE_TIMEOUT = 0.2 # per-poll receive timeout
|
|
STATE_TIMEOUT_DEFAULT = 1.0 # default wait_for_state ceiling
|
|
PWM_SETTLE_SECONDS = 0.1 # let the slave refresh PWM_Frame TX buffer
|
|
DURATION_LSB_SECONDS = 0.2 # AmbLightDuration scale: 1 LSB = 200 ms
|
|
FORCE_OFF_SETTLE_SECONDS = 0.4 # pause after the OFF command
|
|
|
|
# PWM tolerances
|
|
KELVIN_TO_CELSIUS_OFFSET = 273.15
|
|
PWM_ABS_TOL = 3277 # ±5% of 16-bit full scale
|
|
PWM_REL_TOL = 0.05 # ±5% of expected, whichever is larger
|
|
|
|
# Pure utilities
|
|
def ntc_kelvin_to_celsius(ntc_raw: int) -> float
|
|
def pwm_within_tol(actual: int, expected: int) -> bool
|
|
```
|
|
|
|
---
|
|
|
|
## 5. Fixture wiring
|
|
|
|
`tests/hardware/test_mum_alm_animation.py` defines two module-scoped
|
|
fixtures plus an autouse reset. The same pattern applies to any new
|
|
hardware test file targeting MUM.
|
|
|
|
```python
|
|
import pytest
|
|
from ecu_framework.config import EcuTestConfig
|
|
from ecu_framework.lin.base import LinInterface
|
|
from frame_io import FrameIO
|
|
from alm_helpers import AlmTester
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def fio(config: EcuTestConfig, lin: LinInterface, ldf) -> FrameIO:
|
|
if config.interface.type != "mum":
|
|
pytest.skip("interface.type must be 'mum' for this suite")
|
|
return FrameIO(lin, ldf)
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def alm(fio: FrameIO) -> AlmTester:
|
|
decoded = fio.receive("ALM_Status", timeout=1.0)
|
|
if decoded is None:
|
|
pytest.skip("ECU not responding on ALM_Status — check wiring/power")
|
|
nad = int(decoded["ALMNadNo"])
|
|
if not (0x01 <= nad <= 0xFE):
|
|
pytest.skip(f"ECU reports invalid NAD {nad:#x} — auto-addressing first")
|
|
return AlmTester(fio, nad)
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _reset_to_off(alm: AlmTester):
|
|
"""Force LED OFF before and after each test so state doesn't leak."""
|
|
alm.force_off()
|
|
yield
|
|
alm.force_off()
|
|
```
|
|
|
|
The `lin`, `ldf`, and `config` fixtures are provided globally by
|
|
`tests/conftest.py` — see [docs/02_configuration_resolution.md](02_configuration_resolution.md)
|
|
for how they are wired.
|
|
|
|
---
|
|
|
|
## 6. Cookbook
|
|
|
|
### Drive the LED to a color and verify both PWM frames
|
|
|
|
```python
|
|
def test_red_at_full(fio, alm, rp):
|
|
r, g, b = 255, 0, 0
|
|
fio.send("ALM_Req_A",
|
|
AmbLightColourRed=r, AmbLightColourGreen=g, AmbLightColourBlue=b,
|
|
AmbLightIntensity=255,
|
|
AmbLightUpdate=0, AmbLightMode=0, AmbLightDuration=10,
|
|
AmbLightLIDFrom=alm.nad, AmbLightLIDTo=alm.nad)
|
|
|
|
reached, _, history = alm.wait_for_state(LED_STATE_ON, timeout=1.0)
|
|
assert reached, history
|
|
alm.assert_pwm_matches_rgb(rp, r, g, b)
|
|
alm.assert_pwm_wo_comp_matches_rgb(rp, r, g, b)
|
|
```
|
|
|
|
### Toggle a single ConfigFrame bit and restore it
|
|
|
|
```python
|
|
def test_with_compensation_off(fio, alm, rp):
|
|
try:
|
|
fio.send("ConfigFrame",
|
|
ConfigFrame_Calibration=0,
|
|
ConfigFrame_EnableDerating=1,
|
|
ConfigFrame_EnableCompensation=0,
|
|
ConfigFrame_MaxLM=3840)
|
|
time.sleep(0.2)
|
|
# ... drive the LED, observe non-compensated PWM ...
|
|
finally:
|
|
fio.send("ConfigFrame",
|
|
ConfigFrame_Calibration=0,
|
|
ConfigFrame_EnableDerating=1,
|
|
ConfigFrame_EnableCompensation=1,
|
|
ConfigFrame_MaxLM=3840)
|
|
time.sleep(0.2)
|
|
```
|
|
|
|
### Read one signal periodically
|
|
|
|
```python
|
|
nad = fio.read_signal("ALM_Status", "ALMNadNo", timeout=0.5, default=None)
|
|
if nad is None:
|
|
pytest.skip("ECU silent")
|
|
```
|
|
|
|
### Build a malformed payload and send it raw
|
|
|
|
```python
|
|
data = bytearray(fio.pack("ALM_Req_A",
|
|
AmbLightColourRed=0, AmbLightColourGreen=0,
|
|
AmbLightColourBlue=0, AmbLightIntensity=0,
|
|
AmbLightUpdate=0, AmbLightMode=0, AmbLightDuration=0,
|
|
AmbLightLIDFrom=0, AmbLightLIDTo=0))
|
|
data[2] = 0xFF # corrupt one byte
|
|
fio.send_raw(fio.frame_id("ALM_Req_A"), bytes(data))
|
|
```
|
|
|
|
---
|
|
|
|
## 7. Writing a new test
|
|
|
|
### 7.1 Starting point
|
|
|
|
A heavily-annotated, copyable template lives at
|
|
[`tests/hardware/_test_case_template.py`](../tests/hardware/_test_case_template.py).
|
|
The leading underscore stops pytest from collecting it, so the example
|
|
bodies don't run on the bench.
|
|
|
|
Copy it to a new file named `test_<feature>.py` under `tests/hardware/`
|
|
and edit. The template includes:
|
|
|
|
- The standard imports for `frame_io` and `alm_helpers`
|
|
- The three module-level fixtures (`fio`, `alm`, `_reset_to_off`) with
|
|
inline explanations of fixture scope, `autouse`, and `yield`
|
|
- Three skeleton bodies (one per common shape — see §7.3)
|
|
- An appendix listing the most-reached-for patterns
|
|
|
|
### 7.2 The four-phase test pattern
|
|
|
|
Every hardware test that mutates ECU state beyond just the LED should
|
|
follow a **SETUP / PROCEDURE / ASSERT / TEARDOWN** structure with a
|
|
`try`/`finally` so the teardown runs even when an assertion fails.
|
|
|
|
```python
|
|
def test_xyz(fio, alm, rp):
|
|
"""..."""
|
|
# ── SETUP ──────────────────────────────────────
|
|
# Bring the ECU to the exact state THIS test needs, beyond what the
|
|
# autouse reset already gave us. Anything you change here MUST be
|
|
# undone in TEARDOWN below.
|
|
fio.send("ConfigFrame", ConfigFrame_EnableCompensation=0, ...)
|
|
time.sleep(0.2)
|
|
|
|
try:
|
|
# ── PROCEDURE ──────────────────────────────
|
|
# The actions whose effects you are validating.
|
|
fio.send("ALM_Req_A", ...)
|
|
reached, _, history = alm.wait_for_state(LED_STATE_ON, timeout=1.0)
|
|
|
|
# ── ASSERT ─────────────────────────────────
|
|
# Bus-observable expectations. Use `rp("key", value)` to attach
|
|
# diagnostics to the report, then assert.
|
|
rp("led_state_history", history)
|
|
assert reached, history
|
|
alm.assert_pwm_wo_comp_matches_rgb(rp, r, g, b)
|
|
|
|
finally:
|
|
# ── TEARDOWN ───────────────────────────────
|
|
# Always runs. Restores anything SETUP perturbed.
|
|
fio.send("ConfigFrame", ConfigFrame_EnableCompensation=1, ...)
|
|
time.sleep(0.2)
|
|
```
|
|
|
|
### Why this gives you test independence
|
|
|
|
Pytest runs tests in a deterministic order (the order they appear in the
|
|
file). Without strict teardown, a failure midway through one test can
|
|
leave the ECU in a non-default state that breaks every subsequent test
|
|
— turning a single bug into a cascade. The four-phase pattern prevents
|
|
that with two layers:
|
|
|
|
| Layer | What it covers | Where it lives |
|
|
|---|---|---|
|
|
| Common baseline | LED → OFF | autouse `_reset_to_off` fixture |
|
|
| Per-test specifics | ConfigFrame, schedules, mode flags, anything else | the test's own `try`/`finally` |
|
|
|
|
The autouse fixture handles the universal baseline so individual tests
|
|
don't have to think about it; the per-test `try`/`finally` handles
|
|
whatever that specific test mutated.
|
|
|
|
### When you can skip the four phases
|
|
|
|
If your test only sends a frame and observes the LED state (i.e. the
|
|
*only* mutable state involved is something the autouse reset already
|
|
restores), the explicit SETUP/TEARDOWN sections are dead weight — just
|
|
write the procedure straight through. Flavor A in the template
|
|
illustrates this minimal shape.
|
|
|
|
### 7.3 Three flavors in the template
|
|
|
|
| Flavor | When to use it |
|
|
|---|---|
|
|
| **A — minimal** | Test only drives the LED and asserts on PWM/state. The autouse reset is enough. |
|
|
| **B — with isolation** | Test changes any persistent ECU state (ConfigFrame, schedules, NAD, …). Use the `try`/`finally` pattern. |
|
|
| **C — single-signal probe** | "Ask the ECU one thing and check the answer." Uses `fio.read_signal(...)`, no state mutation. |
|
|
|
|
Pick the closest one, delete the others, rename the function and fill
|
|
in the docstring.
|
|
|
|
### 7.4 Tests that drive the PSU and observe the LIN bus
|
|
|
|
For *combined* PSU + LIN scenarios (overvoltage / undervoltage
|
|
tolerance, brown-out behaviour, supply transients) there is a
|
|
dedicated template at
|
|
[`tests/hardware/_test_case_template_psu_lin.py`](../tests/hardware/_test_case_template_psu_lin.py).
|
|
It adds a `psu` fixture (cross-platform port resolution + safe-off
|
|
on close), an autouse `_park_at_nominal` fixture, a
|
|
`wait_for_voltage_status` polling helper, and three flavors:
|
|
|
|
| Flavor | Demonstrates |
|
|
|---|---|
|
|
| A — overvoltage | Drive PSU above the OV threshold, expect `ALMVoltageStatus = 0x02`, restore. |
|
|
| B — undervoltage | Symmetric for UV (`0x01`). |
|
|
| C — sweep | Parametrized walk over `(V, expected_status)` tuples. |
|
|
|
|
For the *settling time* characterization that feeds these tests'
|
|
detect timeouts, see `tests/hardware/test_psu_voltage_settling.py`
|
|
(opt-in via `pytest -m psu_settling`).
|
|
|
|
See [`docs/14_power_supply.md` §6](14_power_supply.md#6-run-the-hardware-test) and [§5 (session-managed power)](14_power_supply.md#5-session-managed-power-the-bench-powers-the-ecu-through-the-psu)
|
|
for the full reference and the constants to tune for your firmware.
|
|
|
|
---
|
|
|
|
## 8. Related docs
|
|
|
|
- [`04_lin_interface_call_flow.md`](04_lin_interface_call_flow.md) — what
|
|
`LinInterface.send`/`receive` does under the hood for each adapter.
|
|
- [`16_mum_internals.md`](16_mum_internals.md) — MUM-specific behaviour
|
|
the helpers rely on (master-driven receive, frame-length map, …).
|
|
- [`17_ldf_parser.md`](17_ldf_parser.md) — how the LDF is loaded and how
|
|
`pack` / `unpack` are implemented.
|
|
- [`13_unit_testing_guide.md`](13_unit_testing_guide.md) — unit-test
|
|
conventions, markers, coverage.
|
|
- [`15_report_properties_cheatsheet.md`](15_report_properties_cheatsheet.md)
|
|
— the standard `rp("key", value)` keys these helpers emit.
|