ecu-tests/docs/05_architecture_overview.md

# Architecture Overview

This document provides a high-level view of the framework’s components and how they interact, plus a Mermaid diagram for quick orientation.

> For the **dynamic wiring** — how a test actually reaches a live
> `LinInterface` at session start, the fixture topology, and the playbook
> for adding a new framework component — see
> [`24_test_wiring.md`](24_test_wiring.md).

## Components

### Framework core (`ecu_framework/`)
- Config Loader — `ecu_framework/config/loader.py` (YAML → dataclasses; re-exported via `ecu_framework.config`)
- LIN Abstraction — `ecu_framework/lin/base.py` (`LinInterface`, `LinFrame`)
- Mock LIN Adapter — `ecu_framework/lin/mock.py`
- MUM LIN Adapter — `ecu_framework/lin/mum.py` (Melexis Universal Master via `pylin` + `pymumclient`)
- BabyLIN Adapter — `ecu_framework/lin/babylin.py` (SDK wrapper → BabyLIN_library.py; **DEPRECATED**, kept for legacy rigs only)
- LDF Database — `ecu_framework/lin/ldf.py` (`LdfDatabase`/`Frame` over `ldfparser`; per-frame `pack`/`unpack`). **Runtime, dynamic.** Loaded fresh each session from whatever LDF the config points at. See [LDF Database vs Generated LIN API](#ldf-database-vs-generated-lin-api-two-layers-one-purpose) below for why this is paired with the generated layer.
- Flasher — `ecu_framework/flashing/hex_flasher.py`
- Power Supply (PSU) control — `ecu_framework/power/owon_psu.py` (serial SCPI + cross-platform port resolver)
- PSU quick demo script — `vendor/Owon/owon_psu_quick_demo.py`

### Hardware test layer (`tests/hardware/`)
- Project-wide fixtures — `tests/conftest.py` (config, lin, ldf, flash_ecu, rp)
- Hardware-suite fixtures — `tests/hardware/conftest.py` (session-scoped, autouse PSU; the bench is powered up once at session start and stays on for every test in the suite)
- MUM-suite fixtures — `tests/hardware/mum/conftest.py` (session-scoped `fio`, `nad`, `alm`; autouse `_require_mum` gate and `_reset_to_off` per-test reset). Tests outside `tests/hardware/mum/` cannot see these — that's how PSU-only and BabyLIN-only tests are kept from accidentally requesting MUM fixtures.
- Generic LDF I/O — `tests/hardware/frame_io.py` (`FrameIO` — send/receive/pack/unpack for any LDF frame plus raw-bus escape hatches). Stringly-typed at this layer (`fio.send("ALM_Req_A", …)`); tests use this **only** for cases the AlmTester facade doesn't model (schema introspection, raw-frame escape hatches, MUM-only `send_raw`).
- ALM domain helpers — `tests/hardware/alm_helpers.py` (`AlmTester` + hand-maintained `IntEnum` classes). The **single contributor-facing API** for ALM tests: per-signal readers (`read_led_state`, `read_voltage_status`, …), per-action senders (`send_color`, `send_config`, …), wait helpers, and cross-frame patterns (`assert_pwm_matches_rgb`). See [`19_frame_io_and_alm_helpers.md`](19_frame_io_and_alm_helpers.md).
- Retired layer (kept under [`deprecated/`](../deprecated/)) — `deprecated/gen_lin_api.py` + `deprecated/_generated/lin_api.py`. Was an LDF→Python generator that emitted typed frame/encoding classes; replaced by the hand-maintained surface in `alm_helpers.py`. See [`22_generated_lin_api.md`](22_generated_lin_api.md) for the historical design.
- PSU settle helpers — `tests/hardware/psu_helpers.py` (`wait_until_settled`, `apply_voltage_and_settle` — measured-rail-then-validation pattern shared by all voltage-changing tests)
- RGB→PWM calculator — `vendor/rgb_to_pwm.py` (consumed by `AlmTester.assert_pwm_*`)
- Test templates (not collected) — `tests/hardware/_test_case_template.py`, `tests/hardware/_test_case_template_psu_lin.py`

### Tests, reporting, artifacts
- Tests (pytest) — modules under `tests/{,unit,plugin,hardware}/`
- Reporting Plugin — `conftest_plugin.py` (docstring → report metadata)
- Reports — `reports/report.html`, `reports/junit.xml`, `reports/summary.md`, `reports/requirements_coverage.json`

## Mermaid architecture diagram

```mermaid
flowchart TB
  subgraph Tests_and_Pytest [Tests & Pytest]
    T[tests/* (test bodies)]
    CF[tests/conftest.py<br/>config, lin, ldf, flash_ecu, rp]
    HCF[tests/hardware/conftest.py<br/>SESSION psu &#40;autouse&#41;]
    MCF[tests/hardware/mum/conftest.py<br/>fio, alm, nad, _require_mum &#40;autouse&#41;,<br/>_reset_to_off &#40;autouse&#41;]
    PL[conftest_plugin.py]
  end

  subgraph Hardware_Helpers [Hardware-test helpers]
    ALM[tests/hardware/alm_helpers.py<br/>AlmTester + typed IntEnums<br/>&#40;contributor-facing API&#41;]
    FIO[tests/hardware/frame_io.py<br/>FrameIO &#40;low-level, rarely used by tests&#41;]
    RGB[vendor/rgb_to_pwm.py]
    TPL[tests/hardware/_test_case_template*.py<br/>not collected]
  end

  subgraph Retired [Retired &#40;deprecated/&#41;]
    GEN[deprecated/_generated/lin_api.py]
    GENSCRIPT[deprecated/gen_lin_api.py]
  end

  subgraph Framework
    CFG[ecu_framework/config/loader.py]
    BASE[ecu_framework/lin/base.py]
    MOCK[ecu_framework/lin/mock.py]
    MUM[ecu_framework/lin/mum.py]
    BABY[ecu_framework/lin/babylin.py<br/>DEPRECATED]
    LDF[ecu_framework/lin/ldf.py]
    FLASH[ecu_framework/flashing/hex_flasher.py]
    POWER[ecu_framework/power/owon_psu.py<br/>SerialParams, OwonPSU,<br/>resolve_port]
  end

  subgraph Artifacts
    REP[reports/report.html<br/>reports/junit.xml<br/>reports/summary.md]
    YAML[config/*.yaml<br/>test_config.yaml<br/>mum.example.yaml<br/>babylin.example.yaml — deprecated]
    PSU_YAML[config/owon_psu.yaml<br/>OWON_PSU_CONFIG]
    MELEXIS[Melexis pylin + pymumclient<br/>MUM @ 192.168.7.2]
    SDK[vendor/BabyLIN_library.py<br/>platform libs<br/>DEPRECATED]
    OWON[vendor/Owon/owon_psu_quick_demo.py]
    LDFFILE[vendor/*.ldf]
    LDFLIB[ldfparser PyPI]
  end

  T --> CF
  T --> HCF
  T --> MCF
  MCF --> FIO
  MCF --> ALM
  CF --> CFG
  CF --> BASE
  CF --> MOCK
  CF --> MUM
  CF --> BABY
  CF --> FLASH
  HCF --> POWER
  T --> ALM
  T -.rare, low-level.-> FIO
  ALM --> FIO
  GENSCRIPT -.was: read LDF.-> LDFFILE
  GENSCRIPT -.was: emit source.-> GEN
  ALM --> RGB
  TPL -.copy & edit.-> T

  PL --> REP

  CFG --> YAML
  CFG --> PSU_YAML
  MUM --> MELEXIS
  BABY --> SDK
  LDF --> LDFLIB
  LDF --> LDFFILE
  POWER --> PSU_YAML
  T --> OWON
  T --> REP
```

## Data and control flow summary

- Tests use fixtures to obtain config and a connected LIN adapter
- Config loader reads YAML (or env override), returns typed dataclasses
- LIN calls are routed through the interface abstraction to the selected adapter
- Hardware tests sit on top of three helpers: `FrameIO` (LDF-driven send /
  receive / pack / unpack for any frame, stringly-typed by frame name),
  the generated `lin_api.py` (typed `AlmReqA.send(fio, …)` wrappers plus
  `LedState`/`Mode`/`Update` enums, so signal/frame typos become import
  errors), and `AlmTester` (ALM_Node domain patterns built on `FrameIO`
  and the generated enums). All three are imported as siblings from
  `tests/hardware/` — see `docs/19_frame_io_and_alm_helpers.md` and
  `docs/22_generated_lin_api.md`
- The hardware-suite `tests/hardware/conftest.py` defines a **session-scoped,
  autouse** `psu` fixture: on benches where the Owon PSU powers the ECU,
  the supply is opened once at session start, parked at
  `config.power_supply.set_voltage` / `set_current`, and left enabled
  for every test. Voltage-tolerance tests perturb voltage and restore
  in `finally`; they never toggle output. See `docs/14_power_supply.md` §5.
- Flasher (optional) uses the same `LinInterface` to program the ECU
- Power supply control (optional) uses `ecu_framework/power/owon_psu.py`
  and reads `config.power_supply` (merged with `config/owon_psu.yaml`
  or `OWON_PSU_CONFIG` when present). The quick demo script under
  `vendor/Owon/` provides a quick manual flow
- Reporting plugin parses docstrings and enriches the HTML report

## LDF Database vs Generated LIN API: two layers, one purpose

> **Historical.** The generated LIN API is retired — see the banner in
> [`22_generated_lin_api.md`](22_generated_lin_api.md). The comparison
> below is kept for traceability: the *runtime* `LdfDatabase` path is
> still active (it's what `FrameIO` calls into); the *generated* path
> column describes a code path that now lives under
> [`deprecated/`](../deprecated/). Today, the analog of the right
> column is the hand-maintained `IntEnum` classes and method surface
> in `tests/hardware/alm_helpers.py`.

There are two pieces of code in this repo whose names both sound like
"the LDF module", and a recurring question is why both exist:

| Aspect | `ecu_framework/lin/ldf.py` (`LdfDatabase`/`Frame`) | `tests/hardware/_generated/lin_api.py` |
| --- | --- | --- |
| **What it is** | Runtime wrapper around `ldfparser` | Source file emitted by `scripts/gen_lin_api.py` |
| **When it runs** | Every test session — `parse_ldf(path)` is called inside the `ldf` fixture (`tests/conftest.py:92`) | Never runs as a parser; it *is* the parser's output, imported like any other module |
| **What it produces** | `Frame` objects whose `.pack(**kw)` / `.unpack(bytes)` route through `ldfparser`'s `encode_raw` / `decode_raw` | `class AlmReqA`, `class LedState(IntEnum)`, etc. — Python literals derived from one LDF |
| **Source of truth** | The LDF file on disk at startup | The LDF file at the time `gen_lin_api.py` was last run (SHA256 in the file header) |
| **Typing model** | Stringly-typed (`db.frame("ALM_Req_A").pack(AmbLight…=…)`) | Statically typed (`AlmReqA.send(fio, AmbLight…=…)`) |
| **Failure mode for a missing/renamed frame** | `KeyError: 'Frame X not found'` at test time | `ImportError: cannot import name 'X'` at collection time, surfaced in CI |
| **Failure mode for an LDF rev** | None — it parses whatever is on disk | The in-sync unit test fails when the LDF SHA256 in the header drifts |
| **Layer in the dependency tree** | Framework core (`ecu_framework/`) — knows nothing about specific frame names | Test code (`tests/hardware/`) — bakes specific frame and signal names in |
| **Lifecycle** | Re-parsed each pytest session | Regenerated only on LDF change, then committed |
| **Coupling to `ldfparser`** | Direct (`from ldfparser import parse_ldf`) | None at runtime; the generator imports it, the generated file does not |

The two answer **orthogonal** questions:

- `ecu_framework/lin/ldf.py` answers *"what bytes go on the wire for this
  frame right now?"* — it has to be dynamic because bit offsets, widths,
  and init values are properties of whichever LDF the bench loaded, and
  must be re-validated against that LDF at startup.
- `tests/hardware/_generated/lin_api.py` answers *"what frame and signal
  names are valid for me to type in test code?"* — it has to be static
  because that question is asked by the IDE, mypy, and pytest's
  collection step, all of which run before any LDF has been parsed.

If only `ecu_framework/lin/ldf.py` existed, every test would keep its
stringly-typed `fio.send("ALM_Req_A", …)` calls and its hand-copied
`LED_STATE_OFF = 0` constants — both of which silently drift when the LDF
changes. If only the generated `lin_api.py` existed, the runtime would
have no path from a frame name to the actual byte layout for the currently
loaded LDF — and worse, the test bench would happily ship bytes encoded
against a *stale* LDF baked into the generator's last run.

### Test-side entry points

> **Updated.** This section originally described three parallel paths
> including the generated typed-wrapper layer. With the generator
> retired, the active picture is simpler: tests reach for `AlmTester`
> by default, and drop down to `FrameIO` only for schema introspection
> or other low-level needs. The diagram below is preserved for
> reference but the `gen_lin_api` node represents the now-retired
> path — see [`deprecated/`](../deprecated/).

`FrameIO` deliberately has no static dependency on
`ecu_framework/lin/ldf.py` (its only `ecu_framework` import is
`LinInterface` + `LinFrame` from `lin/base.py`), so the `ldf` it
receives can be any object with a `.frame(name)` method.

```mermaid
flowchart TB
  T[test code]

  subgraph Paths[three independent ways to address a frame]
    GEN["gen_lin_api typed wrapper<br/>AlmReqA.send&#40;fio, ...&#41;<br/>compile-time name check"]
    FIO["FrameIO stringly-typed<br/>fio.send&#40;'ALM_Req_A', ...&#41;<br/>per-instance frame cache"]
    LDFDIRECT["LdfDatabase directly<br/>ldf.frame&#40;'ALM_Req_A'&#41;.pack&#40;...&#41;<br/>returns bytes, no I/O"]
  end

  T --> GEN
  T --> FIO
  T --> LDFDIRECT

  GEN -.delegates.-> FIO
  FIO -.duck-typed lookup.-> LDFOBJ[ldf-like object<br/>currently LdfDatabase]
  LDFDIRECT --> LDFOBJ
  LDFOBJ --> LDFPARSER[ldfparser - bit layout]

  FIO --> LIN[LinInterface.send / receive]
  LDFDIRECT -->|caller invokes lin.send<br/>with the packed bytes| LIN
  LIN --> WIRE[wire]
```

What each path buys you:

- **`gen_lin_api`** — compile-time name validation. Typo a frame or signal
  name and the IDE / mypy / pytest collection rejects it before any LDF
  is read. Delegates the actual packing to `fio.send`.
- **`FrameIO`** — stringly-typed I/O over the wire. Caches frame
  lookups, supports raw escape hatches (`send_raw` / `receive_raw`) that
  bypass the LDF object entirely.
- **`LdfDatabase` directly** — schema-only access. Useful when a test
  wants to inspect frame layout, pack a buffer without sending, or hand
  the bytes to a non-FrameIO transport.

The LDF object (currently `LdfDatabase`) is consumed by both `FrameIO`
and any direct-use code path. `FrameIO`'s use is via injection — it
never imports `LdfDatabase` and can be tested against a stub. The next
section explains what "duck-typed" means in this codebase and why it
matters architecturally.

Removing any of the three entry points collapses a distinct affordance:

- Drop `gen_lin_api` → tests keep stringly-typed `fio.send("ALM_Req_A", …)`
  and hand-copied state constants, both of which silently drift when the
  LDF changes.
- Drop `FrameIO` → every test that wants high-level I/O has to wire
  `LinInterface` + LDF lookup + pack/unpack itself.
- Drop direct `LdfDatabase` usage → tests can no longer pack a frame
  without sending it, or inspect frame metadata without an I/O attempt.

## Duck typing: how the polymorphism actually works

Both architectural seams above (`FrameIO`'s `ldf` injection, the `lin`
fixture's adapter selection) rely on **duck typing** rather than static
type hierarchies. The Python idiom is:

> If it walks like a duck and quacks like a duck, it's a duck.

Translation: Python doesn't check *what type* of object you pass — it
just calls the methods you call and trusts they work. If they do, the
object is "duck enough." The contract is the **shape of the methods
used**, not the class.

### Example 1: `FrameIO` and the `ldf` parameter

Look at `tests/hardware/frame_io.py` line 44:

```python
class FrameIO:
    def __init__(self, lin: LinInterface, ldf) -> None:
        self._lin = lin
        self._ldf = ldf
```

Two parameters, two very different contracts:

- `lin` carries an annotation (`LinInterface`). That's a **nominal** contract:
  a type checker expects an instance of that class (or a subclass).
- `ldf` has **no annotation** at all. Anything is accepted at the call site.

Then on line 65 `FrameIO` uses `ldf` exactly once, this way:

```python
f = self._ldf.frame(name)
```

That single method call — `.frame(name)` returning something with `.id`,
`.pack(**signals)`, `.unpack(bytes)`, and `.length` — **is** the contract.
Anything with that surface works:

- The real `LdfDatabase` (production)
- A unit-test stub (`class _StubLdf: def frame(self, n): return _StubFrame(n)`)
- A future schema source (cached JSON, in-memory dict, etc.)

`grep` will confirm: `frame_io.py` never writes `from ecu_framework.lin.ldf import LdfDatabase`,
never writes `isinstance(ldf, LdfDatabase)`. The module is structurally
unaware of `LdfDatabase`. That's what "no static dependency" meant in
the previous section's diagram label `duck-typed lookup`.

### Counter-example: what static typing would look like

If `FrameIO` had been written nominally, it would be:

```python
from ecu_framework.lin.ldf import LdfDatabase

class FrameIO:
    def __init__(self, lin: LinInterface, ldf: LdfDatabase) -> None:
        ...
```

The consequences:

- `frame_io.py` would carry a hard module-level dependency on
  `ecu_framework/lin/ldf.py`.
- A unit test could no longer pass a stub without subclassing
  `LdfDatabase` or monkey-patching.
- The `frame_io → ecu_framework/lin/ldf.py` edge in the architecture
  diagram would represent a real coupling.

The codebase deliberately avoided that — the `ldf` parameter being
untyped is intentional, not an oversight.

### Example 2: the `lin` fixture and adapter polymorphism

The same idiom drives the LIN adapter swap. `tests/conftest.py:34` returns
something annotated as `LinInterface`:

```python
@pytest.fixture(scope="session")
def lin(config: EcuTestConfig) -> Iterator[LinInterface]:
    if   config.interface.type == "mock":    lin = MockBabyLinInterface(...)
    elif config.interface.type == "mum":     lin = MumLinInterface(...)
    elif config.interface.type == "babylin": lin = BabyLinInterface(...)
    ...
```

This case has a nominal anchor (`LinInterface` is an `abc.ABC` declaring
the required methods), but the day-to-day swap is duck-typed in spirit:
tests call `lin.send(frame)` / `lin.receive(...)` without caring which
concrete adapter is underneath. All three quack `.send()` / `.receive()`
identically, so one YAML config switch reroutes every test in the suite
without touching a single test body.

### Why this matters

Two practical wins, both load-bearing in this codebase:

1. **Swappability.** A new adapter (CAN, FlexRay, a different LIN master)
   only needs to expose the same method surface. No edits to FrameIO,
   no edits to tests.
2. **Testability.** Unit tests pass minimal stubs — `tests/unit/test_mum_adapter_mocked.py`
   builds fake `pylin` / `pymumclient` objects with just enough method
   surface to exercise the adapter, never importing the real Melexis stack.

### The Python idiom in play: EAFP

The supporting philosophy has a name: **EAFP**, "Easier to Ask Forgiveness
than Permission." Instead of:

```python
if isinstance(ldf, LdfDatabase) and hasattr(ldf, "frame"):
    f = ldf.frame(name)
else:
    raise TypeError(...)
```

…you just write:

```python
f = ldf.frame(name)
```

…and let Python raise `AttributeError` at the point of misuse. The other
half of the idiom is **LBYL**, "Look Before You Leap" — the explicit-checks
style. Python idiomatically prefers EAFP because it composes better with
duck typing: you don't need to enumerate every valid type, only the
behaviours.

### The trade-off

Duck typing is not free. Two costs to be aware of:

- **Implicit contracts.** The type signature `ldf` tells you nothing.
  A reader has to scan the method body to learn that `.frame(name)`,
  `.id`, `.pack()`, `.length` are required. Mitigated here by the
  injection happening in one place (the `fio` fixture) so the duck
  shape is easy to track.
- **Runtime, not compile-time, errors.** A misshaped duck blows up at
  the call site, not at construction. Type checkers can't catch it.
  Mitigated here by the limited number of concrete duck-shapes in the
  codebase — there's really only `LdfDatabase`, and the fixture wires
  it in centrally.

The codebase accepts those costs in exchange for the swappability and
testability wins above. The `LinInterface` abstract base class is the
formal seam where the team chose to spend annotation effort; the `ldf`
slot is where the team chose to keep things light.

## Extending the architecture

- Add new bus adapters by implementing `LinInterface`
- Add new ECU-domain helpers next to `AlmTester` (e.g. `BcmTester`)
  on top of `FrameIO`; share fixtures via
  `tests/hardware/conftest.py` (or a per-adapter conftest like
  `tests/hardware/mum/conftest.py`)
- When the LDF changes (new frame, renamed signal, new encoding-type row):
  add or update the corresponding `read_*` / `send_*` method (and, if
  needed, a new `IntEnum`) in `tests/hardware/alm_helpers.py`. This is
  the maintenance pact that replaced the retired generator
- Add new bench instrument controllers next to `OwonPSU` under
  `ecu_framework/power/` or a new `ecu_framework/instruments/` package,
  expose them as session-scoped fixtures
- Add new report sinks (e.g., JSON or a DB) by extending the plugin