Hosam-Eldin Mostafa
ec218bd5fe
The previous ASCII pipeline implied a single linear stack from gen_lin_api
down through FrameIO down through ecu_framework/lin/ldf.py — and showed
a static dependency from FrameIO to that module. Both are wrong.
What the code actually says (tests/hardware/frame_io.py:34):
from ecu_framework.lin.base import LinFrame, LinInterface
That's the only ecu_framework import in FrameIO. The `ldf` constructor
parameter is duck-typed — FrameIO never imports LdfDatabase and would
work against any object exposing `.frame(name)`. So `frame_io → lin/ldf`
is an injected runtime call, not a module dependency.
Replace the linear ASCII diagram with a Mermaid parallel-paths diagram
that surfaces the three independent ways a tester can address a frame:
- gen_lin_api typed wrapper (compile-time name check)
- FrameIO stringly-typed I/O (with raw send_raw/receive_raw escape
hatches that don't touch the ldf object at all)
- LdfDatabase used directly (schema-only — pack to bytes, no I/O)
…all converging at LinInterface. The prose around the diagram is
rewritten to match: each path's affordance, and what concrete capability
is lost by removing any of the three.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
258 lines
14 KiB
Markdown
258 lines
14 KiB
Markdown
# Architecture Overview
|
||
|
||
This document provides a high-level view of the framework’s components and how they interact, plus a Mermaid diagram for quick orientation.
|
||
|
||
> For the **dynamic wiring** — how a test actually reaches a live
|
||
> `LinInterface` at session start, the fixture topology, and the playbook
|
||
> for adding a new framework component — see
|
||
> [`24_test_wiring.md`](24_test_wiring.md).
|
||
|
||
## Components
|
||
|
||
### Framework core (`ecu_framework/`)
|
||
- Config Loader — `ecu_framework/config/loader.py` (YAML → dataclasses; re-exported via `ecu_framework.config`)
|
||
- LIN Abstraction — `ecu_framework/lin/base.py` (`LinInterface`, `LinFrame`)
|
||
- Mock LIN Adapter — `ecu_framework/lin/mock.py`
|
||
- MUM LIN Adapter — `ecu_framework/lin/mum.py` (Melexis Universal Master via `pylin` + `pymumclient`)
|
||
- BabyLIN Adapter — `ecu_framework/lin/babylin.py` (SDK wrapper → BabyLIN_library.py; **DEPRECATED**, kept for legacy rigs only)
|
||
- LDF Database — `ecu_framework/lin/ldf.py` (`LdfDatabase`/`Frame` over `ldfparser`; per-frame `pack`/`unpack`). **Runtime, dynamic.** Loaded fresh each session from whatever LDF the config points at. See [LDF Database vs Generated LIN API](#ldf-database-vs-generated-lin-api-two-layers-one-purpose) below for why this is paired with the generated layer.
|
||
- Flasher — `ecu_framework/flashing/hex_flasher.py`
|
||
- Power Supply (PSU) control — `ecu_framework/power/owon_psu.py` (serial SCPI + cross-platform port resolver)
|
||
- PSU quick demo script — `vendor/Owon/owon_psu_quick_demo.py`
|
||
|
||
### Hardware test layer (`tests/hardware/`)
|
||
- Project-wide fixtures — `tests/conftest.py` (config, lin, ldf, flash_ecu, rp)
|
||
- Hardware-suite fixtures — `tests/hardware/conftest.py` (session-scoped, autouse PSU; the bench is powered up once at session start and stays on for every test in the suite)
|
||
- MUM-suite fixtures — `tests/hardware/mum/conftest.py` (session-scoped `fio`, `nad`, `alm`; autouse `_require_mum` gate and `_reset_to_off` per-test reset). Tests outside `tests/hardware/mum/` cannot see these — that's how PSU-only and BabyLIN-only tests are kept from accidentally requesting MUM fixtures.
|
||
- Generic LDF I/O — `tests/hardware/frame_io.py` (`FrameIO` — send/receive/pack/unpack for any LDF frame plus raw-bus escape hatches). Stringly-typed at this layer (`fio.send("ALM_Req_A", …)`); typed wrappers live one level up.
|
||
- Generated LIN API — `tests/hardware/_generated/lin_api.py` (auto-emitted from an LDF by `scripts/gen_lin_api.py`; one class per frame, one `IntEnum` per encoding type with logical values). **Build-time, static.** Provides typed names so frame/signal typos become import errors. Design + generation rules in `docs/22_generated_lin_api.md`; relationship to `ecu_framework/lin/ldf.py` covered in [LDF Database vs Generated LIN API](#ldf-database-vs-generated-lin-api-two-layers-one-purpose).
|
||
- ALM domain helpers — `tests/hardware/alm_helpers.py` (`AlmTester` — force_off / wait_for_state / measure_animating_window / assert_pwm_*). Imports typed frames + enums from the generated layer; keeps the non-generatable semantics (polling cadences, PWM tolerances, cross-frame test patterns).
|
||
- PSU settle helpers — `tests/hardware/psu_helpers.py` (`wait_until_settled`, `apply_voltage_and_settle` — measured-rail-then-validation pattern shared by all voltage-changing tests)
|
||
- RGB→PWM calculator — `vendor/rgb_to_pwm.py` (consumed by `AlmTester.assert_pwm_*`)
|
||
- Test templates (not collected) — `tests/hardware/_test_case_template.py`, `tests/hardware/_test_case_template_psu_lin.py`
|
||
|
||
### Tests, reporting, artifacts
|
||
- Tests (pytest) — modules under `tests/{,unit,plugin,hardware}/`
|
||
- Reporting Plugin — `conftest_plugin.py` (docstring → report metadata)
|
||
- Reports — `reports/report.html`, `reports/junit.xml`, `reports/summary.md`, `reports/requirements_coverage.json`
|
||
|
||
## Mermaid architecture diagram
|
||
|
||
```mermaid
|
||
flowchart TB
|
||
subgraph Tests_and_Pytest [Tests & Pytest]
|
||
T[tests/* (test bodies)]
|
||
CF[tests/conftest.py<br/>config, lin, ldf, flash_ecu, rp]
|
||
HCF[tests/hardware/conftest.py<br/>SESSION psu (autouse)]
|
||
MCF[tests/hardware/mum/conftest.py<br/>fio, alm, nad, _require_mum (autouse),<br/>_reset_to_off (autouse)]
|
||
PL[conftest_plugin.py]
|
||
end
|
||
|
||
subgraph Hardware_Helpers [Hardware-test helpers]
|
||
FIO[tests/hardware/frame_io.py<br/>FrameIO (stringly-typed)]
|
||
GEN[tests/hardware/_generated/lin_api.py<br/>AlmReqA, AlmStatus, ... (typed)<br/>LedState, Mode, Update IntEnums]
|
||
ALM[tests/hardware/alm_helpers.py<br/>AlmTester]
|
||
RGB[vendor/rgb_to_pwm.py]
|
||
TPL[tests/hardware/_test_case_template*.py<br/>not collected]
|
||
end
|
||
|
||
subgraph Build_Time [Build-time tooling (not run during tests)]
|
||
GENSCRIPT[scripts/gen_lin_api.py]
|
||
end
|
||
|
||
subgraph Framework
|
||
CFG[ecu_framework/config/loader.py]
|
||
BASE[ecu_framework/lin/base.py]
|
||
MOCK[ecu_framework/lin/mock.py]
|
||
MUM[ecu_framework/lin/mum.py]
|
||
BABY[ecu_framework/lin/babylin.py<br/>DEPRECATED]
|
||
LDF[ecu_framework/lin/ldf.py]
|
||
FLASH[ecu_framework/flashing/hex_flasher.py]
|
||
POWER[ecu_framework/power/owon_psu.py<br/>SerialParams, OwonPSU,<br/>resolve_port]
|
||
end
|
||
|
||
subgraph Artifacts
|
||
REP[reports/report.html<br/>reports/junit.xml<br/>reports/summary.md]
|
||
YAML[config/*.yaml<br/>test_config.yaml<br/>mum.example.yaml<br/>babylin.example.yaml — deprecated]
|
||
PSU_YAML[config/owon_psu.yaml<br/>OWON_PSU_CONFIG]
|
||
MELEXIS[Melexis pylin + pymumclient<br/>MUM @ 192.168.7.2]
|
||
SDK[vendor/BabyLIN_library.py<br/>platform libs<br/>DEPRECATED]
|
||
OWON[vendor/Owon/owon_psu_quick_demo.py]
|
||
LDFFILE[vendor/*.ldf]
|
||
LDFLIB[ldfparser PyPI]
|
||
end
|
||
|
||
T --> CF
|
||
T --> HCF
|
||
T --> MCF
|
||
MCF --> FIO
|
||
MCF --> ALM
|
||
CF --> CFG
|
||
CF --> BASE
|
||
CF --> MOCK
|
||
CF --> MUM
|
||
CF --> BABY
|
||
CF --> FLASH
|
||
HCF --> POWER
|
||
T --> FIO
|
||
T --> GEN
|
||
T --> ALM
|
||
ALM --> FIO
|
||
ALM --> GEN
|
||
GEN -.calls at runtime.-> FIO
|
||
GENSCRIPT -.reads LDF once.-> LDFFILE
|
||
GENSCRIPT -.emits source.-> GEN
|
||
ALM --> RGB
|
||
TPL -.copy & edit.-> T
|
||
|
||
PL --> REP
|
||
|
||
CFG --> YAML
|
||
CFG --> PSU_YAML
|
||
MUM --> MELEXIS
|
||
BABY --> SDK
|
||
LDF --> LDFLIB
|
||
LDF --> LDFFILE
|
||
POWER --> PSU_YAML
|
||
T --> OWON
|
||
T --> REP
|
||
```
|
||
|
||
## Data and control flow summary
|
||
|
||
- Tests use fixtures to obtain config and a connected LIN adapter
|
||
- Config loader reads YAML (or env override), returns typed dataclasses
|
||
- LIN calls are routed through the interface abstraction to the selected adapter
|
||
- Hardware tests sit on top of three helpers: `FrameIO` (LDF-driven send /
|
||
receive / pack / unpack for any frame, stringly-typed by frame name),
|
||
the generated `lin_api.py` (typed `AlmReqA.send(fio, …)` wrappers plus
|
||
`LedState`/`Mode`/`Update` enums, so signal/frame typos become import
|
||
errors), and `AlmTester` (ALM_Node domain patterns built on `FrameIO`
|
||
and the generated enums). All three are imported as siblings from
|
||
`tests/hardware/` — see `docs/19_frame_io_and_alm_helpers.md` and
|
||
`docs/22_generated_lin_api.md`
|
||
- The hardware-suite `tests/hardware/conftest.py` defines a **session-scoped,
|
||
autouse** `psu` fixture: on benches where the Owon PSU powers the ECU,
|
||
the supply is opened once at session start, parked at
|
||
`config.power_supply.set_voltage` / `set_current`, and left enabled
|
||
for every test. Voltage-tolerance tests perturb voltage and restore
|
||
in `finally`; they never toggle output. See `docs/14_power_supply.md` §5.
|
||
- Flasher (optional) uses the same `LinInterface` to program the ECU
|
||
- Power supply control (optional) uses `ecu_framework/power/owon_psu.py`
|
||
and reads `config.power_supply` (merged with `config/owon_psu.yaml`
|
||
or `OWON_PSU_CONFIG` when present). The quick demo script under
|
||
`vendor/Owon/` provides a quick manual flow
|
||
- Reporting plugin parses docstrings and enriches the HTML report
|
||
|
||
## LDF Database vs Generated LIN API: two layers, one purpose
|
||
|
||
There are two pieces of code in this repo whose names both sound like
|
||
"the LDF module", and a recurring question is why both exist:
|
||
|
||
| Aspect | `ecu_framework/lin/ldf.py` (`LdfDatabase`/`Frame`) | `tests/hardware/_generated/lin_api.py` |
|
||
| --- | --- | --- |
|
||
| **What it is** | Runtime wrapper around `ldfparser` | Source file emitted by `scripts/gen_lin_api.py` |
|
||
| **When it runs** | Every test session — `parse_ldf(path)` is called inside the `ldf` fixture (`tests/conftest.py:92`) | Never runs as a parser; it *is* the parser's output, imported like any other module |
|
||
| **What it produces** | `Frame` objects whose `.pack(**kw)` / `.unpack(bytes)` route through `ldfparser`'s `encode_raw` / `decode_raw` | `class AlmReqA`, `class LedState(IntEnum)`, etc. — Python literals derived from one LDF |
|
||
| **Source of truth** | The LDF file on disk at startup | The LDF file at the time `gen_lin_api.py` was last run (SHA256 in the file header) |
|
||
| **Typing model** | Stringly-typed (`db.frame("ALM_Req_A").pack(AmbLight…=…)`) | Statically typed (`AlmReqA.send(fio, AmbLight…=…)`) |
|
||
| **Failure mode for a missing/renamed frame** | `KeyError: 'Frame X not found'` at test time | `ImportError: cannot import name 'X'` at collection time, surfaced in CI |
|
||
| **Failure mode for an LDF rev** | None — it parses whatever is on disk | The in-sync unit test fails when the LDF SHA256 in the header drifts |
|
||
| **Layer in the dependency tree** | Framework core (`ecu_framework/`) — knows nothing about specific frame names | Test code (`tests/hardware/`) — bakes specific frame and signal names in |
|
||
| **Lifecycle** | Re-parsed each pytest session | Regenerated only on LDF change, then committed |
|
||
| **Coupling to `ldfparser`** | Direct (`from ldfparser import parse_ldf`) | None at runtime; the generator imports it, the generated file does not |
|
||
|
||
The two answer **orthogonal** questions:
|
||
|
||
- `ecu_framework/lin/ldf.py` answers *"what bytes go on the wire for this
|
||
frame right now?"* — it has to be dynamic because bit offsets, widths,
|
||
and init values are properties of whichever LDF the bench loaded, and
|
||
must be re-validated against that LDF at startup.
|
||
- `tests/hardware/_generated/lin_api.py` answers *"what frame and signal
|
||
names are valid for me to type in test code?"* — it has to be static
|
||
because that question is asked by the IDE, mypy, and pytest's
|
||
collection step, all of which run before any LDF has been parsed.
|
||
|
||
If only `ecu_framework/lin/ldf.py` existed, every test would keep its
|
||
stringly-typed `fio.send("ALM_Req_A", …)` calls and its hand-copied
|
||
`LED_STATE_OFF = 0` constants — both of which silently drift when the LDF
|
||
changes. If only the generated `lin_api.py` existed, the runtime would
|
||
have no path from a frame name to the actual byte layout for the currently
|
||
loaded LDF — and worse, the test bench would happily ship bytes encoded
|
||
against a *stale* LDF baked into the generator's last run.
|
||
|
||
### Three independent entry points, one wire
|
||
|
||
A tester has three legitimate ways to drive the bus, all converging at
|
||
`LinInterface`. They are **parallel paths**, not a single nested stack —
|
||
`FrameIO` deliberately has no static dependency on `ecu_framework/lin/ldf.py`
|
||
(its only `ecu_framework` import is `LinInterface` + `LinFrame` from
|
||
`lin/base.py`), so the `ldf` it receives can be any object with a
|
||
`.frame(name)` method.
|
||
|
||
```mermaid
|
||
flowchart TB
|
||
T[test code]
|
||
|
||
subgraph Paths[three independent ways to address a frame]
|
||
GEN["gen_lin_api typed wrapper<br/>AlmReqA.send(fio, ...)<br/>compile-time name check"]
|
||
FIO["FrameIO stringly-typed<br/>fio.send('ALM_Req_A', ...)<br/>per-instance frame cache"]
|
||
LDFDIRECT["LdfDatabase directly<br/>ldf.frame('ALM_Req_A').pack(...)<br/>returns bytes, no I/O"]
|
||
end
|
||
|
||
T --> GEN
|
||
T --> FIO
|
||
T --> LDFDIRECT
|
||
|
||
GEN -.delegates.-> FIO
|
||
FIO -.duck-typed lookup.-> LDFOBJ[ldf-like object<br/>currently LdfDatabase]
|
||
LDFDIRECT --> LDFOBJ
|
||
LDFOBJ --> LDFPARSER[ldfparser - bit layout]
|
||
|
||
FIO --> LIN[LinInterface.send / receive]
|
||
LDFDIRECT -->|caller invokes lin.send<br/>with the packed bytes| LIN
|
||
LIN --> WIRE[wire]
|
||
```
|
||
|
||
What each path buys you:
|
||
|
||
- **`gen_lin_api`** — compile-time name validation. Typo a frame or signal
|
||
name and the IDE / mypy / pytest collection rejects it before any LDF
|
||
is read. Delegates the actual packing to `fio.send`.
|
||
- **`FrameIO`** — stringly-typed I/O over the wire. Caches frame
|
||
lookups, supports raw escape hatches (`send_raw` / `receive_raw`) that
|
||
bypass the LDF object entirely.
|
||
- **`LdfDatabase` directly** — schema-only access. Useful when a test
|
||
wants to inspect frame layout, pack a buffer without sending, or hand
|
||
the bytes to a non-FrameIO transport.
|
||
|
||
The LDF object (currently `LdfDatabase`) is consumed by both `FrameIO`
|
||
and any direct-use code path. `FrameIO`'s use is via injection — it
|
||
never imports `LdfDatabase` and can be tested against a stub.
|
||
|
||
Removing any of the three entry points collapses a distinct affordance:
|
||
|
||
- Drop `gen_lin_api` → tests keep stringly-typed `fio.send("ALM_Req_A", …)`
|
||
and hand-copied state constants, both of which silently drift when the
|
||
LDF changes.
|
||
- Drop `FrameIO` → every test that wants high-level I/O has to wire
|
||
`LinInterface` + LDF lookup + pack/unpack itself.
|
||
- Drop direct `LdfDatabase` usage → tests can no longer pack a frame
|
||
without sending it, or inspect frame metadata without an I/O attempt.
|
||
|
||
## Extending the architecture
|
||
|
||
- Add new bus adapters by implementing `LinInterface`
|
||
- Add new ECU-domain helpers next to `AlmTester` (e.g. `BcmTester`)
|
||
on top of `FrameIO` and the generated `lin_api.py`; share fixtures via
|
||
`tests/hardware/conftest.py`
|
||
- When the LDF changes (new frame, renamed signal, new encoding-type row):
|
||
re-run `python scripts/gen_lin_api.py <ldf-path>`, commit the updated
|
||
`tests/hardware/_generated/lin_api.py` alongside the LDF change. The
|
||
in-sync unit test in `tests/unit/test_generated_lin_api_in_sync.py`
|
||
fails CI if the two ever drift
|
||
- Add new bench instrument controllers next to `OwonPSU` under
|
||
`ecu_framework/power/` or a new `ecu_framework/instruments/` package,
|
||
expose them as session-scoped fixtures
|
||
- Add new report sinks (e.g., JSON or a DB) by extending the plugin
|