diff --git a/docs/01_run_sequence.md b/docs/01_run_sequence.md index d6247e6..67c76ea 100644 --- a/docs/01_run_sequence.md +++ b/docs/01_run_sequence.md @@ -1,129 +1,315 @@ -# Run Sequence: What Happens When You Start Tests - -This document walks through the exact order of operations when you run the framework with pytest, what gets called, and where configuration/data is fetched from. - -## High-level flow - -1. You run pytest from PowerShell -2. pytest reads `pytest.ini` and loads configured plugins (including our custom `conftest_plugin`) -3. Test discovery collects tests under `tests/` -4. Session fixtures run: - - `config()` loads YAML configuration - - `lin()` selects and connects the LIN interface (Mock, MUM, or the deprecated BabyLIN) - - `flash_ecu()` optionally flashes the ECU (if enabled) -5. Tests execute using fixtures and call interface methods -6. Our plugin extracts test metadata (Title, Requirements, Steps) from docstrings -7. Reports are written to `reports/report.html` and `reports/junit.xml` - -## Detailed call sequence - -```mermaid -sequenceDiagram - autonumber - participant U as User (PowerShell) - participant P as pytest - participant PI as pytest.ini - participant PL as conftest_plugin.py - participant T as Test Discovery (tests/*) - participant F as Fixtures (conftest.py) - participant C as Config Loader (ecu_framework/config.py) - participant PS as Power Supply (optional) - participant L as LIN Adapter (mock/MUM/BabyLIN) - participant X as HexFlasher (optional) - participant R as Reports (HTML/JUnit) - - U->>P: python -m pytest [args] - P->>PI: Read addopts, markers, plugins - P->>PL: Load custom plugin hooks - P->>T: Collect tests - P->>F: Init session fixtures - F->>C: load_config(workspace_root) - C-->>F: EcuTestConfig (merged dataclasses) - F->>L: Create interface (mock, MUM, or BabyLIN SDK) - L-->>F: Instance ready - F->>L: connect() - alt flash.enabled and hex_path provided - F->>X: HexFlasher(lin).flash_hex(hex_path) - X-->>F: Flash result (ok/fail) - end - opt power_supply.enabled and port provided - Note over PS: owon_psu_quick_demo may open PSU via ecu_framework.power.owon_psu - end - loop for each test - P->>PL: runtest_makereport(item, call) - Note over PL: Parse docstring and attach metadata - P->>L: send()/receive()/request() - L-->>P: Frames or None (timeout) - end - P->>R: Write HTML (with metadata columns) - P->>R: Write JUnit XML -``` - -```text -PowerShell → python -m pytest - ↓ -pytest loads pytest.ini - - addopts: --junitxml, --html, --self-contained-html, -p conftest_plugin - - markers registered - ↓ -pytest collects tests in tests/ - ↓ -Session fixture: config() - → calls ecu_framework.config.load_config(workspace_root) - → determines config file path by precedence - → merges YAML + overrides into dataclasses (EcuTestConfig) - → optionally merges config/owon_psu.yaml (or OWON_PSU_CONFIG) into power_supply - ↓ -Session fixture: lin(config) - → chooses interface by config.interface.type - - mock → ecu_framework.lin.mock.MockBabyLinInterface(...) - - mum → ecu_framework.lin.mum.MumLinInterface(host, lin_device, power_device, ...) - - babylin → ecu_framework.lin.babylin.BabyLinInterface(...) [DEPRECATED] - → lin.connect() - - MUM connect() also powers up the ECU via power_out0 and waits boot_settle_seconds - ↓ -Optional session fixture: flash_ecu(config, lin) - → if config.flash.enabled and hex_path set - → ecu_framework.flashing.HexFlasher(lin).flash_hex(hex_path) - ↓ -Test functions execute - → use the lin fixture to send/receive/request - ↓ -Reporting plugin (conftest_plugin.py) - → pytest_runtest_makereport parses test docstring - → attaches user_properties: title, requirements, steps, expected_result - → pytest-html hooks add Title and Requirements columns - ↓ -Reports written - → reports/report.html (HTML with metadata columns) - → reports/junit.xml (JUnit XML for CI) -``` - -## Where information is fetched from - -- pytest configuration: `pytest.ini` -- YAML config (default): `config/test_config.yaml` -- YAML override via env var: `ECU_TESTS_CONFIG` -- BabyLIN SDK wrapper and SDF path (DEPRECATED): `interface.sdf_path` and `interface.schedule_nr` in YAML -- Test metadata: parsed from each test’s docstring -- Markers: declared in `pytest.ini`, attached in tests via `@pytest.mark.*` - -## Key components involved - -- `tests/conftest.py`: defines `config`, `lin`, and `flash_ecu` fixtures -- `ecu_framework/config.py`: loads and merges configuration into dataclasses -- `ecu_framework/lin/base.py`: abstract LIN interface contract and frame shape -- `ecu_framework/lin/mock.py`: mock behavior for send/receive/request -- `ecu_framework/lin/mum.py`: MUM adapter (Melexis Universal Master via pylin + pymumclient) -- `ecu_framework/lin/babylin.py`: BabyLIN SDK wrapper adapter (DEPRECATED real hardware path via BabyLIN_library.py; emits `DeprecationWarning` on use) -- `ecu_framework/flashing/hex_flasher.py`: placeholder flashing logic -- `conftest_plugin.py`: report customization and metadata extraction - -## Edge cases and behavior - -- If `interface.type` is `babylin` (deprecated) but the SDK wrapper or libraries cannot be loaded, hardware tests are skipped -- If `interface.type` is `mum` but `pylin` / `pymumclient` aren't importable, or `interface.host` is unset, hardware tests are skipped with a clear message -- If `flash.enabled` is true but `hex_path` is missing, flashing fixture skips -- Timeouts are honored in `receive()` and `request()` implementations -- Invalid frame IDs (outside 0x00–0x3F) or data > 8 bytes will raise in `LinFrame` -- MUM `receive()` is master-driven: it requires a frame ID; `receive(id=None)` raises NotImplementedError. Diagnostic frames needing LIN 1.x Classic checksum should use `MumLinInterface.send_raw()`. +# Run Sequence: What Happens When You Start Tests + +This document walks through the exact order of operations when you run +the framework with pytest, what gets called, and where +configuration / data is fetched from. The flow has two layers: + +- **Project-wide** fixtures in `tests/conftest.py` — `config`, `lin`, + `ldf`, `flash_ecu`, `rp`. Apply to every test. +- **Hardware-suite** fixtures in `tests/hardware/conftest.py` — a + session-scoped, autouse PSU power-up plus the public `psu` fixture. + Apply only to tests under `tests/hardware/`. + +## High-level flow + +1. You run pytest from PowerShell (or any shell). +2. pytest reads `pytest.ini` (markers, addopts, `junit_family=legacy`) + and loads `conftest_plugin` for HTML/metadata enrichment. +3. Test discovery collects tests under `tests/`. +4. **Project-wide** session fixtures resolve as needed: + - `config()` loads YAML configuration into typed dataclasses. + - `lin()` selects and connects the LIN adapter (Mock / MUM / + deprecated BabyLIN). + - `ldf()` loads the LDF database (when `interface.ldf_path` is set). + - `flash_ecu()` optionally flashes the ECU. +5. **For hardware tests only**, the session-scoped autouse fixture + `_psu_powers_bench` realizes `_psu_or_none`, which: + - Opens the Owon PSU once via the cross-platform `resolve_port()`, + - Parks it at `config.power_supply.set_voltage` / + `set_current`, + - Enables output and leaves it on for the entire session. + This keeps the ECU powered for every test in the suite — even + tests that don't request `psu` by name. +6. Per-test bodies execute — typically through the helper layer + (`FrameIO`, `AlmTester`, `apply_voltage_and_settle`) rather than + raw `lin.send()` / `lin.receive()`. +7. The `conftest_plugin` parses each test's docstring (Title / + Description / Requirements / Steps / Expected Result) and attaches + the values as JUnit `` entries. +8. At session end the PSU's `safe_off_on_close` sends `output 0` + before releasing the port; reports are written. + +## Detailed call sequence (Mermaid) + +```mermaid +sequenceDiagram + autonumber + participant U as User (shell) + participant P as pytest + participant PI as pytest.ini + participant PL as conftest_plugin.py + participant T as Test discovery (tests/*) + participant CF as tests/conftest.py + participant HCF as tests/hardware/conftest.py + participant C as Config Loader + participant L as LIN Adapter (mock/MUM/BabyLIN) + participant LD as LDF Database + participant PSU as PSU (session-scoped) + participant FH as Helper layer (FrameIO/AlmTester/psu_helpers) + participant X as HexFlasher (optional) + participant R as Reports (HTML/JUnit/Summary) + + U->>P: python -m pytest [args] + P->>PI: addopts, markers, junit_family=legacy + P->>PL: Register custom plugin hooks + P->>T: Collect tests + + rect rgba(200,220,255,0.25) + Note over CF: Project-wide fixtures (every test) + P->>CF: Resolve session fixtures + CF->>C: load_config(workspace_root) + C-->>CF: EcuTestConfig + CF->>L: Create interface (per interface.type) + CF->>L: lin.connect() + L-->>CF: ready (MUM also powers ECU via power_out0) + opt interface.ldf_path set + CF->>LD: LdfDatabase(ldf_path) + end + end + + rect rgba(220,255,220,0.30) + Note over HCF: Hardware-suite fixtures (tests/hardware/*) + P->>HCF: Realize _psu_powers_bench (autouse, session) + HCF->>PSU: resolve_port + open + set V/I + output ON + PSU-->>HCF: powered, leave on for session + end + + alt flash.enabled and hex_path provided + CF->>X: HexFlasher(lin).flash_hex(hex_path) + X-->>CF: ok / fail + end + + loop for each test (function scope) + P->>FH: Test body uses FrameIO.send/receive/read_signal, + Note over FH: AlmTester.* and/or apply_voltage_and_settle + FH->>L: send() / receive() + L-->>FH: Frames / None + FH->>PSU: set_voltage() + measure_voltage_v() (voltage tests) + PSU-->>FH: settled value + P->>PL: pytest_runtest_makereport(item, call) + Note over PL: Parse docstring → user_properties + end + + P->>R: HTML report with metadata columns + P->>R: JUnit XML (junit_family=legacy → record_property entries) + P->>R: summary.md, requirements_coverage.json + + rect rgba(255,220,220,0.30) + Note over PSU: Session teardown + HCF->>PSU: close() → safe_off_on_close sends 'output 0' + end +``` + +## Text flow (project-wide + hardware-suite layers) + +```text +shell → python -m pytest + ↓ +pytest loads pytest.ini + - addopts: -ra --junitxml=… --html=… --tb=short --cov=… + - junit_family = legacy ← required for record_property() round-trip + - markers registered (hardware, mum, babylin (deprecated), + slow, psu_settling, smoke, …) + ↓ +pytest collects tests in tests/ + ↓ +───────────────────────────────────────────────────────────────────── +PROJECT-WIDE fixtures (tests/conftest.py) — apply to every test +───────────────────────────────────────────────────────────────────── +Session: config() + → ecu_framework.config.load_config(workspace_root) + → precedence: in-memory overrides > ECU_TESTS_CONFIG env > + ./config/test_config.yaml > defaults + → optionally merges config/owon_psu.yaml (or OWON_PSU_CONFIG) + into power_supply + → returns EcuTestConfig (typed dataclasses) + +Session: lin(config) + → chooses adapter by interface.type: + mock → ecu_framework.lin.mock.MockBabyLinInterface(...) + mum → ecu_framework.lin.mum.MumLinInterface(host, lin_device, + power_device, …) + babylin → ecu_framework.lin.babylin.BabyLinInterface(...) [DEPRECATED] + → lin.connect() + - MUM also powers the ECU via power_out0 and waits + boot_settle_seconds before sending the first frame + +Session: ldf(config) + → if interface.ldf_path set: LdfDatabase(ldf_path) + → else: skip (tests requesting `ldf` are skipped with a clear msg) + +Session (opt): flash_ecu(config, lin) + → if flash.enabled and flash.hex_path set + → HexFlasher(lin).flash_hex(hex_path) + +Function: rp(record_property) + → convenience wrapper that records both as a JUnit property AND + echoes to captured stdout for fast diagnosis + +───────────────────────────────────────────────────────────────────── +HARDWARE-SUITE fixtures (tests/hardware/conftest.py) — only for +tests under tests/hardware/ +───────────────────────────────────────────────────────────────────── +Session: _psu_or_none(config) + → if power_supply disabled / port unset / unreachable: + yield None (tolerant; does not raise) + → else: + params = SerialParams.from_config(power_supply) + port = resolve_port(power_supply.port, + idn_substr=power_supply.idn_substr, + params=params) + ↑ tries the configured port verbatim, then its + cross-platform translation (COM7 ↔ /dev/ttyS6 on WSL1), + then /dev/ttyUSB* / /dev/ttyACM* on Linux/WSL, + then a full scan_ports() with optional idn_substr filter + psu = OwonPSU(port, params, eol, safe_off_on_close=True) + psu.open() + psu.set_voltage(set_voltage) + psu.set_current(set_current) + psu.set_output(True) ← bench powered ON for session + yield psu + psu.close() ← end of session: 'output 0' first + +Session, autouse: _psu_powers_bench(_psu_or_none) + → realizes _psu_or_none so the bench is powered up even for + tests that don't request `psu` by name (no-op if PSU absent) + +Session: psu(_psu_or_none) + → public alias: skip cleanly if PSU is unavailable + +───────────────────────────────────────────────────────────────────── +TEST BODIES — typically use the helper layer, not lin/psu directly +───────────────────────────────────────────────────────────────────── +Hardware-test helpers (sibling-imported from tests/hardware/): + + frame_io.FrameIO(lin, ldf) + high : send / receive / read_signal (by frame and signal name) + mid : pack / unpack (bytes ↔ signals) + low : send_raw / receive_raw (bypass LDF entirely) + intro : frame, frame_id, frame_length + + alm_helpers.AlmTester(fio, nad) + force_off, read_led_state, wait_for_state, + measure_animating_window, + assert_pwm_matches_rgb, ← uses vendor/rgb_to_pwm.py + assert_pwm_wo_comp_matches_rgb + + psu_helpers + wait_until_settled(psu, target_v, *, tol, interval, timeout) + apply_voltage_and_settle(psu, target_v, *, validation_time, …) + 1. psu.set_voltage(target_v) + 2. poll measure_voltage_v() until within tol (or raise) + 3. sleep validation_time so the firmware-side observer + can detect and republish status + → returns {settled_s, validation_s, final_v, trace} + +Per-file autouse fixtures often layer on a domain baseline: + + test_mum_alm_animation.py:_reset_to_off → alm.force_off() + test_overvolt.py:_park_at_nominal → apply_voltage_and_settle(NOMINAL_V) + force_off + _test_case_template_psu_lin.py:_park_at_nominal → same pattern + +Reporting plugin (conftest_plugin.py) + → pytest_runtest_makereport parses docstring (Title / Description / + Requirements / Test Steps / Expected Result) + → attaches user_properties; pytest-html shows Title + Requirements + columns; junit_family=legacy ensures record_property() round-trips + +Reports written + → reports/report.html (HTML with metadata columns) + → reports/junit.xml (JUnit XML — properties round-trip) + → reports/summary.md (machine-friendly run summary) + → reports/requirements_coverage.json +``` + +## Where information is fetched from + +- pytest configuration: `pytest.ini` (markers, addopts, + `junit_family = legacy`) +- YAML config (default): `config/test_config.yaml` +- YAML override via env var: `ECU_TESTS_CONFIG` +- Per-machine PSU override: `config/owon_psu.yaml` (or + `OWON_PSU_CONFIG`); merged into `power_supply` +- LDF database: `interface.ldf_path` (typically + `vendor/4SEVEN_color_lib_test.ldf`); consumed by the `ldf` fixture + and by `FrameIO` +- RGB→PWM calculator: `vendor/rgb_to_pwm.py`; consumed by + `AlmTester.assert_pwm_*` +- BabyLIN SDF / schedule (DEPRECATED): `interface.sdf_path` and + `interface.schedule_nr` +- Test metadata: parsed from each test's docstring +- Markers: declared in `pytest.ini`, attached in tests via + `@pytest.mark.*` (file-level via `pytestmark = [...]`) + +## Key components involved + +### Project-wide + +- `tests/conftest.py` — defines `config`, `lin`, `ldf`, `flash_ecu`, `rp` +- `conftest_plugin.py` — report customization and metadata extraction +- `ecu_framework/config.py` — YAML → dataclasses +- `ecu_framework/lin/{base,mock,mum,ldf,babylin}.py` — LIN abstraction + and adapters +- `ecu_framework/flashing/hex_flasher.py` — flashing scaffold +- `ecu_framework/power/owon_psu.py` — PSU controller + + `resolve_port()` + +### Hardware-suite (`tests/hardware/`) + +- `conftest.py` — session-scoped autouse PSU fixture (powers the ECU + for the entire session) +- `frame_io.py` — `FrameIO` class (generic LDF-driven I/O) +- `alm_helpers.py` — `AlmTester` class + ALM constants and tolerance + utilities +- `psu_helpers.py` — `wait_until_settled` / + `apply_voltage_and_settle` (settle-then-validate pattern) +- `_test_case_template.py`, `_test_case_template_psu_lin.py` — + copyable starting points (leading underscore → not collected) + +## Edge cases and behaviour + +### LIN side + +- `interface.type == 'babylin'` (deprecated): if the SDK wrapper or + libraries can't load, hardware tests skip cleanly. +- `interface.type == 'mum'`: if `pylin` / `pymumclient` aren't + importable, or `interface.host` is unset, hardware tests skip. +- MUM `receive()` is master-driven: it requires a frame ID; + `receive(id=None)` raises `NotImplementedError`. Diagnostic frames + needing LIN 1.x Classic checksum must go through + `MumLinInterface.send_raw()`. +- `flash.enabled == true` but `hex_path` missing → flashing fixture + skips. +- Invalid frame IDs (outside 0x00–0x3F) or data > 8 bytes raise in + `LinFrame`. + +### PSU / hardware side + +- PSU port resolution: if the configured port can't be opened on this + host (e.g. `COM7` on Linux), `resolve_port()` falls back to its + cross-platform translation (`COM7` → `/dev/ttyS6` on WSL1), then + `/dev/ttyUSB*` / `/dev/ttyACM*`, then a full scan filtered by + `idn_substr`. Returns `None` if nothing responds — the `psu` + fixture then skips cleanly. +- The session-scoped PSU fixture **must not** be closed mid-session. + Tests that perturb voltage **must restore nominal in `finally`**; + they **must not** call `psu.set_output(False)` (would brown out the + ECU and break every later test). +- `apply_voltage_and_settle()` raises `AssertionError` if the rail + doesn't reach the target within `settle_timeout` (default 10 s) — + surfacing real bench problems rather than letting voltage tests + silently assert against the wrong rail. +- `record_property()` / `rp(...)` entries only appear in + `reports/junit.xml` when `junit_family = legacy` is set in + `pytest.ini` (default `xunit2` silently drops them with a + collect-time warning).