ecu-tests/docs/18_test_catalog.md

Test Catalog

Comprehensive description of every test case in the framework — what each one does, what it expects, what hardware it needs, and how to run it. Generated by hand from the source files; rerun pytest --collect-only -q --no-cov to see the live list.

Quick reference

Category	Files	Tests (incl. parametrize expansions)	Hardware?
Unit (pure logic)	6	28	none
Mock-loopback smoke	2	6	none
Plugin self-test	1	1	none
Hardware – MUM	4	12	MUM + ECU
Hardware – BabyLIN (legacy)	4	4	BabyLIN + ECU + Owon PSU
Hardware – Owon PSU	1	1	Owon PSU
Total	18	52	mixed

The numbers count the cases pytest reports when collecting. Some tests are @parametrize-expanded (e.g. test_linframe_invalid_id_raises[-1], [64]) and listed once below with a note on the parameters.

How to run a category

pytest -m "unit"                    # pure unit tests
pytest -m "not hardware"            # everything except hardware (≈ 35 cases)
pytest -m "hardware and mum"        # MUM-only hardware tests
pytest -m "hardware and babylin"    # legacy BabyLIN hardware tests
pytest -m "hardware and not slow"   # hardware excluding the slow auto-addressing test

---

1. Unit tests — tests/unit/

Pure-Python tests that don't touch hardware or external I/O. Run on every PR.

1.1 test_linframe.py — LinFrame validation

Source: tests/unit/test_linframe.py

Test	Markers	Purpose
test_linframe_accepts_valid_ranges	unit	Construct a LinFrame(id=0x3F, data=8 bytes of zero) and assert id/length round-trip cleanly. Ensures the maximum legal LIN classic ID and 8-byte payload are accepted.
test_linframe_invalid_id_raises[-1] / [64]	unit	Parametrized: LinFrame(id=-1) and LinFrame(id=0x40) must raise ValueError. Confirms the 0x00–0x3F clamp on classic LIN IDs.
test_linframe_too_long_raises	unit	LinFrame(id=0x01, data=9 bytes) must raise ValueError. Confirms the 8-byte payload upper bound.

Why it matters: LinFrame is the type every adapter (Mock/MUM/BabyLIN) hands back to tests. If validation drifts, all downstream tests get more permissive silently.

---

1.2 test_config_loader.py — YAML configuration precedence

Source: tests/unit/test_config_loader.py

Test	Markers	Purpose
test_config_precedence_env_overrides	unit	Writes a temp YAML with interface.type: babylin / channel: 7, points ECU_TESTS_CONFIG at it, then loads with overrides={"interface": {"channel": 9}}. Asserts the YAML's type made it through and the in-code override beat the YAML's channel.
test_config_defaults_when_no_file	unit	With no ECU_TESTS_CONFIG and no workspace root, load_config() must return defaults (type: mock, flash.enabled: false).

Precedence order asserted: in-code overrides > ECU_TESTS_CONFIG env > config/test_config.yaml > built-in defaults.

---

1.3 test_babylin_adapter_mocked.py — BabyLIN adapter error path

Source: tests/unit/test_babylin_adapter_mocked.py

Test	Markers	Purpose
test_connect_sdf_error_raises	unit	Inject a fake BabyLIN wrapper whose BLC_loadSDF returns a non-OK code. BabyLinInterface.connect() must raise RuntimeError. Validates that SDK error codes during SDF download surface as Python exceptions instead of being silently ignored.

---

1.4 test_mum_adapter_mocked.py — MUM adapter plumbing

Source: tests/unit/test_mum_adapter_mocked.py

All cases inject fake pymumclient and pylin modules so the adapter can be exercised with no MUM hardware.

Test	Markers	Purpose
test_connect_opens_mum_and_powers_up	unit	connect() calls MelexisUniversalMaster.open_all(host), linmaster.setup(), sets lin_dev.baudrate, and powers up the ECU exactly once.
test_disconnect_powers_down_and_tears_down	unit	disconnect() calls power_control.power_down() and linmaster.teardown() exactly once each.
test_send_publishes_master_frame	unit	lin.send(LinFrame(0x0A, 8 bytes)) calls lin_dev.send_message(master_to_slave=True, frame_id=0x0A, data_length=8, data=[...]).
test_receive_uses_frame_lengths_default	unit	lin.receive(id=0x11) reads the configured length (4) from the default frame_lengths map and returns the slave bytes wrapped in a LinFrame.
test_receive_returns_none_on_pylin_exception	unit	If pylin raises during send_message(master_to_slave=False, ...), receive() must return None (treated as timeout). Stops tests from having to wrap every receive in try/except.
test_receive_without_id_raises	unit	lin.receive(id=None) must raise NotImplementedError. The MUM is master-driven; passive listen is unsupported.
test_send_raw_uses_classic_checksum_path	unit	lin.send_raw(bytes) calls transport_layer.ld_put_raw(data, baudrate=19200). This is the path BSM-SNPD diagnostic frames need (Classic checksum).
test_power_cycle_calls_down_then_up	unit	lin.power_cycle(wait=0) issues at least one extra power_down() and the matching power_up() on top of the connect-time power up.

---

1.5 test_ldf_database.py — LDF parser wrapper

Source: tests/unit/test_ldf_database.py

Module is skipped automatically if ldfparser isn't installed. Uses vendor/4SEVEN_color_lib_test.ldf as fixture data.

Test	Markers	Purpose
test_loads_metadata	unit	db.protocol_version is one of 1.3/2.0/2.1 and db.baudrate == 19200 for the 4SEVEN LDF.
test_lookup_by_name_and_id	unit	db.frame("ALM_Req_A") and db.frame(0x0A) return the same frame; id/name/length match the LDF Frames block.
test_unknown_frame_raises	unit	db.frame("not_a_real_frame") raises FrameNotFound.
test_signal_layout_matches_ldf	unit	frame.signal_layout() for ALM_Req_A contains the exact (start_bit, name, width) tuples from the LDF (spot-checks AmbLightColourRed, AmbLightUpdate, AmbLightMode, AmbLightLIDTo).
test_pack_kwargs_full_payload	unit	frame.pack(...) with all signals provided produces an 8-byte payload ffffffff00000101.
test_pack_unspecified_signals_use_init_value	unit	frame.pack() with no kwargs uses each signal's LDF init_value. Verified by decoding the packed output for ColorConfigFrameRed (which has non-zero init values like 5665).
test_pack_dict_argument	unit	frame.pack({...}) and frame.pack(**{...}) produce identical bytes.
test_pack_rejects_args_and_kwargs_together	unit	frame.pack({"X": 1}, Y=2) raises TypeError.
test_unpack_round_trip	unit	A non-trivial value set (RGB, intensity, mode bits, LID range) packs and unpacks back to the same dict.
test_alm_status_decode_real_payload	unit	unpack(b"\\x07\\x00\\x00\\x00") on ALM_Status yields ALMNadNo == 7.
test_frame_lengths_includes_all_unconditional_frames	unit	db.frame_lengths() contains every unconditional frame ID with a positive length (sanity: ALM_Req_A=8, ALM_Status=4, ConfigFrame=3).
test_frames_returns_wrapped_frame_objects	unit	db.frames() returns wrapped Frame objects whose names cover the expected set (ALM_Req_A, ALM_Status, ConfigFrame…).
test_ldf_repr_does_not_explode	unit	repr(db) includes LdfDatabase and doesn't raise.
test_missing_file_raises_filenotfounderror	unit	LdfDatabase(missing_path) raises FileNotFoundError.

---

1.6 test_hex_flasher.py — flashing scaffold

Source: tests/unit/test_hex_flasher.py

Test	Markers	Purpose
test_hex_flasher_sends_basic_sequence	unit	Writes a minimal Intel HEX (EOF-only) and runs HexFlasher(stub_lin).flash_hex(path). Asserts no exception and that lin.sent is a list. Placeholder until the flasher is fleshed out with UDS — once real UDS is wired in, this test gains real assertions about the byte sequence.

---

2. Mock-loopback smoke — tests/

Tests that exercise the full LinInterface API (send / receive / request) using either the in-process Mock adapter or the BabyLIN adapter with a mock SDK wrapper.

2.1 test_smoke_mock.py — Mock adapter end-to-end

Source: tests/test_smoke_mock.py

Module-local lin fixture forces MockBabyLinInterface regardless of the central config, so these always run as mock-only tests.

Test	Markers	Purpose
TestMockLinInterface::test_mock_send_receive_echo	smoke req_001 req_003	Send LinFrame(0x12, [1,2,3]) and receive it back through the mock's loopback. ID and data must match exactly.
TestMockLinInterface::test_mock_request_synthesized_response	smoke req_002	lin.request(id=0x21, length=4) returns a deterministic frame where data[i] == (id + i) & 0xFF. The mock implements this pattern so request/response logic can be tested without hardware.
TestMockLinInterface::test_mock_receive_timeout_behavior	smoke req_004	lin.receive(id=0xFF, timeout=0.1) (no matching frame queued) returns None and doesn't block longer than the requested timeout.
TestMockLinInterface::test_mock_frame_validation_boundaries[…]	boundary req_001 req_003	Parametrized 4 ways: (id, payload) ∈ {(0x01, [0x55]), (0x3F, [0xAA,0x55]), (0x20, 5 bytes), (0x15, 8 bytes)}. Each frame round-trips through send/receive with byte-for-byte integrity. Covers the legal LIN ID and payload-length boundaries.

---

2.2 test_babylin_wrapper_mock.py — BabyLIN adapter against a mocked SDK

Source: tests/test_babylin_wrapper_mock.py

Constructs BabyLinInterface(wrapper_module=mock_bl) so the adapter exercises real code paths without needing the BabyLIN native library.

Test	Markers	Purpose
test_babylin_sdk_adapter_with_mock_wrapper	babylin smoke req_001	Connect (discover port, open, load SDF, start schedule) → send(LinFrame(0x12, [0xAA,0x55,0x01])) → receive(timeout=0.1). The mock wrapper echoes the transmitted bytes; the test asserts ID and data round-trip.
test_babylin_master_request_with_mock_wrapper[…]	babylin smoke req_001	Parametrized 2 ways. vendor.mock_babylin_wrapper-True: full mock with BLC_sendRawMasterRequest(channel, id, length) — expects the deterministic pattern. _MockBytesOnly-False: shim where only the bytes signature is supported; the adapter falls back to sending zeros and the response is asserted to be zeros of the requested length. Together these cover both SDK signatures the adapter must handle.

---

3. Plugin self-test — tests/plugin/

3.1 test_conftest_plugin_artifacts.py

Source: tests/plugin/test_conftest_plugin_artifacts.py

Test	Markers	Purpose
test_plugin_writes_artifacts	unit	Uses pytest's pytester to run a synthetic test in a temp dir with the reporting plugin loaded. Asserts reports/requirements_coverage.json is created with REQ-001 mapped, that reports/summary.md exists, and that the JSON references the generated report.html and junit.xml. Validates the plugin's full artifact pipeline end-to-end.

---

4. Hardware – MUM (Melexis Universal Master)

Tests gated on interface.type == "mum". All require:

• A reachable MUM (default 192.168.7.2 over USB-RNDIS)
• Melexis pylin and pymumclient Python packages installed
• An ECU wired to the MUM's lin0 and powered through power_out0
• interface.ldf_path pointing at the LDF that matches the ECU

4.1 test_e2e_mum_led_activate.py

Source: tests/hardware/test_e2e_mum_led_activate.py

Test	Markers	Purpose
test_mum_e2e_power_on_then_led_activate	hardware mum	The "smoke + LED on" flow. Reads ALM_Status, decodes ALMNadNo via the LDF, builds an ALM_Req_A payload (full-white RGB at full intensity, immediate setpoint, mode 0) targeting that NAD, sends it, and re-reads ALM_Status to confirm the bus is still alive afterward.

Notes:

• Power-up is implicit — the MUM lin fixture already calls power_control.power_up() on connect.
• Frame layouts come from the ldf fixture, not hand-coded byte positions.

4.2 test_mum_alm_animation.py

Source: tests/hardware/test_mum_alm_animation.py

Suite of automated checks for the four behaviour buckets in vendor/automated_lin_test/test_animation.py. A module-scoped fixture reads the ECU's NAD once; an autouse fixture forces an OFF baseline before and after every test so cases don't bleed state into each other.

Test	Markers	Purpose
test_mode0_immediate_setpoint_drives_led_on	hardware mum	AmbLightMode=0, bright RGB+I, target single NAD. Polls ALMLEDState and asserts it reaches LED_ON within ~1 s.
test_mode1_fade_passes_through_animating	hardware mum	AmbLightMode=1 with AmbLightDuration=10 (≈ 2 s expected). Asserts ALMLEDState enters ANIMATING during the fade and reaches LED_ON afterward.
test_duration_scales_with_lsb[5-0.6] / [10-0.6]	hardware mum	Parametrized: with Duration=N, the ANIMATING window must be within ±0.6 s of N × 0.2 s. Loose tolerance accounts for the 50 ms poll cadence and bus latency.
test_update1_save_does_not_apply_immediately	hardware mum	AmbLightUpdate=1 (Save) with bright payload — ALMLEDState must NOT transition to ANIMATING or LED_ON. Verifies save-only semantics.
test_update2_apply_runs_saved_command	hardware mum	After a save (Update=1), an apply (Update=2) with throwaway payload should execute the saved command — ANIMATING is observed.
test_update3_discard_then_apply_is_noop	hardware mum	Save → Discard (Update=3) → Apply. Apply must be a no-op (no ANIMATING, no LED_ON). Verifies the discard clears the saved buffer.
test_lid_broadcast_targets_node	hardware mum	AmbLightLIDFrom=0x00, AmbLightLIDTo=0xFF with bright RGB. Node must react and reach LED_ON, regardless of its actual NAD.
test_lid_invalid_range_is_ignored	hardware mum	LIDFrom > LIDTo (e.g. 0x14 > 0x0A). Node must ignore the frame — ALMLEDState stays at OFF baseline.

Caveats:

• Visual properties (color, smoothness of fade) cannot be asserted without a camera. These tests assert only what the LIN bus exposes (ALMLEDState transitions, ANIMATING duration). For a human-verified visual run, use the original vendor/automated_lin_test/test_animation.py.
• test_duration_scales_with_lsb polls every 50 ms; the tolerance is intentionally loose. Tighten it once you've measured your firmware's actual jitter.

4.3 test_mum_auto_addressing.py

Source: tests/hardware/test_mum_auto_addressing.py

Test	Markers	Purpose
test_bsm_auto_addressing_changes_nad	hardware mum slow	Drives the full BSM-SNPD sequence (INIT → 16× ASSIGN → STORE → FINALIZE) with a target NAD different from the ECU's current one, then re-reads ALM_Status and asserts ALMNadNo == target. Always restores the original NAD in a finally block (the restore result is recorded as report properties). Uses lin.send_raw() so the LIN 1.x Classic checksum is used — Enhanced would be silently rejected by the firmware.

Notes:

• Marked slow because the full sequence runs in ~3-4 seconds (two BSM cycles plus settle). Skip with -m "hardware and mum and not slow".
• Restore is best-effort: if the second BSM cycle fails, the bench stays at the target NAD. The restore failure is visible as restore_warning / restore_error in the report properties.

4.4 test_e2e_power_on_lin_smoke.py (legacy, BabyLIN-marked)

Source: tests/hardware/test_e2e_power_on_lin_smoke.py

Despite living in tests/hardware/, this file targets the BabyLIN adapter (it predates the MUM migration). See section 5.4.

---

5. Hardware – BabyLIN (legacy)

Tests gated on interface.type == "babylin". Require:

• BabyLIN device + native libraries placed under vendor/
• An SDF compiled from your LDF, path supplied via interface.sdf_path
• For the E2E test: an Owon PSU on a serial port (the BabyLIN doesn't supply ECU power)

5.1 test_babylin_hardware_smoke.py

Source: tests/test_babylin_hardware_smoke.py

Test	Markers	Purpose
test_babylin_connect_receive_timeout	hardware babylin	Minimal sanity: open the BabyLIN device via the configured lin fixture and call lin.receive(timeout=0.2). Accepts either a LinFrame or None (timeout) — verifies the adapter is functional and not crashing.

5.2 test_babylin_hardware_schedule_smoke.py

Source: tests/test_babylin_hardware_schedule_smoke.py

Test	Markers	Purpose
test_babylin_sdk_example_flow	hardware babylin smoke	Verifies interface.type == "babylin" and an sdf_path is set, then exercises the receive path while the configured schedule_nr runs. Mirrors the vendor example flow (open / load SDF / start schedule / receive). Accepts either a frame or a timeout.

5.3 test_hardware_placeholder.py

Source: tests/test_hardware_placeholder.py

Test	Markers	Purpose
test_babylin_placeholder	hardware babylin	Always passes. Used to confirm the marker filter and CI plumbing for hardware jobs without requiring any specific device behaviour.

5.4 test_e2e_power_on_lin_smoke.py

Source: tests/hardware/test_e2e_power_on_lin_smoke.py

Test	Markers	Purpose
test_e2e_power_on_then_cco_rgb_activate	hardware babylin	Full BabyLIN E2E. Powers the ECU through the Owon PSU, switches to the LDF's CCO schedule via lin.start_schedule("CCO") (which resolves the schedule name to its index using BLC_SDF_getScheduleNr), publishes an ALM_Req_A payload with full-white RGB at full intensity, captures bus traffic for ~1 s, and asserts at least one frame was observed. Always disables PSU output in finally.

Notes:

• This test was the original E2E target before the MUM migration. It still works as a BabyLIN smoke test if you flip interface.type: babylin and provide a valid SDF.
• The Owon PSU section of config.power_supply must be enabled (port, set_voltage, set_current, do_set: true).

---

6. Hardware – Owon PSU only

6.1 test_owon_psu.py

Source: tests/hardware/test_owon_psu.py

Test	Markers	Purpose
test_owon_psu_idn_and_optional_set	hardware	Independent of any LIN adapter. Skips unless power_supply.enabled: true and power_supply.port is set. Opens the configured serial port, queries *IDN? (asserts non-empty; optionally checks idn_substr), reads output?, and — if do_set: true — sets V/I, briefly enables output, measures back, then disables. All values are recorded as report properties.

Notes:

• Useful as a pure-PSU bench check before running any LIN E2E test.
• Settings can live in config/test_config.yaml (central) or config/owon_psu.yaml (per-machine override; the latter wins).

---

Test naming conventions

When adding new tests, follow these patterns so the catalog stays scannable:

• Unit tests live in tests/unit/ and carry @pytest.mark.unit. Filename starts with test_<thing>_<scope> (e.g., test_mum_adapter_mocked.py).
• Mock smoke tests live in tests/ and use either the in-process Mock adapter (override the lin fixture locally) or an injected SDK mock wrapper.
• Hardware tests live in tests/hardware/ (preferred) or tests/ (legacy) and carry @pytest.mark.hardware plus an adapter marker (mum / babylin).
• Slow tests (>5 s) carry @pytest.mark.slow so they can be excluded with -m "not slow".
• Requirement traceability is via req_NNN markers on the test function and a Requirements: line in the docstring (parsed by the reporting plugin).

Docstring format

The reporting plugin extracts these fields from each test's docstring and renders them in the HTML report:

"""
Title: <short title>

Description:
    <what the test validates and why>

Requirements: REQ-001, REQ-002

Test Steps:
    1. <step one>
    2. <step two>

Expected Result:
    <succinct expected outcome>
"""

See docs/03_reporting_and_metadata.md and docs/15_report_properties_cheatsheet.md for the full schema.

Related docs

• docs/12_using_the_framework.md — How to actually run the various suites
• docs/04_lin_interface_call_flow.md — What send / receive do per adapter
• docs/16_mum_internals.md — MUM adapter implementation details
• docs/17_ldf_parser.md — ldf fixture and Frame.pack / unpack
• docs/06_requirement_traceability.md — How req_NNN markers feed the coverage JSON