diff --git a/docs/01_run_sequence.md b/docs/01_run_sequence.md index 67c76ea..23664e8 100644 --- a/docs/01_run_sequence.md +++ b/docs/01_run_sequence.md @@ -257,7 +257,7 @@ Reports written - `tests/conftest.py` — defines `config`, `lin`, `ldf`, `flash_ecu`, `rp` - `conftest_plugin.py` — report customization and metadata extraction -- `ecu_framework/config.py` — YAML → dataclasses +- `ecu_framework/config/loader.py` — YAML → dataclasses (re-exported via `ecu_framework.config`) - `ecu_framework/lin/{base,mock,mum,ldf,babylin}.py` — LIN abstraction and adapters - `ecu_framework/flashing/hex_flasher.py` — flashing scaffold diff --git a/docs/23_config_loader_internals.md b/docs/23_config_loader_internals.md new file mode 100644 index 0000000..04cd565 --- /dev/null +++ b/docs/23_config_loader_internals.md @@ -0,0 +1,194 @@ +# Configuration Loader Internals + +This document explains *how* the configuration loader is implemented. For the +user-facing "what can I configure and where does it come from" perspective, see +[`02_configuration_resolution.md`](02_configuration_resolution.md). The two are +companions: `02` answers "what do I write in YAML?", this file answers "what +does the loader do with what I wrote?". + +File: `ecu_framework/config/loader.py` + +## Pipeline at a glance + +```text + defaults (dict) + └─▶ merge YAML at $ECU_TESTS_CONFIG (if env set & file exists) + └─▶ merge YAML at /config/test_config.yaml (if exists) + └─▶ merge PSU side-channel (env OWON_PSU_CONFIG or + /config/owon_psu.yaml) + └─▶ merge in-memory overrides (caller-supplied) + └─▶ coerce types & build EcuTestConfig +``` + +Two layers run sequentially: + +1. **Dict layer** — every source contributes a plain `dict`. They are merged + with `_deep_update` so nested sections combine key-by-key. +2. **Dataclass layer** — once merged, `_to_dataclass` casts the values to their + declared types and constructs `EcuTestConfig`. This is the boundary at which + YAML's type fuzziness stops. + +Keeping the merge in the dict layer (rather than merging dataclasses) makes the +precedence story trivial: it's just a sequence of writes into one dict, and the +last writer wins. + +## Precedence — and why it reads "backwards" + +The `load_config` docstring lists precedence highest-to-lowest: + +| Rank | Source | Where in code | +|---|---|---| +| 1 (highest) | `overrides` dict passed to `load_config` | Applied **last** | +| 2 | YAML at `$ECU_TESTS_CONFIG` | Applied if env points at an existing file | +| 3 | YAML at `/config/test_config.yaml` | Fallback when env unset | +| 4 (lowest) | Built-in defaults | The starting `base` dict | + +In the implementation, sources are *applied* in reverse order of that table +(lowest → highest). That's exactly what "highest precedence" means here: +each merge step overwrites earlier values for the same key, so the **last** +writer wins. The "1) ... 4)" comments inside `load_config` annotate by +precedence rank, not by call order. + +## `_deep_update` — the merge semantics + +```python +def _deep_update(base, updates): + for k, v in updates.items(): + if isinstance(v, dict) and isinstance(base.get(k), dict): + base[k] = _deep_update(base[k], v) + else: + base[k] = v + return base +``` + +**Rules:** + +- Dict-on-both-sides → recurse, so nested overlays don't clobber siblings. + This is what lets a YAML file override just `interface.bitrate` without + re-stating the rest of the `interface` block. +- Anything else (scalar, list, mismatched types) → replace wholesale. +- **Lists are replaced, not concatenated.** This is deliberate: list-concat + semantics surprise users who expect "set this list to X" to mean exactly that. + If concatenation is ever needed for a specific field, do it explicitly at the + call site, not in the merge primitive. +- Mutation happens in place; the return value is the same `base` object, + returned for chaining convenience (used when merging the PSU side-channel). + +## `_to_dataclass` — defensive type coercion + +YAML's type inference is generous: `"19200"` (quoted) comes through as a string, +`"true"` is not a bool, and hex-keyed mappings may arrive as either int or +string keys depending on the YAML writer. Rather than propagate that fuzziness, +the loader casts at the dataclass boundary: + +```python +type=str(iface.get("type", "mock")).lower(), +channel=int(iface.get("channel", 1)), +bitrate=int(iface.get("bitrate", 19200)), +... +``` + +Casts that fail raise — and that's the right behavior. A config value that +can't be interpreted is a bug to surface early, not silently fall back from. + +### Special-case: `frame_lengths` keys + +`frame_lengths` maps a LIN frame ID (int) to a payload length (int). YAML can +write the key as a hex int (`0x0A`), a decimal int (`10`), or a quoted string +(`"0x0A"`). Coercion handles all three: + +```python +key = int(k, 0) if isinstance(k, str) else int(k) +``` + +`int(k, 0)` with base `0` means "infer from prefix" — `"0x0A"` parses as hex, +`"10"` as decimal. Entries that fail to parse are skipped silently rather than +aborting the whole load, because one typo in a frame-length map shouldn't +prevent the rest of the configuration from coming up. + +## PSU side-channel + +Power-supply settings (COM port, baudrate, IDN substring) are typically +**bench-specific** and shouldn't be committed alongside test config. The loader +honors a dedicated overlay file just for the `power_supply` section: + +- `$OWON_PSU_CONFIG` (env var → path) wins, else +- `/config/owon_psu.yaml` if it exists. + +This file is deep-merged into the existing `power_supply` block, so the main +YAML can still provide defaults (e.g. `idn_substr: OWON`) while the bench file +overrides only the parts that vary by machine. Recommended workflow: + +``` +config/test_config.yaml # committed; common defaults +config/owon_psu.yaml # gitignored; per-bench serial settings +``` + +## Dataclass schema quirks + +### Forward reference: `EcuTestConfig.power_supply` + +```python +@dataclass +class EcuTestConfig: + ... + power_supply: "PowerSupplyConfig" = field(default_factory=lambda: PowerSupplyConfig()) + +@dataclass +class PowerSupplyConfig: + ... +``` + +`PowerSupplyConfig` is referenced *before* it is defined. This works because: + +1. `from __future__ import annotations` (PEP 563) turns *all* type annotations + into strings at module load time, so `"PowerSupplyConfig"` as an annotation + never triggers a name lookup. +2. The `default_factory` is a lambda, which defers evaluation of the bare name + `PowerSupplyConfig` until `EcuTestConfig()` is actually instantiated — by + which point the module body has finished executing and the name is bound. + +The ordering is intentional: `EcuTestConfig` is the most-used type, so it lives +near the top of the file where readers find it first. If you ever drop the +`from __future__ import annotations` line, this ordering breaks; the lambda +default would still work, but the string annotation would need updating. + +### Mutable defaults must use `default_factory` + +`field(default_factory=dict)` (and `default_factory=InterfaceConfig`, +`default_factory=lambda: PowerSupplyConfig()`) is required because Python +shares default values across instances by default. Using `field(default={})` +on a dataclass field is a `ValueError` at class-creation time — the +`default_factory` form is the only correct way. + +## Known wart: defaults live in two places + +The defaults for every field exist twice: + +1. As dataclass field defaults — e.g. `type: str = "mock"` on `InterfaceConfig`. +2. As entries in the `base` dict inside `load_config`. + +Both must agree, and a drift between them would be silently wrong (the +loader's defaults would win for the YAML path, while the dataclass defaults +would win for callers that construct `InterfaceConfig()` directly). + +Why it's still this way: the dict is needed because `_deep_update` operates on +dicts; the dataclass defaults are needed because callers may construct configs +directly without going through `load_config`. If a third construction path +appears, extract defaults to a single `DEFAULTS` mapping that both layers read +from. + +## Test surface + +Unit tests live in `tests/unit/test_config_loader.py`. They cover the +override precedence chain and the dataclass-construction defaults. When +adding a new field, add at minimum: + +1. The dataclass field with a default. +2. The matching default in the `base` dict in `load_config`. +3. The matching cast line in `_to_dataclass`. +4. A unit test asserting it round-trips through `load_config(overrides=...)`. + +Skipping (3) is the most common bug — the field will appear to work because +the dataclass default carries it, but YAML/env overlays for that field will +be silently dropped. diff --git a/docs/DEVELOPER_COMMIT_GUIDE.md b/docs/DEVELOPER_COMMIT_GUIDE.md index 17acea3..5e51e6e 100644 --- a/docs/DEVELOPER_COMMIT_GUIDE.md +++ b/docs/DEVELOPER_COMMIT_GUIDE.md @@ -5,7 +5,7 @@ This guide explains exactly what to commit to source control for this repository ## Commit these files ### Core framework (source) -- `ecu_framework/config.py` +- `ecu_framework/config/` (`__init__.py`, `loader.py`) - `ecu_framework/lin/base.py` - `ecu_framework/lin/mock.py` - `ecu_framework/lin/babylin.py` (deprecated, retained for backward compatibility) diff --git a/ecu_framework/config.py b/ecu_framework/config.py deleted file mode 100644 index 80c4c00..0000000 --- a/ecu_framework/config.py +++ /dev/null @@ -1,270 +0,0 @@ -from __future__ import annotations # Postponed annotations for forward references and speed - -import os # For environment variables and filesystem checks -import pathlib # Path handling across platforms -from dataclasses import dataclass, field # Lightweight typed containers -from typing import Any, Dict, Optional # Type hints for clarity - -import yaml # Safe YAML parsing for configuration files - - -@dataclass -class FlashConfig: - """Flashing-related configuration. - - enabled: Whether to trigger flashing at session start. - hex_path: Path to the firmware HEX file (if any). - """ - - enabled: bool = False # Off by default - hex_path: Optional[str] = None # No default file path - - -@dataclass -class InterfaceConfig: - """LIN interface configuration. - - type: Adapter type — "mock" (simulated), "mum" (Melexis Universal Master, current), - or "babylin" (DEPRECATED BabyLIN SDK). - channel: Channel index to use (0-based in most SDKs); BabyLIN-specific (deprecated). - bitrate: Effective LIN bitrate; the MUM uses this directly, the BabyLIN SDF may override. - dll_path: DEPRECATED. Legacy/optional pointer to vendor DLLs when using ctypes (not used by SDK wrapper). - node_name: Optional friendly name for display/logging. - func_names: DEPRECATED. Legacy mapping for ctypes function names; ignored by SDK wrapper. - sdf_path: DEPRECATED. Path to the SDF to load on connect (BabyLIN only). - schedule_nr: DEPRECATED. Schedule index to start after connect (BabyLIN only). -1 = skip. - host: MUM IP address (MUM only). - lin_device: MUM LIN device name (MUM only, default 'lin0'). - power_device: MUM power-control device name (MUM only, default 'power_out0'). - boot_settle_seconds: Delay after MUM power-up before sending the first frame. - frame_lengths: Optional map of frame_id (int) -> data length (int) used by the - MUM adapter when receiving slave-published frames. - """ - - type: str = "mock" # "mock", "mum", or "babylin" (deprecated) - channel: int = 1 - bitrate: int = 19200 - dll_path: Optional[str] = None # deprecated (BabyLIN) - node_name: Optional[str] = None - func_names: Dict[str, str] = field(default_factory=dict) # deprecated (BabyLIN) - # BabyLIN-specific (deprecated) - sdf_path: Optional[str] = None - schedule_nr: int = 0 - # MUM-specific - host: Optional[str] = None - lin_device: str = "lin0" - power_device: str = "power_out0" - boot_settle_seconds: float = 0.5 - frame_lengths: Dict[int, int] = field(default_factory=dict) - # Optional LDF path; when set, tests/fixtures can load an LdfDatabase - # and the MUM adapter auto-merges the LDF's frame lengths into its map. - ldf_path: Optional[str] = None - - -@dataclass -class EcuTestConfig: - """Top-level, fully-typed configuration for the framework. - - interface: Settings for LIN communication (mock, MUM, or the deprecated BabyLIN). - flash: Optional flashing behavior configuration. - """ - - interface: InterfaceConfig = field(default_factory=InterfaceConfig) - flash: FlashConfig = field(default_factory=FlashConfig) - # Serial power supply (e.g., Owon) configuration - # Test code can rely on these values to interact with PSU if enabled - power_supply: "PowerSupplyConfig" = field(default_factory=lambda: PowerSupplyConfig()) - - -@dataclass -class PowerSupplyConfig: - """Serial power supply configuration (e.g., Owon PSU). - - enabled: Whether PSU tests/features should be active. - port: Serial device (e.g., COM4 on Windows, /dev/ttyUSB0 on Linux). - baudrate/timeout/eol: Basic line settings; eol often "\n" or "\r\n". - parity: One of "N", "E", "O". - stopbits: 1 or 2. - xonxoff/rtscts/dsrdtr: Flow control flags. - idn_substr: Optional substring to assert in *IDN? responses. - do_set/set_voltage/set_current: Optional demo/test actions. - """ - - enabled: bool = False - port: Optional[str] = None - baudrate: int = 115200 - timeout: float = 1.0 - eol: str = "\n" - parity: str = "N" - stopbits: float = 1.0 - xonxoff: bool = False - rtscts: bool = False - dsrdtr: bool = False - idn_substr: Optional[str] = None - do_set: bool = False - set_voltage: float = 1.0 - set_current: float = 0.1 - - -DEFAULT_CONFIG_RELATIVE = pathlib.Path("config") / "test_config.yaml" # Default config path relative to repo root -ENV_CONFIG_PATH = "ECU_TESTS_CONFIG" # Env var to override config file location - - -def _deep_update(base: Dict[str, Any], updates: Dict[str, Any]) -> Dict[str, Any]: - """Recursively merge dict 'updates' into dict 'base'. - - - Nested dicts are merged by key - - Scalars/collections at any level are replaced entirely - - Mutation occurs in-place on 'base' and the same object is returned - """ - for k, v in updates.items(): # Iterate all update keys - if isinstance(v, dict) and isinstance(base.get(k), dict): # Both sides dict → recurse - base[k] = _deep_update(base[k], v) - else: # Otherwise replace - base[k] = v - return base # Return the mutated base for chaining - - -def _to_dataclass(cfg: Dict[str, Any]) -> EcuTestConfig: - """Convert a merged plain dict config into strongly-typed dataclasses. - - Defensive casting is used to ensure correct types even if YAML contains strings. - """ - iface = cfg.get("interface", {}) # Sub-config for interface - flash = cfg.get("flash", {}) # Sub-config for flashing - psu = cfg.get("power_supply", {}) # Sub-config for power supply - # Coerce frame_lengths keys to int (YAML may parse numeric keys as int already, - # but accept hex strings like "0x0A: 8" too). - raw_fl = iface.get("frame_lengths", {}) or {} - frame_lengths: Dict[int, int] = {} - if isinstance(raw_fl, dict): - for k, v in raw_fl.items(): - try: - key = int(k, 0) if isinstance(k, str) else int(k) - frame_lengths[key] = int(v) - except (TypeError, ValueError): - continue - - return EcuTestConfig( - interface=InterfaceConfig( - type=str(iface.get("type", "mock")).lower(), - channel=int(iface.get("channel", 1)), - bitrate=int(iface.get("bitrate", 19200)), - dll_path=iface.get("dll_path"), - node_name=iface.get("node_name"), - func_names=dict(iface.get("func_names", {}) or {}), - sdf_path=iface.get("sdf_path"), - schedule_nr=int(iface.get("schedule_nr", 0)), - host=iface.get("host"), - lin_device=str(iface.get("lin_device", "lin0")), - power_device=str(iface.get("power_device", "power_out0")), - boot_settle_seconds=float(iface.get("boot_settle_seconds", 0.5)), - frame_lengths=frame_lengths, - ldf_path=iface.get("ldf_path"), - ), - flash=FlashConfig( - enabled=bool(flash.get("enabled", False)), # Coerce to bool - hex_path=flash.get("hex_path"), # Optional hex path - ), - power_supply=PowerSupplyConfig( - enabled=bool(psu.get("enabled", False)), - port=psu.get("port"), - baudrate=int(psu.get("baudrate", 115200)), - timeout=float(psu.get("timeout", 1.0)), - eol=str(psu.get("eol", "\n")), - parity=str(psu.get("parity", "N")), - stopbits=float(psu.get("stopbits", 1.0)), - xonxoff=bool(psu.get("xonxoff", False)), - rtscts=bool(psu.get("rtscts", False)), - dsrdtr=bool(psu.get("dsrdtr", False)), - idn_substr=psu.get("idn_substr"), - do_set=bool(psu.get("do_set", False)), - set_voltage=float(psu.get("set_voltage", 1.0)), - set_current=float(psu.get("set_current", 0.1)), - ), - ) - - -def load_config(workspace_root: Optional[str] = None, overrides: Optional[Dict[str, Any]] = None) -> EcuTestConfig: - """Load configuration from YAML file, environment, overrides, or defaults. - - Precedence (highest to lowest): - 1. in-memory 'overrides' dict - 2. YAML file specified by env var ECU_TESTS_CONFIG - 3. YAML at ./config/test_config.yaml (relative to workspace_root) - 4. built-in defaults in this function - """ - # Start with built-in defaults; minimal, safe baseline - base: Dict[str, Any] = { - "interface": { - "type": "mock", # mock by default for developer friendliness - "channel": 1, - "bitrate": 19200, - }, - "flash": { - "enabled": False, - "hex_path": None, - }, - "power_supply": { - "enabled": False, - "port": None, - "baudrate": 115200, - "timeout": 1.0, - "eol": "\n", - "parity": "N", - "stopbits": 1.0, - "xonxoff": False, - "rtscts": False, - "dsrdtr": False, - "idn_substr": None, - "do_set": False, - "set_voltage": 1.0, - "set_current": 0.1, - }, - } - - cfg_path: Optional[pathlib.Path] = None # Resolved configuration file path - - # 2) Environment variable can point to any YAML file - env_path = os.getenv(ENV_CONFIG_PATH) - if env_path: - candidate = pathlib.Path(env_path) - if candidate.is_file(): # Only accept existing files - cfg_path = candidate - - # 3) Fallback to default path under the provided workspace root - if cfg_path is None and workspace_root: - candidate = pathlib.Path(workspace_root) / DEFAULT_CONFIG_RELATIVE - if candidate.is_file(): - cfg_path = candidate - - # Load YAML file if we have one - if cfg_path and cfg_path.is_file(): - with open(cfg_path, "r", encoding="utf-8") as f: - file_cfg = yaml.safe_load(f) or {} # Parse YAML safely; empty → {} - if isinstance(file_cfg, dict): # Only merge dicts - _deep_update(base, file_cfg) - - # Optionally merge a dedicated PSU YAML if present (or env var path) - # This allows users to keep sensitive or machine-specific serial settings separate - psu_env = os.getenv("OWON_PSU_CONFIG") - psu_default = None - if workspace_root: - candidate = pathlib.Path(workspace_root) / "config" / "owon_psu.yaml" - if candidate.is_file(): - psu_default = candidate - psu_path: Optional[pathlib.Path] = pathlib.Path(psu_env) if psu_env else psu_default - if psu_path and psu_path.is_file(): - with open(psu_path, "r", encoding="utf-8") as f: - psu_cfg = yaml.safe_load(f) or {} - if isinstance(psu_cfg, dict): - base.setdefault("power_supply", {}) - # Merge PSU YAML into power_supply section - base["power_supply"] = _deep_update(base["power_supply"], psu_cfg) - - # 1) In-memory overrides always win - if overrides: - _deep_update(base, overrides) - - # Convert to typed dataclasses for ergonomic downstream usage - return _to_dataclass(base) diff --git a/ecu_framework/config/__init__.py b/ecu_framework/config/__init__.py new file mode 100644 index 0000000..9bca41d --- /dev/null +++ b/ecu_framework/config/__init__.py @@ -0,0 +1,30 @@ +""" +Configuration package. + +Exports: +- EcuTestConfig: Top-level typed configuration container +- InterfaceConfig: LIN interface settings (mock / MUM / deprecated BabyLIN) +- FlashConfig: Flashing settings (enabled, hex_path) +- PowerSupplyConfig: Serial PSU (Owon) settings +- load_config: Resolve YAML + env + overrides into a typed EcuTestConfig +- DEFAULT_CONFIG_RELATIVE, ENV_CONFIG_PATH: Public constants used by load_config +""" +from .loader import ( + DEFAULT_CONFIG_RELATIVE, + ENV_CONFIG_PATH, + EcuTestConfig, + FlashConfig, + InterfaceConfig, + PowerSupplyConfig, + load_config, +) + +__all__ = [ + "EcuTestConfig", + "InterfaceConfig", + "FlashConfig", + "PowerSupplyConfig", + "load_config", + "DEFAULT_CONFIG_RELATIVE", + "ENV_CONFIG_PATH", +] diff --git a/ecu_framework/config/loader.py b/ecu_framework/config/loader.py new file mode 100644 index 0000000..99a3104 --- /dev/null +++ b/ecu_framework/config/loader.py @@ -0,0 +1,452 @@ +""" +Configuration loader: YAML + environment + in-memory overrides → typed dataclasses. + +Design at a glance +================== + +The loader is a small pipeline: + + defaults (dict) + └─▶ merge YAML at $ECU_TESTS_CONFIG (if env var set & file exists) + └─▶ merge YAML at workspace_root/config/test_config.yaml (if exists) + └─▶ merge PSU side-channel YAML (env OWON_PSU_CONFIG or workspace_root/config/owon_psu.yaml) + └─▶ merge in-memory overrides (caller-supplied) + └─▶ coerce types & build EcuTestConfig dataclass + +The "merge" step is a recursive dict update (see ``_deep_update``). Nested dicts +combine key-by-key; everything else is replaced wholesale. The final ``_to_dataclass`` +step does *defensive* type coercion — YAML happily produces strings where ints are +expected, so we cast at the boundary rather than trusting the parser. + +Why this shape +============== + +- **Two layers (dict → dataclass).** The merge happens at the dict layer because + ``_deep_update`` is dict-shaped and easy to reason about. The dataclass layer is + the *public* contract callers use. Keeping these separate means the merge + semantics don't leak into consumer code. +- **PSU side-channel.** Serial port settings are bench-specific and shouldn't be + committed alongside test config. The optional ``owon_psu.yaml`` (or + ``$OWON_PSU_CONFIG``) lets users keep them out of version control while still + participating in the precedence stack. +- **In-memory overrides last.** The docstring of ``load_config`` lists overrides + as precedence #1 (highest). In the code they're applied *last* — that's exactly + what "highest precedence" means in a sequential-merge model: the last writer wins. + +Known minor wart +================ + +Defaults live in two places: as dataclass field defaults (e.g. ``type: str = "mock"``) +*and* in the ``base`` dict inside ``load_config``. Both must agree, and a drift +between them would be silently wrong. The base dict exists because the merge step +needs a starting dict; the dataclass defaults exist because callers may construct +configs directly without going through ``load_config``. If a third caller path +appears, consider extracting defaults to a single ``DEFAULTS`` mapping. +""" +from __future__ import annotations # PEP 563: makes type annotations strings, so forward references like the one in EcuTestConfig.power_supply don't require reordering definitions. + +import os # Environment variables (ECU_TESTS_CONFIG, OWON_PSU_CONFIG) and filesystem checks +import pathlib # Cross-platform path handling; preferred over os.path for new code +from dataclasses import dataclass, field # field(default_factory=...) is required for any mutable default (dict, list, nested dataclass) +from typing import Any, Dict, Optional # Any is used at the YAML boundary where we can't promise more + +import yaml # PyYAML; we only ever use safe_load — never load() — because YAML can be a code-execution vector + + +# --------------------------------------------------------------------------- +# Dataclass schema +# --------------------------------------------------------------------------- +# These three dataclasses are the public contract: anything outside this module +# that wants to know "what is configurable" reads them. Adding a field here is +# the only place you need to touch to surface a new option — _to_dataclass() +# below will need a matching coercion line. + +@dataclass +class FlashConfig: + """Flashing-related configuration. + + Attributes: + enabled: Whether to trigger ECU flashing at session start. Default off + so unit/mock runs never touch hardware. + hex_path: Path to the firmware HEX file. ``None`` means "no flashing + possible even if enabled is True" — callers must check. + """ + + enabled: bool = False + hex_path: Optional[str] = None + + +@dataclass +class InterfaceConfig: + """LIN interface configuration — covers all three adapter types in one schema. + + Fields are grouped by which adapter consumes them; fields not relevant to the + selected ``type`` are simply ignored at runtime. Keeping them in one dataclass + (rather than a per-adapter union) means YAML files don't need to change shape + when you switch between mock / MUM / BabyLIN. + + Attributes: + type: Adapter selector — ``"mock"`` (no hardware), ``"mum"`` (Melexis + Universal Master — the current hardware path), or ``"babylin"`` + (DEPRECATED — kept only so existing rigs keep working). + channel: BabyLIN channel index (0-based). Ignored by MUM and mock. + bitrate: Effective LIN bitrate in bit/s. The MUM applies it directly; + BabyLIN typically takes it from the SDF, so this field is + informational in that case. + dll_path: DEPRECATED. Pointer to vendor DLLs from the old ctypes-based + BabyLIN adapter. The SDK wrapper does not use this. + node_name: Optional friendly identifier for logs/reports. + func_names: DEPRECATED. Was a remapping table for the ctypes adapter's + function names; ignored by the SDK wrapper. + sdf_path: DEPRECATED (BabyLIN). Path to the SDF that BabyLIN loads + on connect. Required for typical BabyLIN operation. + schedule_nr: DEPRECATED (BabyLIN). Schedule index to start after + connect. ``-1`` means "do not start any schedule". + host: MUM IP address (MUM only). Required when ``type == "mum"``. + The MUM's USB-RNDIS default is ``192.168.7.2``. + lin_device: MUM LIN device name. Default ``"lin0"`` matches MUM + firmware conventions. + power_device: MUM power-control device name. Default ``"power_out0"`` + is the standard MUM power-out channel. + boot_settle_seconds: Sleep after MUM power-up before the master sends + its first frame. Tuning this avoids brown-outs on slow-booting ECUs. + frame_lengths: ``{frame_id: data_length}`` map used by the MUM to know + how many bytes to read from slave-published frames. Keys may be + written as hex strings in YAML (``0x0A``) — see _to_dataclass(). + ldf_path: Optional path to an LDF file. When set, an ``ldf`` fixture + can expose an ``LdfDatabase`` for ``pack``/``unpack``, and the MUM + adapter auto-merges frame lengths from the LDF. Relative paths + resolve against the workspace root. + """ + + # Adapter selector. + type: str = "mock" + # BabyLIN-only knobs (deprecated path) + channel: int = 1 + bitrate: int = 19200 + dll_path: Optional[str] = None + node_name: Optional[str] = None + func_names: Dict[str, str] = field(default_factory=dict) + sdf_path: Optional[str] = None + schedule_nr: int = 0 + # MUM-only knobs + host: Optional[str] = None + lin_device: str = "lin0" + power_device: str = "power_out0" + boot_settle_seconds: float = 0.5 + # MUM frame-length hints (and LDF override target) + frame_lengths: Dict[int, int] = field(default_factory=dict) + # LDF integration — shared by tests + MUM adapter + ldf_path: Optional[str] = None + + +@dataclass +class EcuTestConfig: + """Top-level typed configuration container. + + This is what ``load_config()`` returns and what most fixtures/tests + type-annotate against. New top-level config groups (e.g. a future + "reporting" section) get added here as a new ``field()``. + + Note on field ordering: + ``power_supply`` is annotated as the string ``"PowerSupplyConfig"`` + and uses a lambda default_factory because ``PowerSupplyConfig`` is + defined *below* this class. The ``from __future__ import annotations`` + import at the top of the module turns all annotations into strings, + and the lambda defers the name lookup until ``EcuTestConfig()`` is + actually instantiated — by which point ``PowerSupplyConfig`` exists + in the module namespace. This lets us keep ``EcuTestConfig`` at the + top as the "main" type readers see first. + """ + + interface: InterfaceConfig = field(default_factory=InterfaceConfig) + flash: FlashConfig = field(default_factory=FlashConfig) + # Forward reference resolved at instantiation time — see the note above. + power_supply: "PowerSupplyConfig" = field(default_factory=lambda: PowerSupplyConfig()) + + +@dataclass +class PowerSupplyConfig: + """Serial power supply (Owon) configuration. + + Defined after ``EcuTestConfig`` deliberately so the most-used type appears + at the top of the file; see the ordering note in ``EcuTestConfig``. + + Attributes: + enabled: Master switch — when False, PSU-dependent tests skip and + ``owon_psu`` helpers no-op rather than open a serial port. + port: Serial device. Windows-style (``COM4``) or POSIX-style + (``/dev/ttyUSB0``); the cross-platform resolver in + ``ecu_framework.power`` normalizes between them. + baudrate / timeout / eol: Standard line settings. ``eol`` is either + ``"\\n"`` or ``"\\r\\n"`` depending on the device firmware. + parity / stopbits: Standard serial framing knobs. + xonxoff / rtscts / dsrdtr: Flow-control flags; most Owon units want + all three off. + idn_substr: If set, the PSU helper will assert that the response to + ``*IDN?`` contains this substring before proceeding — guards + against picking up the wrong device on a multi-COM bench. + do_set / set_voltage / set_current: Convenience knobs for the demo + and smoke tests; production test cases drive the PSU directly. + """ + + enabled: bool = False + port: Optional[str] = None + baudrate: int = 115200 + timeout: float = 1.0 + eol: str = "\n" + parity: str = "N" # one of "N", "E", "O" + stopbits: float = 1.0 # 1 or 2 (float, since pyserial accepts 1.5 for some chips) + xonxoff: bool = False + rtscts: bool = False + dsrdtr: bool = False + idn_substr: Optional[str] = None + do_set: bool = False + set_voltage: float = 1.0 + set_current: float = 0.1 + + +# --------------------------------------------------------------------------- +# Public constants +# --------------------------------------------------------------------------- +# Surface as part of the public API so callers can override paths consistently +# (e.g., a custom CLI tool that wants to read the same env var as the loader). + +DEFAULT_CONFIG_RELATIVE = pathlib.Path("config") / "test_config.yaml" # Path under workspace_root that the loader looks for when no env var is set. +ENV_CONFIG_PATH = "ECU_TESTS_CONFIG" # Env var name; an absolute or relative path to a YAML file. Wins over DEFAULT_CONFIG_RELATIVE but loses to in-memory overrides. + + +# --------------------------------------------------------------------------- +# Internal merge helper +# --------------------------------------------------------------------------- + +def _deep_update(base: Dict[str, Any], updates: Dict[str, Any]) -> Dict[str, Any]: + """Recursively merge ``updates`` into ``base``. + + Semantics: + - If a key holds a dict on *both* sides, recurse so nested sections + combine key-by-key. This is what makes YAML overlays predictable: + you can override a single nested key without re-stating the whole + section. + - If a key holds a non-dict on either side, the value from ``updates`` + replaces what was in ``base`` wholesale. Lists are *replaced*, not + concatenated — that's a deliberate choice: list-concat semantics + surprise users who expect "set this list to X" to mean exactly that. + - Mutation happens in place on ``base``. The function returns the + same object for chaining convenience (used by the PSU merge below). + + Why mutate in place: + Performance is not the reason — the configs are tiny. The reason is + that the caller (``load_config``) builds ``base`` once and threads it + through several merge steps; copying at each step would obscure the + sequential precedence story. + """ + for k, v in updates.items(): + # Both sides are dicts → recurse so we don't clobber sibling keys. + if isinstance(v, dict) and isinstance(base.get(k), dict): + base[k] = _deep_update(base[k], v) + else: + # Scalar / list / mismatched types → replace. + base[k] = v + return base + + +# --------------------------------------------------------------------------- +# Dict → dataclass coercion +# --------------------------------------------------------------------------- + +def _to_dataclass(cfg: Dict[str, Any]) -> EcuTestConfig: + """Convert a merged plain-dict config into strongly-typed dataclasses. + + Why defensive casting: + YAML's type inference is generous — a value that *looks* like a number + may come through as a string (e.g. when the user quotes ``"19200"``) + and a bool may come through as the string ``"true"``. Rather than + propagate that fuzziness, we cast at this boundary so downstream code + gets the types it actually annotated against. Casts that fail raise, + which is the right behavior: a config that can't be interpreted is a + bug to surface early. + + Notes on specific fields: + - ``type`` is lowercased so YAML like ``"MUM"`` or ``"Mock"`` works. + - ``frame_lengths`` keys are parsed with ``int(k, 0)`` when the key + is a string. The ``0`` base means "infer from prefix": ``"0x0A"`` + parses as hex, ``"10"`` as decimal. Invalid keys are skipped + silently rather than failing the whole load — a typo in one frame + shouldn't abort startup. + """ + iface = cfg.get("interface", {}) + flash = cfg.get("flash", {}) + psu = cfg.get("power_supply", {}) + + # ---- frame_lengths key coercion ---- + # Goal: accept both ``0x0A: 8`` (YAML hex int) and ``"0x0A": 8`` (string-keyed + # because some YAML writers quote keys). int(k, 0) handles both; skipping bad + # entries is intentional (see docstring). + raw_fl = iface.get("frame_lengths", {}) or {} + frame_lengths: Dict[int, int] = {} + if isinstance(raw_fl, dict): + for k, v in raw_fl.items(): + try: + key = int(k, 0) if isinstance(k, str) else int(k) + frame_lengths[key] = int(v) + except (TypeError, ValueError): + # Bad entry — skip silently so one typo doesn't break startup. + continue + + return EcuTestConfig( + interface=InterfaceConfig( + type=str(iface.get("type", "mock")).lower(), + channel=int(iface.get("channel", 1)), + bitrate=int(iface.get("bitrate", 19200)), + dll_path=iface.get("dll_path"), + node_name=iface.get("node_name"), + func_names=dict(iface.get("func_names", {}) or {}), + sdf_path=iface.get("sdf_path"), + schedule_nr=int(iface.get("schedule_nr", 0)), + host=iface.get("host"), + lin_device=str(iface.get("lin_device", "lin0")), + power_device=str(iface.get("power_device", "power_out0")), + boot_settle_seconds=float(iface.get("boot_settle_seconds", 0.5)), + frame_lengths=frame_lengths, + ldf_path=iface.get("ldf_path"), + ), + flash=FlashConfig( + enabled=bool(flash.get("enabled", False)), + hex_path=flash.get("hex_path"), + ), + power_supply=PowerSupplyConfig( + enabled=bool(psu.get("enabled", False)), + port=psu.get("port"), + baudrate=int(psu.get("baudrate", 115200)), + timeout=float(psu.get("timeout", 1.0)), + eol=str(psu.get("eol", "\n")), + parity=str(psu.get("parity", "N")), + stopbits=float(psu.get("stopbits", 1.0)), + xonxoff=bool(psu.get("xonxoff", False)), + rtscts=bool(psu.get("rtscts", False)), + dsrdtr=bool(psu.get("dsrdtr", False)), + idn_substr=psu.get("idn_substr"), + do_set=bool(psu.get("do_set", False)), + set_voltage=float(psu.get("set_voltage", 1.0)), + set_current=float(psu.get("set_current", 0.1)), + ), + ) + + +# --------------------------------------------------------------------------- +# Public entry point +# --------------------------------------------------------------------------- + +def load_config(workspace_root: Optional[str] = None, overrides: Optional[Dict[str, Any]] = None) -> EcuTestConfig: + """Load configuration from defaults, YAML files, and in-memory overrides. + + Args: + workspace_root: Repository root used to resolve the default config path + (``/config/test_config.yaml``) and the optional + PSU YAML (``/config/owon_psu.yaml``). When + ``None``, those file lookups are skipped — useful for unit tests + that want to drive the loader purely from ``overrides``. + overrides: An optional dict applied last. Use this from tests that + need to flip a single value without writing a YAML file. + + Returns: + A fully-populated ``EcuTestConfig``. Never returns ``None``; missing + sources fall back to defaults rather than failing. + + Precedence (highest wins): + 1. ``overrides`` (in-memory) + 2. YAML at ``$ECU_TESTS_CONFIG`` (env var → file) + 3. YAML at ``workspace_root/config/test_config.yaml`` + 4. Built-in defaults + + In the implementation below, the steps are *applied* in the reverse + order (lowest first, highest last) because each merge replaces values + from the previous one — so the *last* writer wins, which is by design + the *highest*-precedence source. + """ + # 4) Built-in defaults — the floor everything else builds on. + # NOTE: these duplicate the dataclass field defaults. See the module docstring's + # "Known minor wart" section for why and what to do if a third caller path appears. + base: Dict[str, Any] = { + "interface": { + "type": "mock", + "channel": 1, + "bitrate": 19200, + }, + "flash": { + "enabled": False, + "hex_path": None, + }, + "power_supply": { + "enabled": False, + "port": None, + "baudrate": 115200, + "timeout": 1.0, + "eol": "\n", + "parity": "N", + "stopbits": 1.0, + "xonxoff": False, + "rtscts": False, + "dsrdtr": False, + "idn_substr": None, + "do_set": False, + "set_voltage": 1.0, + "set_current": 0.1, + }, + } + + # Resolve which YAML file (if any) to load for the main config. + cfg_path: Optional[pathlib.Path] = None + + # 3) Env var ECU_TESTS_CONFIG — wins over the workspace default. + # We only accept the path if the file actually exists; pointing at a + # missing file is treated as "no env override" rather than an error so + # CI environments can have the var set unconditionally. + env_path = os.getenv(ENV_CONFIG_PATH) + if env_path: + candidate = pathlib.Path(env_path) + if candidate.is_file(): + cfg_path = candidate + + # 2) Workspace-relative default — used when no env override is in play. + if cfg_path is None and workspace_root: + candidate = pathlib.Path(workspace_root) / DEFAULT_CONFIG_RELATIVE + if candidate.is_file(): + cfg_path = candidate + + # Apply the main YAML overlay if resolved. + if cfg_path and cfg_path.is_file(): + with open(cfg_path, "r", encoding="utf-8") as f: + file_cfg = yaml.safe_load(f) or {} # yaml.safe_load returns None for an empty file — normalize to {}. + if isinstance(file_cfg, dict): # A YAML scalar/list at the top level would parse but isn't a valid config shape; ignore it. + _deep_update(base, file_cfg) + + # ---- PSU side-channel ---- + # Why a side-channel: bench-specific serial port settings (COM4 vs + # /dev/ttyUSB0, baudrate quirks, IDN substring) should usually NOT live + # in the committed test config. Splitting them into their own file lets + # users gitignore ``config/owon_psu.yaml`` while still committing + # ``config/test_config.yaml``. The env var OWON_PSU_CONFIG mirrors the + # main config's env var pattern. + psu_env = os.getenv("OWON_PSU_CONFIG") + psu_default = None + if workspace_root: + candidate = pathlib.Path(workspace_root) / "config" / "owon_psu.yaml" + if candidate.is_file(): + psu_default = candidate + psu_path: Optional[pathlib.Path] = pathlib.Path(psu_env) if psu_env else psu_default + if psu_path and psu_path.is_file(): + with open(psu_path, "r", encoding="utf-8") as f: + psu_cfg = yaml.safe_load(f) or {} + if isinstance(psu_cfg, dict): + # Ensure the section exists before deep-merging into it. + base.setdefault("power_supply", {}) + base["power_supply"] = _deep_update(base["power_supply"], psu_cfg) + + # 1) In-memory overrides — applied LAST so they win over all file sources. + if overrides: + _deep_update(base, overrides) + + # Final step: cast the merged dict into typed dataclasses for callers. + return _to_dataclass(base)