539 lines
20 KiB
Markdown
539 lines
20 KiB
Markdown
# Docker Image for the ECU Test Framework
|
|
|
|
This guide covers packaging the framework into a Docker image so it
|
|
can run as a reproducible unit on developer laptops, CI runners, and
|
|
host machines that talk to a real bench.
|
|
|
|
There are **two distinct images** to keep separate in your head:
|
|
|
|
| Image | Purpose | Hardware? | Where it runs |
|
|
|---|---|---|---|
|
|
| **`ecu-tests:mock`** | Unit tests, mock-LIN smoke tests, plugin self-tests, doc/coverage generation | None | Any developer laptop, CI runner |
|
|
| **`ecu-tests:hw`** | Real-bench tests against a MUM and/or an Owon PSU | Yes (USB serial, network reachable MUM) | Lab machine attached to the bench |
|
|
|
|
The two share the same Dockerfile and a build-arg switch — the
|
|
hardware variant adds device-passthrough config and the Melexis
|
|
packages.
|
|
|
|
---
|
|
|
|
## 1. Why dockerize?
|
|
|
|
| Pain | What the image fixes |
|
|
|---|---|
|
|
| "Works on my machine" — different pyserial, ldfparser, pytest versions | Pinned `requirements.txt`, frozen base image, deterministic build |
|
|
| Onboarding a new developer takes a day | `docker run …` and you're testing |
|
|
| CI flake from a system Python upgrade | Image is the unit, CI doesn't care about the runner's Python |
|
|
| Auditors / security ask "what software runs on the bench?" | A single OCI artifact with a known digest |
|
|
|
|
What dockerization **does not** fix:
|
|
|
|
- It does not get you the Melexis `pylin` / `pymumclient` /
|
|
`pylinframe` packages. Those are **not on PyPI**; they ship inside
|
|
the Melexis IDE installer. You have to provide them at build time
|
|
(see §5).
|
|
- It does not magically pass USB devices through. Hardware tests
|
|
need explicit `--device` flags (see §4).
|
|
- It does not paper over OS-level requirements (host network mode
|
|
on Linux, USB/IP on Windows/WSL, etc.).
|
|
|
|
---
|
|
|
|
## 2. Architecture
|
|
|
|
```
|
|
┌──────────────────────┐
|
|
│ Owon PSU │
|
|
┌─── --device /dev/ttyUSB0 ─┤ /dev/ttyUSB0 etc. │
|
|
│ └──────────────────────┘
|
|
┌───────────────┴───────────────┐
|
|
│ ecu-tests:hw container │
|
|
│ │
|
|
│ /workspace │ ┌──────────────────────┐
|
|
│ ├── ecu_framework/ │ │ MUM │
|
|
│ ├── tests/ │ │ 192.168.7.2 (RNDIS) │
|
|
│ ├── config/ ◄───┤ │
|
|
│ └── vendor/melexis/ │ └──────────────────────┘
|
|
│ ├── pylin/ │ (--network host)
|
|
│ ├── pymumclient/ │
|
|
│ └── pylinframe/ │
|
|
│ │
|
|
│ /reports ◄─── -v $PWD/reports:/reports
|
|
└───────────────────────────────┘
|
|
```
|
|
|
|
Key choices:
|
|
|
|
- **`/workspace`** is the repo. Either baked into the image (default
|
|
for CI) or bind-mounted from the host (for iteration).
|
|
- **`/reports`** is a volume so report HTML/XML lands on the host
|
|
filesystem and survives the container.
|
|
- **The Melexis packages** live under `vendor/melexis/` inside the
|
|
image (or bind-mounted; see §5). The framework imports them via
|
|
`pylin` and `pymumclient` because the `vendor/automated_lin_test/
|
|
install_packages.sh` script copies them into `site-packages` of
|
|
the venv during image build.
|
|
- **MUM access**: the MUM appears as a network device at
|
|
`192.168.7.2`. On Linux you use `--network host` so the container
|
|
shares the host's USB-RNDIS interface; on Windows/macOS Desktop
|
|
the picture is more nuanced (§4.3).
|
|
- **PSU access**: the Owon is a USB-serial device. Pass it through
|
|
with `--device /dev/ttyUSB0:/dev/ttyUSB0` and inside the container
|
|
configure `config.power_supply.port: /dev/ttyUSB0`.
|
|
|
|
---
|
|
|
|
## 3. Dockerfile
|
|
|
|
A multi-stage Dockerfile keeps the runtime image lean. The `builder`
|
|
stage compiles wheels (and runs `pip install` against a writable
|
|
filesystem); the `runtime` stage only contains what's needed to
|
|
execute tests.
|
|
|
|
Save as `docker/Dockerfile`:
|
|
|
|
```dockerfile
|
|
# syntax=docker/dockerfile:1.6
|
|
#
|
|
# ecu-tests image — mock-only by default, hardware variant via
|
|
# --build-arg INCLUDE_MELEXIS=1
|
|
#
|
|
# Build:
|
|
# docker build -f docker/Dockerfile -t ecu-tests:mock .
|
|
# docker build -f docker/Dockerfile -t ecu-tests:hw \
|
|
# --build-arg INCLUDE_MELEXIS=1 \
|
|
# --secret id=melexis_tarball,src=./melexis-pkgs.tar.gz \
|
|
# .
|
|
#
|
|
# The hardware build needs the Melexis Python packages bundled into
|
|
# a tarball (pylin/, pymumclient/, pylinframe/ — three directories).
|
|
# See docs/20_docker_image.md §5.
|
|
|
|
ARG PYTHON_VERSION=3.11
|
|
|
|
# ──────────────────────────────────────────────────────────────────────
|
|
# Stage 1: builder — pip-install deps into a venv under /opt/venv
|
|
# ──────────────────────────────────────────────────────────────────────
|
|
FROM python:${PYTHON_VERSION}-slim AS builder
|
|
|
|
ARG INCLUDE_MELEXIS=0
|
|
|
|
# OS deps:
|
|
# build-essential, libffi-dev — for any wheel that needs a compiler
|
|
# libusb-1.0-0 — pyserial uses it at runtime; keep both
|
|
# builder and runtime parity
|
|
# git — only if requirements.txt references VCS deps
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends \
|
|
build-essential \
|
|
libffi-dev \
|
|
libusb-1.0-0 \
|
|
git \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
ENV PYTHONDONTWRITEBYTECODE=1 \
|
|
PIP_NO_CACHE_DIR=1 \
|
|
PIP_DISABLE_PIP_VERSION_CHECK=1
|
|
|
|
RUN python -m venv /opt/venv
|
|
ENV PATH="/opt/venv/bin:${PATH}"
|
|
|
|
WORKDIR /build
|
|
COPY requirements.txt ./
|
|
RUN pip install --upgrade pip wheel \
|
|
&& pip install -r requirements.txt
|
|
|
|
# Melexis packages — bundled in via Docker BuildKit secret so the
|
|
# proprietary tarball never ends up in an image layer.
|
|
RUN --mount=type=secret,id=melexis_tarball,required=false \
|
|
if [ "$INCLUDE_MELEXIS" = "1" ]; then \
|
|
set -e; \
|
|
test -s /run/secrets/melexis_tarball \
|
|
|| { echo 'INCLUDE_MELEXIS=1 but no melexis_tarball secret bound'; exit 2; }; \
|
|
SITE_PACKAGES=$(python -c "import site; print(site.getsitepackages()[0])"); \
|
|
tar -xzf /run/secrets/melexis_tarball -C "$SITE_PACKAGES"; \
|
|
python -c "import pylin, pymumclient; print('melexis pkgs OK')"; \
|
|
fi
|
|
|
|
# ──────────────────────────────────────────────────────────────────────
|
|
# Stage 2: runtime — slim image with just the venv + repo
|
|
# ──────────────────────────────────────────────────────────────────────
|
|
FROM python:${PYTHON_VERSION}-slim AS runtime
|
|
|
|
# Runtime-only OS deps. pyserial needs libusb at runtime for some
|
|
# USB-serial chips; ldfparser is pure Python.
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends \
|
|
libusb-1.0-0 \
|
|
ca-certificates \
|
|
tini \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# Pull the prebuilt venv (with Melexis pkgs if requested) from builder.
|
|
COPY --from=builder /opt/venv /opt/venv
|
|
|
|
ENV PYTHONDONTWRITEBYTECODE=1 \
|
|
PYTHONUNBUFFERED=1 \
|
|
PATH="/opt/venv/bin:${PATH}"
|
|
|
|
# Repo. .dockerignore should exclude .venv, reports, vendor/BabyLIN*,
|
|
# __pycache__, .pytest_cache.
|
|
WORKDIR /workspace
|
|
COPY . /workspace
|
|
|
|
# Reports live on a mounted volume so they survive the container.
|
|
RUN mkdir -p /reports
|
|
VOLUME ["/reports"]
|
|
|
|
# Drop privileges. Inherit any host-side serial group via runtime
|
|
# `--group-add` (USB-serial devices on Linux are typically owned by
|
|
# the dialout group).
|
|
RUN useradd -m -u 1000 -s /bin/bash tester
|
|
USER tester
|
|
|
|
# tini handles signal forwarding so Ctrl-C cleanly tears down pytest.
|
|
ENTRYPOINT ["/usr/bin/tini", "--"]
|
|
|
|
# Default: collect-only so an accidental `docker run` doesn't fire
|
|
# hardware tests on a misconfigured bench.
|
|
CMD ["pytest", "-m", "not hardware", "--collect-only", "-q"]
|
|
```
|
|
|
|
A matching `.dockerignore` (place at repo root):
|
|
|
|
```
|
|
.git
|
|
.venv
|
|
__pycache__
|
|
.pytest_cache
|
|
.coverage*
|
|
reports/*
|
|
!reports/.gitkeep
|
|
htmlcov
|
|
*.egg-info
|
|
vendor/BabyLIN library
|
|
vendor/BabyLIN_library.py
|
|
docs/_build
|
|
```
|
|
|
|
---
|
|
|
|
## 4. Building & running
|
|
|
|
> **Don't have Docker yet?** Install steps for WSL (both Docker
|
|
> Desktop and Docker-Engine-in-WSL paths, plus `usbipd-win` for USB
|
|
> passthrough) live in
|
|
> [`docker/README.md`](../docker/README.md#prerequisites--install-docker-on-wsl).
|
|
|
|
### 4.1 Mock-only image (the CI image)
|
|
|
|
```bash
|
|
# Build
|
|
docker build -f docker/Dockerfile -t ecu-tests:mock .
|
|
|
|
# Run the mock suite, write reports to ./reports
|
|
docker run --rm \
|
|
-v "$PWD/reports:/reports" \
|
|
-e ECU_TESTS_CONFIG=config/test_config.yaml \
|
|
ecu-tests:mock \
|
|
pytest -m "not hardware" -v --junitxml=/reports/junit.xml \
|
|
--html=/reports/report.html --self-contained-html
|
|
```
|
|
|
|
Works on any Linux/macOS/Windows host that runs Docker. No hardware
|
|
involved. Suitable for GitHub Actions, GitLab CI, Jenkins, etc.
|
|
|
|
### 4.2 Hardware image — local Linux bench
|
|
|
|
```bash
|
|
# Bundle the Melexis packages (one-time, on a machine that has Melexis IDE)
|
|
tar -czf melexis-pkgs.tar.gz \
|
|
-C "/path/to/Melexis/site-packages" \
|
|
pylin pymumclient pylinframe
|
|
|
|
# Build hardware image
|
|
DOCKER_BUILDKIT=1 docker build \
|
|
-f docker/Dockerfile \
|
|
-t ecu-tests:hw \
|
|
--build-arg INCLUDE_MELEXIS=1 \
|
|
--secret id=melexis_tarball,src=./melexis-pkgs.tar.gz \
|
|
.
|
|
|
|
# Run hardware tests
|
|
docker run --rm \
|
|
--network host \
|
|
--device /dev/ttyUSB0:/dev/ttyUSB0 \
|
|
--group-add dialout \
|
|
-v "$PWD/reports:/reports" \
|
|
-v "$PWD/config/test_config.yaml:/workspace/config/test_config.yaml:ro" \
|
|
-e ECU_TESTS_CONFIG=/workspace/config/test_config.yaml \
|
|
ecu-tests:hw \
|
|
pytest -m "hardware and mum" -v \
|
|
--junitxml=/reports/junit.xml \
|
|
--html=/reports/report.html --self-contained-html
|
|
```
|
|
|
|
The flags:
|
|
|
|
| Flag | Why |
|
|
|---|---|
|
|
| `--network host` | The MUM is reachable at `192.168.7.2` via USB-RNDIS on the host. Bridged networking would hide that interface. |
|
|
| `--device /dev/ttyUSB0:/dev/ttyUSB0` | Owon PSU passthrough. Adjust to whatever `ls /dev/ttyUSB*` reports on the host. |
|
|
| `--group-add dialout` | Without it the container's `tester` user can't open the serial port. |
|
|
| `-v config/test_config.yaml:…:ro` | Lets you tweak bench config without rebuilding. |
|
|
|
|
### 4.3 Hardware image — Windows / WSL2 / macOS
|
|
|
|
| Host | What works | Caveat |
|
|
|---|---|---|
|
|
| Windows Docker Desktop | Use `usbipd-win` to forward the USB-serial adapter into the WSL2 backend, then `--device /dev/ttyUSB0`. MUM access via `--network host` works because Docker Desktop bridges WSL2 to the host network. | The COM-port name in the host shell is irrelevant; the container sees a Linux device file. |
|
|
| WSL2 (no Docker Desktop) | Same usbipd-win flow. | The WSL2 distro must be the active integration target for Docker. |
|
|
| macOS Docker Desktop | **USB passthrough is not supported.** | Workaround: run a thin TCP-to-serial bridge on the host (e.g. `socat`) and have the container connect to that. Documented but fiddly. |
|
|
|
|
For Windows-native (no WSL), Docker Desktop's "Windows container"
|
|
mode can pass through COM ports but isn't tested with this framework.
|
|
|
|
### 4.4 Interactive / iteration mode
|
|
|
|
When you're developing tests, bind-mount the repo so edits show up
|
|
without rebuilding:
|
|
|
|
```bash
|
|
docker run --rm -it \
|
|
--network host \
|
|
--device /dev/ttyUSB0:/dev/ttyUSB0 \
|
|
--group-add dialout \
|
|
-v "$PWD:/workspace" \
|
|
-v "$PWD/reports:/reports" \
|
|
ecu-tests:hw \
|
|
bash
|
|
```
|
|
|
|
Inside the container:
|
|
|
|
```bash
|
|
pytest tests/hardware/test_mum_alm_animation.py -v
|
|
```
|
|
|
|
---
|
|
|
|
## 5. The Melexis-package obstacle
|
|
|
|
`pylin`, `pymumclient`, and `pylinframe` ship inside the **Melexis
|
|
IDE** installation, not on PyPI:
|
|
|
|
```
|
|
C:\Program Files\Melexis\Melexis IDE\plugins\com.melexis.mlxide.python_<ver>\python\Lib\site-packages\
|
|
├── pylin/
|
|
├── pymumclient/
|
|
└── pylinframe/
|
|
```
|
|
|
|
`vendor/automated_lin_test/install_packages.sh` copies them into a
|
|
host venv. For Docker, the equivalent is a tarball passed as a build
|
|
secret:
|
|
|
|
```bash
|
|
# Once per machine that has Melexis IDE installed, or once on a
|
|
# build server that has a snapshot. Adjust the path to your install.
|
|
MELEXIS_SITE="/mnt/c/Program Files/Melexis/Melexis IDE/plugins/com.melexis.mlxide.python_1.2.0.202408130945/python/Lib/site-packages"
|
|
|
|
tar -czf melexis-pkgs.tar.gz \
|
|
-C "$MELEXIS_SITE" \
|
|
pylin pymumclient pylinframe
|
|
```
|
|
|
|
Pass to `docker build` via BuildKit secret as shown above. The
|
|
secret content is **not** baked into any image layer; it's mounted
|
|
only for the `RUN` statement that consumes it.
|
|
|
|
### License hygiene
|
|
|
|
- Don't push `ecu-tests:hw` to a public registry — the layer that
|
|
copied the Melexis files into `site-packages` carries proprietary
|
|
code.
|
|
- Use a private registry (internal Harbor, GitHub Container Registry
|
|
with a private repo, AWS ECR, …) gated by the same access controls
|
|
as the Melexis IDE itself.
|
|
- For a public mock-only image, build with `--build-arg
|
|
INCLUDE_MELEXIS=0` (the default) and the proprietary bits never
|
|
enter the image.
|
|
|
|
---
|
|
|
|
## 6. docker-compose example
|
|
|
|
`docker/compose.hw.yml`:
|
|
|
|
```yaml
|
|
services:
|
|
ecu-tests:
|
|
image: ecu-tests:hw
|
|
build:
|
|
context: ..
|
|
dockerfile: docker/Dockerfile
|
|
args:
|
|
INCLUDE_MELEXIS: "1"
|
|
secrets:
|
|
- melexis_tarball
|
|
network_mode: host # MUM reachable at 192.168.7.2
|
|
devices:
|
|
- "/dev/ttyUSB0:/dev/ttyUSB0"
|
|
group_add:
|
|
- dialout
|
|
volumes:
|
|
- ../reports:/reports
|
|
- ../config/test_config.yaml:/workspace/config/test_config.yaml:ro
|
|
environment:
|
|
ECU_TESTS_CONFIG: /workspace/config/test_config.yaml
|
|
command: >
|
|
pytest -m "hardware and mum" -v
|
|
--junitxml=/reports/junit.xml
|
|
--html=/reports/report.html --self-contained-html
|
|
|
|
secrets:
|
|
melexis_tarball:
|
|
file: ../melexis-pkgs.tar.gz
|
|
```
|
|
|
|
Build & run:
|
|
|
|
```bash
|
|
docker compose -f docker/compose.hw.yml build
|
|
docker compose -f docker/compose.hw.yml up --abort-on-container-exit
|
|
```
|
|
|
|
---
|
|
|
|
## 7. CI/CD integration
|
|
|
|
### GitHub Actions — mock-only
|
|
|
|
```yaml
|
|
# .github/workflows/test-mock.yml
|
|
name: tests (mock)
|
|
on: [push, pull_request]
|
|
jobs:
|
|
mock:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: docker/setup-buildx-action@v3
|
|
- name: Build mock image
|
|
run: docker build -f docker/Dockerfile -t ecu-tests:mock .
|
|
- name: Run mock suite
|
|
run: |
|
|
mkdir -p reports
|
|
docker run --rm -v "$PWD/reports:/reports" ecu-tests:mock \
|
|
pytest -m "not hardware" -v \
|
|
--junitxml=/reports/junit.xml \
|
|
--html=/reports/report.html --self-contained-html
|
|
- uses: actions/upload-artifact@v4
|
|
if: always()
|
|
with:
|
|
name: reports
|
|
path: reports/
|
|
```
|
|
|
|
### Self-hosted runner for hardware
|
|
|
|
A self-hosted runner on the lab machine, labelled `bench`, runs the
|
|
hardware job. The runner has `melexis-pkgs.tar.gz` cached locally
|
|
and the USB-serial port at a known path:
|
|
|
|
```yaml
|
|
# .github/workflows/test-hw.yml
|
|
name: tests (hardware)
|
|
on:
|
|
workflow_dispatch:
|
|
schedule:
|
|
- cron: '0 4 * * *' # nightly at 04:00 lab time
|
|
jobs:
|
|
hardware:
|
|
runs-on: [self-hosted, bench]
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- name: Build hardware image
|
|
run: |
|
|
DOCKER_BUILDKIT=1 docker build \
|
|
-f docker/Dockerfile -t ecu-tests:hw \
|
|
--build-arg INCLUDE_MELEXIS=1 \
|
|
--secret id=melexis_tarball,src=/var/lib/bench/melexis-pkgs.tar.gz \
|
|
.
|
|
- name: Run hardware suite
|
|
run: |
|
|
mkdir -p reports
|
|
docker run --rm \
|
|
--network host \
|
|
--device /dev/ttyUSB0:/dev/ttyUSB0 \
|
|
--group-add dialout \
|
|
-v "$PWD/reports:/reports" \
|
|
ecu-tests:hw \
|
|
pytest -m "hardware and not slow" -v \
|
|
--junitxml=/reports/junit.xml \
|
|
--html=/reports/report.html --self-contained-html
|
|
- uses: actions/upload-artifact@v4
|
|
if: always()
|
|
with:
|
|
name: hw-reports
|
|
path: reports/
|
|
```
|
|
|
|
---
|
|
|
|
## 8. Troubleshooting
|
|
|
|
| Symptom | Likely cause | Fix |
|
|
|---|---|---|
|
|
| `ModuleNotFoundError: No module named 'pylin'` | Image built without `INCLUDE_MELEXIS=1`, or the tarball was empty | Verify with `docker run --rm ecu-tests:hw python -c "import pylin; print(pylin.__file__)"` |
|
|
| `serial.SerialException: could not open port` | USB device not passed through, or wrong path | `--device /dev/ttyUSB0:/dev/ttyUSB0` on the host; check with `ls /dev/ttyUSB*` on the host first |
|
|
| `Permission denied: '/dev/ttyUSB0'` | Container user not in the host's serial-group | Add `--group-add dialout` (or whatever group owns the device on the host) |
|
|
| MUM unreachable at 192.168.7.2 | Container on bridge network instead of host network | Add `--network host`. On Docker Desktop, Windows/macOS, see §4.3 |
|
|
| Reports empty / not on host | `/reports` not bind-mounted | `-v "$PWD/reports:/reports"` |
|
|
| Build fails on Apple Silicon | Multi-arch wheels missing for some dep | Add `--platform linux/amd64` to `docker build` and use Rosetta emulation, or rebuild from source |
|
|
| Tests run as root accidentally | Custom `USER` override at runtime | Don't pass `--user 0`; the image runs as `tester` (uid 1000) on purpose |
|
|
| pytest-html missing CSS in report | Forgot `--self-contained-html` | Add it to the pytest command line so the HTML stands alone |
|
|
|
|
---
|
|
|
|
## 9. Limitations and intentional non-goals
|
|
|
|
- **No GUI** — `docker run` doesn't render LED color, smoothness of
|
|
fade, or any other optical property. Hardware tests still assert
|
|
only what's on the LIN bus, just like running the framework
|
|
natively.
|
|
- **No firmware flashing yet** — the `HexFlasher` is a scaffold;
|
|
baking a working UDS flasher into the image is future work. When
|
|
it lands, the flashing path will need access to the same serial
|
|
device and the same network as the tests.
|
|
- **No live monitoring** — the image runs a pytest invocation and
|
|
exits. If you want a long-lived "test agent" container, wrap
|
|
pytest in a daemon (e.g. a small Flask app that triggers runs on
|
|
webhook); not provided here.
|
|
- **No multi-bench orchestration** — one container, one bench. For
|
|
N benches, run N containers with distinct `--device` /
|
|
`--network` configs, ideally orchestrated by Compose or
|
|
Kubernetes.
|
|
- **Deprecated BabyLIN path** — the image deliberately does **not**
|
|
package the BabyLIN SDK. If you genuinely need it on a legacy rig,
|
|
see `docs/08_babylin_internals.md` and add the SDK directly to
|
|
the host venv; don't try to dockerize the deprecated path.
|
|
|
|
---
|
|
|
|
## 10. Related docs
|
|
|
|
- [`docs/02_configuration_resolution.md`](02_configuration_resolution.md)
|
|
— how `ECU_TESTS_CONFIG` and `OWON_PSU_CONFIG` envs feed the test
|
|
fixtures (used by the container).
|
|
- [`docs/12_using_the_framework.md`](12_using_the_framework.md) —
|
|
the non-container reference flow.
|
|
- [`docs/14_power_supply.md`](14_power_supply.md) — PSU port
|
|
resolution (cross-platform). The container sees Linux device
|
|
paths.
|
|
- [`docs/21_yocto_image_for_raspberry_pi.md`](21_yocto_image_for_raspberry_pi.md)
|
|
— if you'd rather have the framework run *on* an embedded board
|
|
rather than from a container on a host PC.
|
|
- `vendor/automated_lin_test/install_packages.sh` — the native-venv
|
|
equivalent of the Docker Melexis-bundle step.
|