Hosam-Eldin.mostafa/ecu-tests
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
ecu-tests/docs/20_docker_image.md

20 KiB
Raw Blame History

Docker Image for the ECU Test Framework

This guide covers packaging the framework into a Docker image so it can run as a reproducible unit on developer laptops, CI runners, and host machines that talk to a real bench.

There are two distinct images to keep separate in your head:

Image Purpose Hardware? Where it runs
ecu-tests:mock Unit tests, mock-LIN smoke tests, plugin self-tests, doc/coverage generation None Any developer laptop, CI runner
ecu-tests:hw Real-bench tests against a MUM and/or an Owon PSU Yes (USB serial, network reachable MUM) Lab machine attached to the bench

The two share the same Dockerfile and a build-arg switch — the hardware variant adds device-passthrough config and the Melexis packages.

1. Why dockerize?

Pain What the image fixes
"Works on my machine" — different pyserial, ldfparser, pytest versions Pinned requirements.txt, frozen base image, deterministic build
Onboarding a new developer takes a day docker run … and you're testing
CI flake from a system Python upgrade Image is the unit, CI doesn't care about the runner's Python
Auditors / security ask "what software runs on the bench?" A single OCI artifact with a known digest

What dockerization does not fix:

  • It does not get you the Melexis pylin / pymumclient / pylinframe packages. Those are not on PyPI; they ship inside the Melexis IDE installer. You have to provide them at build time (see §5).
  • It does not magically pass USB devices through. Hardware tests need explicit --device flags (see §4).
  • It does not paper over OS-level requirements (host network mode on Linux, USB/IP on Windows/WSL, etc.).

2. Architecture

                                           ┌──────────────────────┐
                                           │  Owon PSU            │
                ┌─── --device /dev/ttyUSB0 ─┤  /dev/ttyUSB0 etc.   │
                │                          └──────────────────────┘
┌───────────────┴───────────────┐
│ ecu-tests:hw container        │
│                               │
│  /workspace                   │   ┌──────────────────────┐
│   ├── ecu_framework/          │   │  MUM                 │
│   ├── tests/                  │   │  192.168.7.2 (RNDIS) │
│   ├── config/                 ◄───┤                      │
│   └── vendor/melexis/         │   └──────────────────────┘
│        ├── pylin/             │       (--network host)
│        ├── pymumclient/       │
│        └── pylinframe/        │
│                               │
│  /reports  ◄─── -v $PWD/reports:/reports
└───────────────────────────────┘

Key choices:

  • /workspace is the repo. Either baked into the image (default for CI) or bind-mounted from the host (for iteration).
  • /reports is a volume so report HTML/XML lands on the host filesystem and survives the container.
  • The Melexis packages live under vendor/melexis/ inside the image (or bind-mounted; see §5). The framework imports them via pylin and pymumclient because the vendor/automated_lin_test/ install_packages.sh script copies them into site-packages of the venv during image build.
  • MUM access: the MUM appears as a network device at 192.168.7.2. On Linux you use --network host so the container shares the host's USB-RNDIS interface; on Windows/macOS Desktop the picture is more nuanced (§4.3).
  • PSU access: the Owon is a USB-serial device. Pass it through with --device /dev/ttyUSB0:/dev/ttyUSB0 and inside the container configure config.power_supply.port: /dev/ttyUSB0.

3. Dockerfile

A multi-stage Dockerfile keeps the runtime image lean. The builder stage compiles wheels (and runs pip install against a writable filesystem); the runtime stage only contains what's needed to execute tests.

Save as docker/Dockerfile:

# syntax=docker/dockerfile:1.6
#
# ecu-tests image — mock-only by default, hardware variant via
#   --build-arg INCLUDE_MELEXIS=1
#
# Build:
#   docker build -f docker/Dockerfile -t ecu-tests:mock .
#   docker build -f docker/Dockerfile -t ecu-tests:hw \
#       --build-arg INCLUDE_MELEXIS=1 \
#       --secret id=melexis_tarball,src=./melexis-pkgs.tar.gz \
#       .
#
# The hardware build needs the Melexis Python packages bundled into
# a tarball (pylin/, pymumclient/, pylinframe/ — three directories).
# See docs/20_docker_image.md §5.

ARG PYTHON_VERSION=3.11

# ──────────────────────────────────────────────────────────────────────
# Stage 1: builder — pip-install deps into a venv under /opt/venv
# ──────────────────────────────────────────────────────────────────────
FROM python:${PYTHON_VERSION}-slim AS builder

ARG INCLUDE_MELEXIS=0

# OS deps:
#   build-essential, libffi-dev — for any wheel that needs a compiler
#   libusb-1.0-0                — pyserial uses it at runtime; keep both
#                                  builder and runtime parity
#   git                         — only if requirements.txt references VCS deps
RUN apt-get update \
 && apt-get install -y --no-install-recommends \
        build-essential \
        libffi-dev \
        libusb-1.0-0 \
        git \
 && rm -rf /var/lib/apt/lists/*

ENV PYTHONDONTWRITEBYTECODE=1 \
    PIP_NO_CACHE_DIR=1 \
    PIP_DISABLE_PIP_VERSION_CHECK=1

RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:${PATH}"

WORKDIR /build
COPY requirements.txt ./
RUN pip install --upgrade pip wheel \
 && pip install -r requirements.txt

# Melexis packages — bundled in via Docker BuildKit secret so the
# proprietary tarball never ends up in an image layer.
RUN --mount=type=secret,id=melexis_tarball,required=false \
    if [ "$INCLUDE_MELEXIS" = "1" ]; then \
        set -e; \
        test -s /run/secrets/melexis_tarball \
          || { echo 'INCLUDE_MELEXIS=1 but no melexis_tarball secret bound'; exit 2; }; \
        SITE_PACKAGES=$(python -c "import site; print(site.getsitepackages()[0])"); \
        tar -xzf /run/secrets/melexis_tarball -C "$SITE_PACKAGES"; \
        python -c "import pylin, pymumclient; print('melexis pkgs OK')"; \
    fi

# ──────────────────────────────────────────────────────────────────────
# Stage 2: runtime — slim image with just the venv + repo
# ──────────────────────────────────────────────────────────────────────
FROM python:${PYTHON_VERSION}-slim AS runtime

# Runtime-only OS deps. pyserial needs libusb at runtime for some
# USB-serial chips; ldfparser is pure Python.
RUN apt-get update \
 && apt-get install -y --no-install-recommends \
        libusb-1.0-0 \
        ca-certificates \
        tini \
 && rm -rf /var/lib/apt/lists/*

# Pull the prebuilt venv (with Melexis pkgs if requested) from builder.
COPY --from=builder /opt/venv /opt/venv

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    PATH="/opt/venv/bin:${PATH}"

# Repo. .dockerignore should exclude .venv, reports, vendor/BabyLIN*,
# __pycache__, .pytest_cache.
WORKDIR /workspace
COPY . /workspace

# Reports live on a mounted volume so they survive the container.
RUN mkdir -p /reports
VOLUME ["/reports"]

# Drop privileges. Inherit any host-side serial group via runtime
# `--group-add` (USB-serial devices on Linux are typically owned by
# the dialout group).
RUN useradd -m -u 1000 -s /bin/bash tester
USER tester

# tini handles signal forwarding so Ctrl-C cleanly tears down pytest.
ENTRYPOINT ["/usr/bin/tini", "--"]

# Default: collect-only so an accidental `docker run` doesn't fire
# hardware tests on a misconfigured bench.
CMD ["pytest", "-m", "not hardware", "--collect-only", "-q"]

A matching .dockerignore (place at repo root):

.git
.venv
__pycache__
.pytest_cache
.coverage*
reports/*
!reports/.gitkeep
htmlcov
*.egg-info
vendor/BabyLIN library
vendor/BabyLIN_library.py
docs/_build

4. Building & running

Don't have Docker yet? Install steps for WSL (both Docker Desktop and Docker-Engine-in-WSL paths, plus usbipd-win for USB passthrough) live in docker/README.md.

4.1 Mock-only image (the CI image)

# Build
docker build -f docker/Dockerfile -t ecu-tests:mock .

# Run the mock suite, write reports to ./reports
docker run --rm \
  -v "$PWD/reports:/reports" \
  -e ECU_TESTS_CONFIG=config/test_config.yaml \
  ecu-tests:mock \
  pytest -m "not hardware" -v --junitxml=/reports/junit.xml \
                           --html=/reports/report.html --self-contained-html

Works on any Linux/macOS/Windows host that runs Docker. No hardware involved. Suitable for GitHub Actions, GitLab CI, Jenkins, etc.

4.2 Hardware image — local Linux bench

# Bundle the Melexis packages (one-time, on a machine that has Melexis IDE)
tar -czf melexis-pkgs.tar.gz \
    -C "/path/to/Melexis/site-packages" \
    pylin pymumclient pylinframe

# Build hardware image
DOCKER_BUILDKIT=1 docker build \
    -f docker/Dockerfile \
    -t ecu-tests:hw \
    --build-arg INCLUDE_MELEXIS=1 \
    --secret id=melexis_tarball,src=./melexis-pkgs.tar.gz \
    .

# Run hardware tests
docker run --rm \
  --network host \
  --device /dev/ttyUSB0:/dev/ttyUSB0 \
  --group-add dialout \
  -v "$PWD/reports:/reports" \
  -v "$PWD/config/test_config.yaml:/workspace/config/test_config.yaml:ro" \
  -e ECU_TESTS_CONFIG=/workspace/config/test_config.yaml \
  ecu-tests:hw \
  pytest -m "hardware and mum" -v \
         --junitxml=/reports/junit.xml \
         --html=/reports/report.html --self-contained-html

The flags:

Flag Why
--network host The MUM is reachable at 192.168.7.2 via USB-RNDIS on the host. Bridged networking would hide that interface.
--device /dev/ttyUSB0:/dev/ttyUSB0 Owon PSU passthrough. Adjust to whatever ls /dev/ttyUSB* reports on the host.
--group-add dialout Without it the container's tester user can't open the serial port.
-v config/test_config.yaml:…:ro Lets you tweak bench config without rebuilding.

4.3 Hardware image — Windows / WSL2 / macOS

Host What works Caveat
Windows Docker Desktop Use usbipd-win to forward the USB-serial adapter into the WSL2 backend, then --device /dev/ttyUSB0. MUM access via --network host works because Docker Desktop bridges WSL2 to the host network. The COM-port name in the host shell is irrelevant; the container sees a Linux device file.
WSL2 (no Docker Desktop) Same usbipd-win flow. The WSL2 distro must be the active integration target for Docker.
macOS Docker Desktop USB passthrough is not supported. Workaround: run a thin TCP-to-serial bridge on the host (e.g. socat) and have the container connect to that. Documented but fiddly.

For Windows-native (no WSL), Docker Desktop's "Windows container" mode can pass through COM ports but isn't tested with this framework.

4.4 Interactive / iteration mode

When you're developing tests, bind-mount the repo so edits show up without rebuilding:

docker run --rm -it \
  --network host \
  --device /dev/ttyUSB0:/dev/ttyUSB0 \
  --group-add dialout \
  -v "$PWD:/workspace" \
  -v "$PWD/reports:/reports" \
  ecu-tests:hw \
  bash

Inside the container:

pytest tests/hardware/test_mum_alm_animation.py -v

5. The Melexis-package obstacle

pylin, pymumclient, and pylinframe ship inside the Melexis IDE installation, not on PyPI:

C:\Program Files\Melexis\Melexis IDE\plugins\com.melexis.mlxide.python_<ver>\python\Lib\site-packages\
    ├── pylin/
    ├── pymumclient/
    └── pylinframe/

vendor/automated_lin_test/install_packages.sh copies them into a host venv. For Docker, the equivalent is a tarball passed as a build secret:

# Once per machine that has Melexis IDE installed, or once on a
# build server that has a snapshot. Adjust the path to your install.
MELEXIS_SITE="/mnt/c/Program Files/Melexis/Melexis IDE/plugins/com.melexis.mlxide.python_1.2.0.202408130945/python/Lib/site-packages"

tar -czf melexis-pkgs.tar.gz \
    -C "$MELEXIS_SITE" \
    pylin pymumclient pylinframe

Pass to docker build via BuildKit secret as shown above. The secret content is not baked into any image layer; it's mounted only for the RUN statement that consumes it.

License hygiene

  • Don't push ecu-tests:hw to a public registry — the layer that copied the Melexis files into site-packages carries proprietary code.
  • Use a private registry (internal Harbor, GitHub Container Registry with a private repo, AWS ECR, …) gated by the same access controls as the Melexis IDE itself.
  • For a public mock-only image, build with --build-arg INCLUDE_MELEXIS=0 (the default) and the proprietary bits never enter the image.

6. docker-compose example

docker/compose.hw.yml:

services:
  ecu-tests:
    image: ecu-tests:hw
    build:
      context: ..
      dockerfile: docker/Dockerfile
      args:
        INCLUDE_MELEXIS: "1"
      secrets:
        - melexis_tarball
    network_mode: host          # MUM reachable at 192.168.7.2
    devices:
      - "/dev/ttyUSB0:/dev/ttyUSB0"
    group_add:
      - dialout
    volumes:
      - ../reports:/reports
      - ../config/test_config.yaml:/workspace/config/test_config.yaml:ro
    environment:
      ECU_TESTS_CONFIG: /workspace/config/test_config.yaml
    command: >
      pytest -m "hardware and mum" -v
             --junitxml=/reports/junit.xml
             --html=/reports/report.html --self-contained-html

secrets:
  melexis_tarball:
    file: ../melexis-pkgs.tar.gz

Build & run:

docker compose -f docker/compose.hw.yml build
docker compose -f docker/compose.hw.yml up --abort-on-container-exit

7. CI/CD integration

GitHub Actions — mock-only

# .github/workflows/test-mock.yml
name: tests (mock)
on: [push, pull_request]
jobs:
  mock:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - name: Build mock image
        run: docker build -f docker/Dockerfile -t ecu-tests:mock .
      - name: Run mock suite
        run: |
          mkdir -p reports
          docker run --rm -v "$PWD/reports:/reports" ecu-tests:mock \
            pytest -m "not hardware" -v \
                   --junitxml=/reports/junit.xml \
                   --html=/reports/report.html --self-contained-html
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: reports
          path: reports/

Self-hosted runner for hardware

A self-hosted runner on the lab machine, labelled bench, runs the hardware job. The runner has melexis-pkgs.tar.gz cached locally and the USB-serial port at a known path:

# .github/workflows/test-hw.yml
name: tests (hardware)
on:
  workflow_dispatch:
  schedule:
    - cron: '0 4 * * *'    # nightly at 04:00 lab time
jobs:
  hardware:
    runs-on: [self-hosted, bench]
    steps:
      - uses: actions/checkout@v4
      - name: Build hardware image
        run: |
          DOCKER_BUILDKIT=1 docker build \
            -f docker/Dockerfile -t ecu-tests:hw \
            --build-arg INCLUDE_MELEXIS=1 \
            --secret id=melexis_tarball,src=/var/lib/bench/melexis-pkgs.tar.gz \
            .
      - name: Run hardware suite
        run: |
          mkdir -p reports
          docker run --rm \
            --network host \
            --device /dev/ttyUSB0:/dev/ttyUSB0 \
            --group-add dialout \
            -v "$PWD/reports:/reports" \
            ecu-tests:hw \
            pytest -m "hardware and not slow" -v \
                   --junitxml=/reports/junit.xml \
                   --html=/reports/report.html --self-contained-html
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: hw-reports
          path: reports/

8. Troubleshooting

Symptom Likely cause Fix
ModuleNotFoundError: No module named 'pylin' Image built without INCLUDE_MELEXIS=1, or the tarball was empty Verify with docker run --rm ecu-tests:hw python -c "import pylin; print(pylin.__file__)"
serial.SerialException: could not open port USB device not passed through, or wrong path --device /dev/ttyUSB0:/dev/ttyUSB0 on the host; check with ls /dev/ttyUSB* on the host first
Permission denied: '/dev/ttyUSB0' Container user not in the host's serial-group Add --group-add dialout (or whatever group owns the device on the host)
MUM unreachable at 192.168.7.2 Container on bridge network instead of host network Add --network host. On Docker Desktop, Windows/macOS, see §4.3
Reports empty / not on host /reports not bind-mounted -v "$PWD/reports:/reports"
Build fails on Apple Silicon Multi-arch wheels missing for some dep Add --platform linux/amd64 to docker build and use Rosetta emulation, or rebuild from source
Tests run as root accidentally Custom USER override at runtime Don't pass --user 0; the image runs as tester (uid 1000) on purpose
pytest-html missing CSS in report Forgot --self-contained-html Add it to the pytest command line so the HTML stands alone

9. Limitations and intentional non-goals

  • No GUI — docker run doesn't render LED color, smoothness of fade, or any other optical property. Hardware tests still assert only what's on the LIN bus, just like running the framework natively.
  • No firmware flashing yet — the HexFlasher is a scaffold; baking a working UDS flasher into the image is future work. When it lands, the flashing path will need access to the same serial device and the same network as the tests.
  • No live monitoring — the image runs a pytest invocation and exits. If you want a long-lived "test agent" container, wrap pytest in a daemon (e.g. a small Flask app that triggers runs on webhook); not provided here.
  • No multi-bench orchestration — one container, one bench. For N benches, run N containers with distinct --device / --network configs, ideally orchestrated by Compose or Kubernetes.
  • Deprecated BabyLIN path — the image deliberately does not package the BabyLIN SDK. If you genuinely need it on a legacy rig, see docs/08_babylin_internals.md and add the SDK directly to the host venv; don't try to dockerize the deprecated path.

10. Related docs