Releases & Updates

Current state, breaking changes, migration notes, and what is coming next. Check GET /api/health/version or the MCP get_system_version tool to see the exact version running on your station.

How Maestro Is Released

Maestro is distributed as Docker container images through a private GitLab registry. There are no installer packages or downloadable binaries to manage. Updating is:

docker compose pull
docker compose up -d

Images are tagged with both latest and the commit SHA of the build. Database migrations apply automatically on API startup via EF Core. Stations should be updated after the central server — never before.

To check the running version:

# REST API
curl http://localhost:7000/api/health/version

# MCP tool
get_system_version()

Current Release

What Is Included

The current release includes all of the following as shipped features:

Core execution

• YAML-defined test sequences with full step lifecycle (precondition, retry, repeat, timeout, run_on_abort, post_execution_action)

• .NET 10 runner (64-bit, amd64/arm64) and .NET Framework 4.8 runner (32-bit Windows)

• Python 3.11/3.12 runner with automatic virtual environment and pip dependency management

• Cross-runner variable sharing via Redis — a .NET step and a Python step in the same sequence share variables transparently

• Mock runner and Delay runner for development and stabilisation steps

• Operator prompt steps with four input modes: number, text, boolean, list

• Unattended mode for automated and CI execution

Station configuration

• Two-tier (global + station-local) PostgreSQL-backed key-value store

• Config snapshot stored on every execution record for post-mortem analysis

• cfg.* variable injection into every step automatically

Results and traceability

• Relational measurement storage (not log strings) — queryable with SQL for SPC, yield, and trend analysis

• Git commit hash stored on every execution record

• Step-level and execution-level verdicts with full measurement expand in the UI

• Inline image thumbnails and interactive Plotly chart rendering in reports

• Permanent report URLs

Package management

• Git-backed package registry with lifecycle states (NotReleased, Evaluation, Released, Obsolete)

• Per-station lifecycle override without touching the source registry

• Two-level YAML validation (structural without stack, semantic with stack)

• VS Code authoring integration with live schema feedback

Fleet monitoring (Orchestra)

• Multi-station fleet overview dashboard — station cards with current test, recent verdicts, yield percentage

• Station detail pages

• REST API for CI/CD station discovery and execution summary

MCP server (AI integration)

• 41 tools across system, config, package, execution, results, and YAML categories

• HTTP transport (port 7004) — compatible with Claude Desktop, GitHub Copilot, Cursor, and any MCP-compatible client

• Multi-station support with per-response station label

• Unattended mode via start_test(unattendedMode: true)

• Service log access without SSH (get_service_logs, get_system_events)

Infrastructure

• PostgreSQL 15/16 with EF Core automatic migrations

• Redis 7.x variable bus

• SignalR real-time event streaming to UI and external consumers

• Docker Compose deployment for Linux amd64, Linux arm64 (including Raspberry Pi 64-bit)

• Multi-arch image manifests — no separate images per platform

Recent Changes

Python runner: module loaded once per execution, not per step

Previous behaviour (documented incorrectly): The Python runner reloaded the test module on every step call.

Actual behaviour (now documented correctly): The Python runner loads each module once on the first step call within an execution and caches it for the lifetime of that execution. Module-level state — instrument handles, open connections, cached values — is preserved across all steps within a single execution. The module is reloaded fresh for the next execution.

Impact: Code that relied on module-level state being reset between steps within one execution will behave differently than expected. Code that opens an instrument connection at module level (not inside a function) will find that connection still open in subsequent steps — which is often the intended behaviour.

AccordionQ2 Python client API coverage

Added documentation and anti-pattern guidance for the Python AccordionQ2Client:

• Full .NET → Python API mapping for all client methods

• Channel direction configuration must precede any channel read or write

• GetValueAsync always returns string — explicit numeric parsing required

• All AccordionQ2Client methods are async — .Wait() or .Result required in synchronous step functions

Measurement operator clarification

Comparison operators (equal, notequal, gt, gte, lt, lte) apply to type: numeric measurements only. For type: boolean and type: string, the only evaluation mechanism is the expected: field. The operator: log modifier is supported on all types to make a measurement informational-only (always PASS, value recorded).

Cross-runner variable documentation

Clarified that output variables exist in a single execution-scoped store regardless of which runner produced them. A variable written by a runner: dotnet step is immediately available to a subsequent runner: python step with no special handling.

Breaking Changes

These changes require action when upgrading from an older deployment.

Step type syntax: type: runner removed

Old syntax (no longer valid):

- name: "Measure voltage"
  type: runner
  runner_type: net10.0
  assembly: "Tests.dll"

Current syntax:

- name: "Measure voltage"
  runner: dotnet
  runner_type: net10.0
  assembly: "Tests.dll"

Migration: Replace type: runner with runner: dotnet or runner: python. The type: field is now used only for built-in step types (delay, mock, prompt, sequence). Runner steps omit type: and use runner: instead.

The YAML validator reports Unknown step type 'runner' on files that still use the old syntax.

Variable template syntax: {{test.variable}} removed

Old syntax (no longer resolved):

value: "{{test.measured_voltage}}"

Current syntax:

value: "{{measured_voltage}}"

Migration: Remove the test. prefix from all variable references in YAML files. The old Scriban dot-prefix syntax is no longer resolved — files using it will produce literal {{test.variable}} strings in measurement values and parameters, which is a silent failure rather than an error. Search all YAML files: grep -r "{{test\\." tests/.

Deprecated Functionality

type: mock in non-development packages

Mock steps exist for development and local testing only. They should not appear in packages promoted to Evaluation or Released. Mock steps do not call any runner — they return the static target: value from YAML and always pass.

The YAML validator does not currently block mock steps in released packages, but a future release will introduce a warning when a package containing mock steps is promoted. Audit your packages now: grep -r "type: mock" tests/.

Implicit steps: at the top level

Early Maestro package formats allowed steps: as a top-level key without a wrapping test: block. This layout is no longer supported. All step arrays must be nested under test::

# ❌ Old layout — no longer supported
steps:
  - name: "Measure voltage"
    ...

# ✅ Current layout
test:
  name: "Board Functional Test"
  version: "1.0.0"
  steps:
    - name: "Measure voltage"
      ...

The YAML validator reports a structural error on files using the old layout.

Known Issues in the Current Release

Authentication not enforced at the API level

The REST API and MCP server do not enforce authentication. Any client that can reach port 7000 or 7004 can call any endpoint. This is a deliberate current-state decision, not an oversight — the IPermissionService interface is fully wired and a custom implementation can be deployed.

Mitigation: Restrict access at the network layer (VLAN, firewall, VPN) until API-level authentication is available in your deployment. Do not expose Maestro ports to untrusted networks.

cfg.* and exec.* cannot be used directly in precondition: expressions

DynamicExpresso interprets the dot in cfg.DMM_VISA as C# member access, not as a namespace separator. Preconditions that reference cfg.* or exec.* directly will throw at runtime (aborting the step) or evaluate incorrectly.

Workaround: Copy the config value into a bare variable in a preceding setup step. See Troubleshooting — Precondition always evaluates to false.

Dynamic measurement limits stored as YAML text, not resolved values

When a measurement uses a variable for its limit (e.g. low_limit: "{{computed_lower}}"), the stored low_limit column in the database reflects the YAML template text, not the runtime-resolved value. The verdict column is always computed correctly. Post-hoc SQL queries that recalculate actual_value against the stored limit columns will produce incorrect results for steps with dynamic limits.

Impact: Affects only measurements with variable-substituted limits. Static limits (numeric literals) are not affected.

Configuration changes require an API restart to take effect

Changes to Station Config (via the UI or API) do not take effect in-flight. The running test uses the config snapshot taken at execution start. Changes apply to the next execution. Changes to appsettings.json or .env require an API container restart.

A Restart Services button is on the roadmap (see below) to make this explicit in the UI.

No built-in result CSV export

The Test Results page does not currently have a one-click CSV export. Results can be exported via the REST API (GET /api/testresults with filters) or queried directly from PostgreSQL. A UI-level CSV export is a P1 roadmap item.

GitHub Actions / Orchestra end-to-end integration requires manual setup

The Orchestra fleet API and station execution API both exist, but there is no reusable GitHub Actions composite action, no POST /api/orchestra/dispatch endpoint for atomic station selection, and the APIs are currently unauthenticated. CI/CD teams integrating with Orchestra must implement the discover → dispatch → poll cycle manually using the existing REST API. This is documented in Integrations & Interfaces — CI/CD Integration.

Upcoming Features

Items listed by priority. The roadmap reflects current planning intent and is subject to change.

P1 — High value, next up

P2 — Planned, lower urgency

Consciously deferred

These items appeared in earlier planning and were explicitly decided against:

Migration Guides

Migrating from type: runner to runner: syntax

1. Search all YAML files in all packages: grep -rn "type: runner" tests/

2. For each match, replace:

   YAML

   type: runner
   runner_type: net10.0
   

   with:

   YAML

   runner: dotnet
   runner_type: net10.0
   

   Or for Python:

   YAML

   runner: python
   runner_type: python3.11
   

3. Run py validate.py — all files must pass before committing

4. Bump the version in package.json

5. Commit, push, refresh registry, and re-activate on each station

Migrating from {{test.variable}} to {{variable}} syntax

1. Search all YAML files: grep -rn "{{test\\." tests/

2. Remove the test. prefix from every match: {{test.voltage}} → {{voltage}}

3. Check that each affected variable is declared in the variables: block (it must be, since the old syntax was the same variable store)

4. Run py validate.py

5. Bump version, commit, deploy

Upgrading a multi-station deployment

Always upgrade in this order:

1. Back up PostgreSQL before any upgrade: docker exec workflowengine-postgres pg_dump -U postgres testautomation | gzip > backup-$(date +%Y%m%d).sql.gz

2. Stop all stations — prevents running executions from writing to a schema that is about to change

3. Upgrade the central server: docker compose pull && docker compose up -d on the central host

4. Wait for migrations — watch docker compose logs api until you see Database is up to date

5. Upgrade stations one at a time: docker compose pull && docker compose up -d

6. Verify each station after upgrade: curl <http://localhost:7000/api/health> should return {"status":"Healthy"}

Do not skip step 2. A station running an older schema version against a migrated database may fail silently or write malformed records.