Getting Started with Maestro

For new users, evaluators, and teams onboarding to Maestro.

What is Maestro?

Maestro is a test orchestration platform for hardware and software test stations. It coordinates the execution of structured test sequences, collects measurements and results, and presents everything through a real-time web interface — without requiring operators to touch code.

Test logic is written once by engineers (in Python or .NET), described in version-controlled YAML, and then deployed as a test package that any authorized station can run. Maestro handles scheduling, execution, measurement evaluation, and reporting.

What Maestro does

• Executes structured test sequences defined in YAML

• Coordinates multiple test runners (.NET and Python) in parallel or in sequence

• Evaluates measurements against pass/fail limits

• Stores results, logs, and measurement data in a central database

• Provides a real-time operator interface for monitoring and interaction

• Exposes a REST API for CI/CD and automation integration

• Supports multi-station fleet monitoring via Orchestra

What Maestro does not do

• Maestro is not a test framework like pytest, NUnit, or Robot Framework — it does not own test logic

• Maestro does not replace your instrument drivers or hardware libraries

• Maestro does not manage source control — test packages live in your own Git repositories

• Maestro does not run on the instrument itself — it runs alongside it on a test station host

How Maestro compares to alternatives

Maestro Concepts & Terminology

Understanding a handful of core concepts makes the rest of Maestro immediately clear.

Station

A station is a single Maestro deployment — typically one per physical test bench or virtual machine. Each station runs its own Maestro server process and has its own configuration, results database, and connected runners.

Test Package

A test package is the deployable unit of work. It contains:

• A YAML test definition describing the sequence of steps

• One or more runner implementations (Python modules or .NET assemblies) that provide the step logic

• A package.json manifest declaring the package name, version, and dependencies

Packages are versioned and can be promoted between environments (development → staging → production).

Test Definition (YAML)

The YAML file is the authoritative description of what runs. It defines steps, their order, variables, measurements, limits, and control flow (retries, preconditions, timeouts). Engineers write and version-control this file; operators never edit it directly.

Station Configuration

A station config is a named set of settings that customizes a test package for a specific station or environment. Configurations are stored in the Maestro database and have two tiers:

• Global — shared defaults across all stations

• Station-local — overrides specific to one station (e.g., COM port assignments, calibration offsets)

Test Execution (Run)

A run is a single execution of a test definition against a specific configuration. Each run produces a timestamped record containing step results, measurements, logs, and a final verdict.

Verdict

Every step and every run resolves to one of three verdicts:

Measurement

A measurement is a numeric value captured during a step and evaluated against configured limits. Measurements are stored individually, enabling trend analysis and per-limit failure reporting.

Runner

A runner is a process that executes step implementations. Runners communicate with the Maestro server via gRPC. Two runner types are supported:

• .NET runner — for C# step libraries; supports both .NET 10 (64-bit) and .NET Framework 4.8 (32-bit, for legacy instrument compatibility)

• Python runner — for Python step libraries; runs in a managed virtual environment

Orchestra

Orchestra is the multi-station monitoring service. It aggregates heartbeat and status data from all registered stations into a central dashboard, giving fleet operators a single view across a site.

MCP Server

The MCP (Model Context Protocol) server exposes Maestro's execution, results, and documentation to AI assistants (e.g., Claude, GitHub Copilot). It enables AI-assisted test authoring, result analysis, and unattended operator prompt handling.

Quick Start: First Test Run

Prerequisites

Before starting, ensure the following are available on the host machine:

• Docker Desktop (for the server stack)

• A modern browser (Chrome, Edge, or Firefox)

For package development, you will additionally need:

• .NET 10 SDK (for .NET runner packages)

• Python 3.11 or later (for Python runner packages)

Step 1 — Start the Maestro server stack

Maestro ships as a Docker Compose stack. In the directory containing your docker-compose.yml:

docker compose up -d

This starts the following services:

• PostgreSQL — results, configuration, and package database

• Redis — variable bus and real-time pub/sub

• Maestro API — REST endpoints and YAML compiler

• Maestro UI — Blazor web interface

Step 2 — Open the web interface

Navigate to <http://localhost:5000> (or the configured host address) in your browser. You will see the Maestro operator interface.

Step 3 — Load a test package

1. Go to Packages in the left navigation.

2. Click Upload and select your test package .zip, or point Maestro at your package registry.

3. The package appears in the package list once validation passes.

Step 4 — Select a configuration

1. Go to Configurations and select or create a station configuration for your package.

2. Verify that station-specific values (port assignments, device addresses) are correct for your environment.

Step 5 — Start a run

1. Go to Test Monitor.

2. Select your package and configuration.

3. Click Run.

4. Watch step progress, measurements, and verdicts update in real time.

Step 6 — Review results

When the run completes, navigate to Test Results to view the full execution record: step-level verdicts, measurement values vs. limits, and the captured log.

Typical Maestro Workflows

Manual testing (operator-driven)

An operator opens the web UI, selects a package and configuration, starts the run, responds to any operator prompts (e.g., "Connect DUT and press Continue"), and reviews the verdict. No command line or coding required.

Automated production runs

A CI/CD pipeline or MES (Manufacturing Execution System) triggers a run via the Maestro REST API, passing the package name, configuration, and optional variable overrides. Results are polled or streamed via SignalR. The final verdict and measurement data are written back to the calling system.

# Example: trigger a run via REST API
POST /api/executions
{
  "package": "board-functional-test",
  "version": "2.1.0",
  "configuration": "station-04-prod"
}

CI/CD package validation

During development, engineers push a test package to the Maestro package registry as part of their CI pipeline. Maestro validates the YAML schema and package structure and rejects malformed packages before they reach a production station.

AI-assisted authoring and analysis

With the MCP server enabled, an AI assistant can query Maestro documentation, inspect live test results, analyze measurement trends, and propose YAML changes — all from within the assistant's chat interface.

User Roles and Permissions

Maestro defines three built-in roles:

Role assignments are managed in the Admin section of the web interface. A user may hold multiple roles.

Note: Granular permission gating is an active roadmap item. Current builds enforce role separation at the UI level; API-level enforcement is being rolled out progressively.

System Requirements & Prerequisites

Server (Maestro stack)

Runner hosts

Client (web UI)

Any modern browser with JavaScript enabled. No installation required. Tested on:

• Google Chrome 120+

• Microsoft Edge 120+

• Mozilla Firefox 120+

Supported Platforms & Versions

Maestro follows a rolling release model. Server and runner packages should be kept at matching versions. Mismatched versions may cause gRPC contract errors at startup.