--- language: "en" --- # Using Maestro Day-to-day task reference for operators, engineers, and anyone running, reviewing, or automating tests in Maestro. *** ** * ** *** ## Projects \& Configuration In Maestro, what other tools call a "project" is a **test package** --- a versioned bundle of YAML test definitions and the runner code that executes them. Configuration is managed separately through **Station Config**, a two-tier key-value store that keeps hardware-specific values out of YAML files. ### Creating a Package (Project) 1. Open the Maestro UI and navigate to **Packages** (`/packages`) 2. Click **+ New Package** 3. Click **Create** . Maestro scaffolds the package directory with a `package.json` manifest, YAML test file stubs, and a validation script. If you provided a GitLab URL, the package is immediately committed and pushed to that repository. It will appear in the package browser once the registry catalog is updated. ### Editing Package Settings Package metadata (name, version, description, lifecycle status) is defined in `package.json` at the root of the package repository. Edit the file directly in your editor and push the change. To update the station's copy: 1. Go to **Packages** → find the package 2. Click **↻ Refresh Registry** to pull catalog changes 3. Click **↓ Download** to fetch the updated version To change a package's lifecycle status without editing the source repository --- for example to mark a version obsolete on one station only --- use the lifecycle override: PATCH /api/packages/{name}/lifecycle Content-Type: application/json { "status": "Obsolete" } Valid values: `NotReleased`, `Evaluation`, `Released`, `Obsolete`. ### Managing Station Configuration Station Config holds hardware addresses, port assignments, fixture IDs, and any other values that differ between stations. YAML files reference these via `{{cfg.KEY_NAME}}` so the same test definition runs unchanged on every station. **View and edit configuration in the UI:** 1. Navigate to ` 2. Configuration is shown in two sections: * **Global** --- defaults shared by all stations * **Station-local** --- overrides specific to this station **Add or update a value via the API:** PowerShell # Add a station-local instrument address $body = @{ stationId = "ST-01" key = "DMM_VISA" value = "TCPIP::192.168.1.100::INSTR" category = "Instruments" description = "Digital multimeter address" updatedBy = "admin" } | ConvertTo-Json Invoke-RestMethod -Uri "http://localhost:7000/api/config" ` -Method PUT -Body $body -ContentType "application/json" **View the merged config for a station** (what a running test actually sees): PowerShell Invoke-RestMethod -Uri "http://localhost:7000/api/config/merged/ST-01" > Configuration changes require an API restart before they take effect at runtime. ### Importing and Exporting Configurations Maestro does not have a dedicated import/export UI button, but every config value is accessible through the REST API, making it straightforward to script a full export and import. **Export all config for a station:** PowerShell $config = Invoke-RestMethod -Uri "http://localhost:7000/api/config/merged/ST-01" $config | ConvertTo-Json -Depth 5 | Out-File "station-config-ST-01.json" **Import config from a file:** PowerShell $entries = Get-Content "station-config-ST-01.json" | ConvertFrom-Json foreach ($entry in $entries) { $body = $entry | ConvertTo-Json Invoke-RestMethod -Uri "http://localhost:7000/api/config" ` -Method PUT -Body $body -ContentType "application/json" } This approach is also suitable for provisioning a new station from a known-good configuration baseline, or for promoting configuration from a development station to production. *** ** * ** *** ## Running Tests ### Running a Test Manually 1. Navigate to **Test Monitor** (`/test-monitor`) 2. The **Test File Path** field validates in real time: * Green border = file exists on the server, ready to run * Red border = file not found --- check the path and re-enter 3. Click **Start Test**. The button is only enabled when the SignalR connection is active (green dot in the top bar) and the file path is valid. **While the test runs**, the page shows three live panels: | Panel | Contents | |---------------|--------------------------------------------------------------------------------------------| | **Steps** | Every step in execution order with status icons, elapsed time, and live measurement values | | **Variables** | All runtime variables updated as each step completes | | **Logs** | Structured log output from runner code --- useful for debugging | Expand any step row to see individual measurement points: actual value, lower limit, upper limit, and pass/fail for each. ### Operator Prompts Some test steps pause and display a prompt asking the operator to perform a physical action, read an instrument, or confirm an observation. When a prompt appears: * Read the message and complete the described action * Enter a value if an input field is shown (the field type --- numeric, text, boolean, or list --- is determined by the test definition) * Click the appropriate response button to continue Entered values are recorded as measurements and are available to later steps. A numeric prompt validates the entered value against its configured limits before continuing. ### Scheduling Test Runs Maestro does not include a built-in scheduler. Scheduled or recurring runs are triggered externally using the REST API: **Trigger a run via PowerShell (Windows Task Scheduler)** PowerShell $body = @{ yamlFilePath = "/data/tat-packages/board-functional-test/tests/main.yaml" serialNumber = "NIGHTLY-$(Get-Date -Format 'yyyyMMdd')" operatorId = "scheduler" stationId = "ST-01" } | ConvertTo-Json Invoke-RestMethod -Uri "http://localhost:7000/api/testexecution/run" ` -Method POST -Body $body -ContentType "application/json" **Trigger a run from a bash cron job (Linux)** Bash #!/bin/bash curl -s -X POST http://localhost:7000/api/testexecution/run \ -H "Content-Type: application/json" \ -d "{ \"yamlFilePath\": \"/data/tat-packages/board-test/tests/main.yaml\", \"serialNumber\": \"NIGHTLY-$(date +%Y%m%d)\", \"operatorId\": \"scheduler\", \"stationId\": \"ST-01\" }" Use `GET /api/packages/test-files` to discover the exact server-side YAML path for a given package before building your scheduled trigger. ### Running Tests Headless / Unattended For automated and CI use cases where no operator is present, pass `unattendedMode: true` in the execution request. This prevents prompt steps from blocking indefinitely. PowerShell $body = @{ yamlFilePath = "/data/tat-packages/board-test/tests/main.yaml" serialNumber = "CI-001" operatorId = "ci-pipeline" unattendedMode = $true } | ConvertTo-Json Invoke-RestMethod -Uri "http://localhost:7000/api/testexecution/run" ` -Method POST -Body $body -ContentType "application/json" **How unattended mode handles prompt steps:** | Prompt type | Behaviour | |---------------------------------------------|--------------------------------------------------------------------------------| | Button-only prompt | Auto-selects the first button with action `Continue`, then `Pass`, then `Fail` | | Value-input prompt with `input.default` set | Uses the declared default value automatically | | Value-input prompt with no default | Fails the step with an explanatory message; execution continues | A warning log entry is written for every auto-responded prompt: [UNATTENDED] Auto-responding to prompt 'Inspect solder joints' with button 'OK' (action: Continue) **Making a test unattended-safe:** * Every button-only prompt should have at least one button with `action: Continue` or `action: Pass` * Every value-input prompt should declare `input.default` * Validate with the YAML Validator page before deploying to an automated pipeline * Run the test interactively at least once before enabling unattended mode in production > Unattended mode is not appropriate for tests involving visual inspection or physical adjustments that require human judgment. Auto-clicking through those steps may allow defective units to pass. ### Stopping or Cancelling a Run **From the UI:** Click **Abort** in the execution status banner on the Test Monitor page. The current step is allowed to complete before execution halts. All results up to that point are saved. **From the API:** PowerShell # Abort a specific execution Invoke-RestMethod -Uri "http://localhost:7000/api/testexecution/{executionId}/abort" ` -Method POST The execution record is saved with verdict `UNDETERMINED` and all completed step results are preserved. *** ** * ** *** ## Results \& Analysis ### Viewing Test Results Navigate to **Test Results** (`/test-results`). All historical executions are searchable. **Search filters:** | Filter | Description | |--------------------|----------------------------------------------------------| | **Serial Number** | Device under test --- supports partial match | | **Test Name** | Name from the YAML definition --- supports partial match | | **Verdict** | All, PASS, FAIL, or UNDETERMINED | | **From / To Date** | Narrow to a time window | | **Operator ID** | Filter by who ran the test | | **Station ID** | Filter by which station | Leave all fields blank and click **Search** to return the most recent executions. **Opening a detailed report:** Click **View** on any row. The report shows: * Header: test name, serial number, operator, station, start/end time, duration, overall verdict * **Step Results** table: every step with status, duration, verdict, and expandable measurements * **Inline Images**: thumbnails of any PNG/JPEG/SVG artifacts emitted by steps (click to view full size) * **Interactive Charts**: Plotly figures rendered with full zoom, pan, and hover --- useful for waveform and spectrum data * **Nested Executions**: sub-sequences shown as collapsible sections Each report has a permanent URL (`/test-results/{id}`) that can be bookmarked or shared. ### Understanding Pass / Fail Logic Verdicts are determined at three levels: **Measurement level** --- a single numeric data point: * `PASS` if `low_limit ≤ value ≤ high_limit` * `FAIL` if the value falls outside the limits * `UNDETERMINED` if the measurement could not be taken **Step level** --- the worst verdict across all measurements in that step: * If any measurement fails → step is `FAIL` * If all measurements pass → step is `PASS` * If no measurements were taken → `UNDETERMINED` **Execution level** --- the worst verdict across all steps: * If any step fails → execution is `FAIL` * If all steps pass → execution is `PASS` * If any step is undetermined and none fail → `UNDETERMINED` **Step control behaviour** --- how the test proceeds after a failure is defined in YAML per step: | `post_execution_action` | Behaviour on FAIL | |-------------------------|---------------------------------------| | `continue` (default) | Move to the next step regardless | | `terminate-on-fail` | Stop the entire execution immediately | Skipped steps (excluded by a `precondition` that evaluated to false) do not affect the execution verdict. ### Exporting Results **Print or save as PDF:** Open any report and use your browser's **Print** function (`Ctrl+P`). The report layout hides navigation chrome when printing and is formatted for A4/Letter. Interactive charts are hidden during print --- use the Plotly toolbar's PNG export button to include a chart snapshot before printing. **Export via API:** PowerShell # Get the full execution report as JSON Invoke-RestMethod -Uri "http://localhost:7000/api/testexecution/{executionId}/report" # Get all measurements for an execution Invoke-RestMethod -Uri "http://localhost:7000/api/testexecution/{executionId}/measurements" # Get execution logs Invoke-RestMethod -Uri "http://localhost:7000/api/testexecution/{executionId}/logs" **Bulk export with filtering:** PowerShell # All FAIL results in a date range $params = "verdict=FAIL&from=2026-01-01T00:00:00Z&to=2026-01-31T23:59:59Z" Invoke-RestMethod -Uri "http://localhost:7000/api/testresults?$params" **Database queries** (for bulk analysis and SPC): Because measurements are stored as structured relational rows --- not log strings --- they are directly queryable with SQL: SQL -- Yield rate for a specific test over the last 30 days SELECT COUNT(*) FILTER (WHERE verdict = 'PASS') AS pass_count, COUNT(*) AS total_count, ROUND(100.0 * COUNT(*) FILTER (WHERE verdict = 'PASS') / COUNT(*), 1) AS yield_pct FROM test_executions WHERE test_name = 'board-functional-test' AND started_at >= NOW() - INTERVAL '30 days'; -- Measurement drift for a specific limit over time SELECT started_at, actual_value, low_limit, high_limit FROM measurements WHERE measurement_name = 'VOUT_3V3' AND station_id = 'ST-01' ORDER BY started_at DESC LIMIT 500; ### Comparing Runs The Maestro UI shows one execution at a time. For side-by-side comparison, use the API or database directly. **Compare two specific executions:** PowerShell $run1 = Invoke-RestMethod -Uri "http://localhost:7000/api/testexecution/101/report" $run2 = Invoke-RestMethod -Uri "http://localhost:7000/api/testexecution/102/report" # Compare step verdicts $run1.steps | Select-Object name, verdict | Format-Table $run2.steps | Select-Object name, verdict | Format-Table **Compare measurement values for the same serial number across runs:** SQL SELECT e.started_at, e.station_id, m.measurement_name, m.actual_value, m.low_limit, m.high_limit, m.verdict FROM measurements m JOIN test_executions e ON m.execution_id = e.id WHERE e.serial_number = 'UNIT-042' AND m.measurement_name = 'VOUT_3V3' ORDER BY e.started_at DESC; *** ** * ** *** ## Automation ### Creating Automated Workflows The Maestro REST API exposes the full execution model. Any system that can make an HTTP request can trigger, monitor, and retrieve test results. **Minimal trigger-and-poll workflow:** PowerShell # 1. Start the test $body = @{ yamlFilePath = "/data/tat-packages/board-test/tests/main.yaml" serialNumber = "UNIT-042" operatorId = "automation" unattendedMode = $true } | ConvertTo-Json $result = Invoke-RestMethod ` -Uri "http://localhost:7000/api/testexecution/run" ` -Method POST -Body $body -ContentType "application/json" $executionId = $result.executionId # 2. Poll until terminal state do { Start-Sleep -Seconds 2 $status = Invoke-RestMethod ` -Uri "http://localhost:7000/api/testexecution/$executionId/status" } while ($status.state -notin @("Completed", "Failed", "Aborted")) # 3. Retrieve the report $report = Invoke-RestMethod ` -Uri "http://localhost:7000/api/testexecution/$executionId/report" Write-Host "Verdict: $($report.verdict)" exit $(if ($report.verdict -eq 'PASS') { 0 } else { 1 }) **Real-time monitoring with SignalR** (for low-latency pipelines): For production automation where polling latency is unacceptable, connect to the SignalR hub and subscribe to execution events. This is the same channel the Maestro UI uses and delivers step updates, measurement values, and completion events as they happen. JavaScript // JavaScript example --- e.g., in a Node.js CI bridge const { HubConnectionBuilder } = require("@microsoft/signalr"); const connection = new HubConnectionBuilder() .withUrl("http://localhost:7000/hubs/testexecution") .build(); connection.on("StepCompleted", (step) => { console.log(`[${step.verdict}] ${step.name}`); }); connection.on("ExecutionCompleted", (execution) => { console.log(`Final verdict: ${execution.verdict}`); process.exit(execution.verdict === "PASS" ? 0 : 1); }); await connection.start(); ### Triggering Maestro from External Systems **From a GitLab CI pipeline:** YAML # .gitlab-ci.yml hardware-test: stage: test script: - | EXECUTION=$(curl -sf -X POST http://$MAESTRO_HOST:7000/api/testexecution/run \ -H "Content-Type: application/json" \ -d "{ \"yamlFilePath\": \"$TEST_FILE_PATH\", \"serialNumber\": \"CI-$CI_PIPELINE_ID\", \"operatorId\": \"gitlab-ci\", \"unattendedMode\": true }") EXECUTION_ID=$(echo $EXECUTION | jq -r .executionId) - | while true; do STATUS=$(curl -sf http://$MAESTRO_HOST:7000/api/testexecution/$EXECUTION_ID/status) STATE=$(echo $STATUS | jq -r .state) [ "$STATE" = "Completed" ] || [ "$STATE" = "Failed" ] || [ "$STATE" = "Aborted" ] && break sleep 3 done - | VERDICT=$(curl -sf http://$MAESTRO_HOST:7000/api/testexecution/$EXECUTION_ID/report | jq -r .verdict) [ "$VERDICT" = "PASS" ] || exit 1 variables: MAESTRO_HOST: "maestro-station-01.internal" TEST_FILE_PATH: "/data/tat-packages/board-test/tests/main.yaml" **From a GitHub Actions workflow:** YAML - name: Run Maestro test run: | RESULT=$(curl -sf -X POST http://$MAESTRO_HOST:7000/api/testexecution/run \ -H "Content-Type: application/json" \ -d '{ "yamlFilePath": "${{ env.TEST_FILE_PATH }}", "serialNumber": "GH-${{ github.run_id }}", "operatorId": "github-actions", "unattendedMode": true }') EXECUTION_ID=$(echo $RESULT | jq -r .executionId) echo "EXECUTION_ID=$EXECUTION_ID" >> $GITHUB_ENV - name: Wait for result run: | while true; do STATE=$(curl -sf http://$MAESTRO_HOST:7000/api/testexecution/$EXECUTION_ID/status | jq -r .state) [[ "$STATE" =~ ^(Completed|Failed|Aborted)$ ]] && break sleep 3 done VERDICT=$(curl -sf http://$MAESTRO_HOST:7000/api/testexecution/$EXECUTION_ID/report | jq -r .verdict) [[ "$VERDICT" == "PASS" ]] || (echo "Test failed with verdict: $VERDICT" && exit 1) **From a MES (Manufacturing Execution System):** Maestro exposes a `POST /api/testexecution/run` endpoint that accepts a serial number and returns an execution ID. The MES can: 1. Submit a run when a unit arrives at the test station (pass the production serial number) 2. Poll `GET /api/testexecution/{id}/status` for completion 3. Retrieve the verdict from `GET /api/testexecution/{id}/report` 4. Write the verdict and measurement data back to the MES For advanced MES integration, Maestro includes a pluggable `IMesService` interface with lifecycle hooks for routing validation (can the unit be tested on this station?), pre-run validation (is the unit eligible?), and result reporting (push verdict back to MES). Contact the Maestro team for implementation details. ### Best Practices for Stable Automation **Discover test file paths before hardcoding them.** The server-side path for a YAML file depends on where the package is installed and is not predictable from the package name alone. Always call `GET /api/packages/test-files` to retrieve the exact path, and store that value in your pipeline configuration. PowerShell # List all available test files with their server-side paths Invoke-RestMethod -Uri "http://localhost:7000/api/packages/test-files" **Always enable** `unattendedMode` in automated pipelines. Without it, any prompt step will block indefinitely and your pipeline will hang. Review every test that runs in automation for prompt safety before enabling. **Wait for runner warm-up after a fresh package install.** The first execution after installing a new package version may fail with `runner unavailable` while the runner loads the new assembly. Wait 5--10 seconds after install before triggering the first run, or poll `GET /api/health` until all runners report healthy. **Use execution IDs, not serial numbers, for polling.** The execution ID returned from `POST /api/testexecution/run` is the stable handle for that specific run. Serial numbers are not unique across time. **Build in retry logic for transient network faults.** The REST API is synchronous and stateless --- if a trigger request fails due to a network blip, there is no partial state to clean up. Wrap your trigger call in a retry loop with a short backoff (2--3 attempts, 5-second intervals). **Set timeouts in your pipeline.** A stalled runner will never transition to a terminal state on its own. Set a pipeline timeout appropriate for the expected test duration, and call the abort endpoint if the timeout is reached. PowerShell $timeoutSeconds = 300 $elapsed = 0 do { Start-Sleep -Seconds 5 $elapsed += 5 $status = Invoke-RestMethod -Uri "http://localhost:7000/api/testexecution/$executionId/status" if ($elapsed -ge $timeoutSeconds) { Invoke-RestMethod -Uri "http://localhost:7000/api/testexecution/$executionId/abort" -Method POST throw "Test execution timed out after $timeoutSeconds seconds" } } while ($status.state -notin @("Completed", "Failed", "Aborted")) **Capture the config snapshot for traceability.** Every execution record stores the exact `cfg.*` values that were active at the time. Retrieve this with `GET /api/testexecution/{id}/report` and archive it alongside your CI artifacts so you can reproduce the exact conditions of any historical run.