Guidelines for Developers and QA Teams

This page complements the paper “Runtime Verification and Field-based Testing for ROS-Based Robotic Systems” and is an online repository of the guideline catalog for ROS-based Robotics projects.

You can further find the guidelines and information on the evaluation process. Reproduction kits, specifications, and accompanying code can be found in our replication package.

What’s New

• March 2026 — New exemplar added: UPPAAL automated verification toolchain for ROS 2 (PE2).
• March 2026 — New exemplars: R2D2 and ROFER ROS 2 fuzzers (MTA1).
• March 2026 — New exemplars: FRET (SDB1) and FRET+Ogma+Copilot pipeline (MTA2).

View all news →

Browse by Role

For Developers

Guidelines for preparing ROS-based systems for verification and testing.

• CI1–CI3 — Constraint Identification
• CD1–CD2 — Code Design
• I1–I4 — Instrumentation

For QA Teams

Guidelines for performing runtime verification and field-based testing.

• PE1–PE2 — Prepare Environment
• SDB1–SDB3 — Specify Behavior
• MTA1–MTA2 — Monitor & Test
• SE1–SE2 — System Execution
• AR1–AR2 — Analysis & Reporting

Guideline Catalog

Guidelines

Constraint Identification

• CI1 — Identify timing constraints
• CI2 — Identify security and privacy constraints
• CI3 — Identify safety constraints

Code Design and Implementation

• CD1 — Strive for ROS nodes with single responsibility
• CD2 — Ensure global time monotonicity of events and states

Instrumentation

• I1 — Provide an API for querying and updating internal lifecycle
• I2 — Provide an API for logging and filtering
• I3 — Provide an API for injecting faults in execution scenarios
• I4 — Isolate components for testing

Prepare Execution Environment

• PE1 — Understand the overhead acceptance criteria
• PE2 — Create models for runtime assessment

Specify (Un)desired Behavior

• SDB1 — Define properties using a logic-based language
• SDB2 — Use Domain Specific Languages (DSLs) to specify properties
• SDB3 — Use languages and tools to scenario-based specification of test cases

Monitor and Test Automation

• MTA1 — Improve the robustness of the system by performing noise and fault injection
• MTA2 — Exploit automation for test case generation, test case prioritization and selection, oracle and monitor generation

System Execution

• SE1 — Use record-and-replay when performing exploratory field tests
• SE2 — No GUIs! Prioritize headless simulation

Analysis and Reporting

• AR1 — Perform postmortem analysis to diagnose non-passing test cases
• AR2 — Use reliable tooling in order to manage field data