Abstract—As embedded systems in safety-critical domains, such as
transportation, become increasingly complex, ensuring their reliability thorough
testing becomes essential. Manual testing methods are often time-consuming,
error-prone, and inadequate to cover all possible failure scenarios, especially
in systems that require a high level of functional safety and robustness. To
address these challenges, the need for automation in the testing process is
evident, particularly for real-time, embedded, and hardware-dependent systems
where validation is crucial for ensuring safe operation.
This paper
proposes a structured workflow for automating the testing of a safety-critical
embedded subsystem, utilizing a real hardware-in-the-loop (HIL) environment.
The approach covers all major phases of the testing process, including test
specification, execution, and results assessment. The inspiring idea is to try
to shift the focus of intellectual effort toward the early stages of the
validation process, facilitating a clear and shared understanding between
testers and developers regarding the system's behavioral and functional
requirements.
The first phase
involves creating a detailed test specification, which includes: developing a
behavioral model of the feature to be validated; defining data probes for
monitoring the subsystem under test; creating simulation scenarios to provide
stimuli to the subsystem’s external interfaces. In the second phase,
executable test scripts are generated either automatically or
semi-automatically using multi-technology scripting languages, followed by the
creation of a sequence of test cases. The final phase involves executing the
tests on the actual hardware platform, collecting execution logs, and running
automated checks to evaluate the results.
The workflow and tools are evaluated using
a case study of the onboard subsystem hardware and software platform developed
by Hitachi Rail