Automated unit testing has long been a staple in software development, providing a reliable method for ensuring code quality and stability. However, its application to Manufacturing Execution System (MES) projects presents unique challenges. The real-time data requirements of MES systems make it difficult to structure meaningful and effective tests, as these systems often depend on live inputs that are not easily replicated in a test environment. Additionally, many MES platforms lack the built-in support or compatibility needed to host and leverage automated unit testing frameworks, further complicating efforts to implement such practices. Finally, MES configurators, who are typically more focused on system configuration and integration than on traditional software development practices, often lack the knowledge or experience necessary to design and execute automated unit tests, leading to a gap in the testing strategies employed within MES projects.
However, considering automated testing strategies in MES initiatives is crucial due to the high stakes involved in coordinating manufacturing operations. MES serves as the backbone of production environments, ensuring that processes run smoothly and efficiently. Any changes or improvements made to these systems, if not properly tested, could lead to significant disruptions in production, potentially causing costly delays, quality issues, or even complete production stoppages. Automated testing allows for the rapid verification of changes, ensuring that new configurations or updates do not negatively impact production. This ability to quickly and reliably test system modifications is essential for maintaining continuity and stability in the manufacturing process, where even minor issues can have drastic consequences on output and overall operational efficiency. By implementing automated testing, organizations can reduce the risk of unintended production impacts, ensuring that their MES continues to function optimally as the system evolves.
MES vendors are gradually beginning to embrace the importance of automated testing functionality, recognizing its value in maintaining system integrity during continuous improvement efforts. Platforms like Critical Manufacturing have taken the lead by incorporating inherent support for automated unit testing, making it easier for the team to implement and manage these testing strategies directly within the system. Other platforms, such as Parsec TrakSys, may not offer native support for automated testing, but they allow for the integration of backend software development modules that can host and run automated testing frameworks. This flexibility enables organizations to tailor their testing approach to suit their specific MES environment, ensuring that the system remains robust and reliable as it adapts to evolving production needs. As more vendors integrate such capabilities, the adoption of automated testing in MES projects is likely to increase, further safeguarding production processes from the risks associated with system changes.
In the case of TrakSys, RoviSys has implemented an automated testing harness, leveraging both the APIs exposed by TrakSys and the MSTest framework provided with Microsoft .NET to develop automated unit tests. The ability of the TrakSys platform to host native Microsoft .NET code enables this harness.
One of the key challenges of automating testing within an MES platform is ensuring consistent process data is contained within the data store. For example, a known process order, in-progress run or batch, specific machine state, quality test results, and various other entity values all may need to be established, which can be a tedious undertaking when done manually. Additionally, with traditional manual testing, once an entity is seeded and a test executed this entity can no longer be leveraged again because the system state has changed. A PO may have been completed and thus cannot be leveraged again when testing how the system handles pending PO activation for example.
The RoviSys test harness allows system state to be established at the beginning of the automated test via the usage of interface calls that wrap the entities, then the test harness 'tears down' this state at the conclusion of the automated test. An interface call can be made within the test to establish PO number '32456' with a status of 'Pending', then remove the PO at the end of the test, allowing that same PO to be re-seeded if the automated test is repeated.
The test harness further allows specific system operations to be performed using the established system state and then allows results to be verified. For example, maybe the goal of the test is to ensure that upon activation of a PO, a batch is established with an initial quantity of zero, and the target quantity is written to an output parameter. Leveraging the automated testing framework, the PO number with a specific target quantity can be established with a status of pending, calls can be made to activate the PO, and the resulting batch number and output parameter values can be queried and verified. At the end, all this data is removed, leaving the system in the same state that it was in prior to test execution.
Via the MSTest framework, unit tests can be executed in bulk and results seen at a glance. During deployment exercises (such as a CI / CD pipeline), a battery of unit tests can be quickly executed that confirms no breaking changes were introduced, streamlining manual testing processes that may take hours or even days to complete, or worse are simply not performed due to time constraints.
Usage of automated unit testing enables the technical team to create enterprise-level MES solutions that reduce the risk of undue production impacts. It is a consideration we make during every MES initiative as part of our DxExecution offering.
