System Testing for Medical Equipmentf for US Market

Our primary objective was to conduct comprehensive system testing of the medical irrigation pump to ensure compliance with regulatory requirements. We operated on the system testing level, focusing directly on the end-user experience, as part of a structured testing process that included various testing levels, from code to user interface.

We were tasked with:

• Designing and executing test cases based on client specifications

• Assembling and configuring test setups  

• Analyzing results to identify defects and areas for improvement

• Using primarily black-box testing methodology focused on end-user experience. We also used gray-box testing approaches in some cases.

Implementation Process

We were integrated into the existing testing structure, working within an agile framework. Our team was a critical addition to the client's testing capabilities, as we were the first system-level testing specialists at a local branch.

Our part of the testing team consisted of four testers. Throughout the project, we flexibly allocated resources between test case creation, test environment setup, and test execution. In the final phase, three of our testers focused on test execution, while one specialist managed and documented identified defects.

To accelerate the development and testing cycle, daily meetings were held where our testing team provided immediate feedback on test results. This approach significantly reduced the time between issue identification and resolution, helping to maintain project momentum.

Microsoft Azure was used to track our workload and progress, though later in the project this shifted to Excel-based tracking.

Our involvement significantly improved the reliability and regulatory compliance of the client's medical devices. Key benefits included:

System Performance and Safety Improvements

• Our testing helped improve the precision of device parameter settings, contributing to better procedural control in clinical environments.

• We contributed to the optimization of safety algorithms that are critical for patient protection during procedures.

• We identified opportunities to enhance the user interface, resulting in more intuitive operation and reduced risk of user error.

Accelerated Project Timeline

• Our team's contribution accelerated the project timeline by an estimated 6-9 months.

• We completed the system testing phase within the planned timeline.

• Our ability to quickly adapt to the client's proprietary testing tools allowed for immediate productivity.

Knowledge Transfer and Team Development

• We trained new colleagues specializing in system testing at the client's local branch.

• We established testing procedures that could be reused in future projects.

• We provided comprehensive knowledge transfer to ensure continuity after our engagement.

Supporting Quality Assurance

• We conducted a thorough system analysis that led to the identification and resolution of potential reliability issues.

• We thoroughly verified all critical safety functions.

• We ensured systematic test coverage across all device functions and operational modes.

Testing Tools

For visualization and verification of test results, we utilized the proprietary software tool developed by the client. This tool enabled us to create graphs combining data from external pressure sensors and weight sensors along with internal values measured directly by the device. Given the key role of this tool in the testing process, it was essential for our team to become familiar with its functionality within the first two weeks of the project. It became our team's fundamental tool for graphical representation of test results and precise numerical verification of measured values.

For monitoring and adjusting the device's internal variables, we utilized another proprietary tool, developed by the client. We also simulated device connections for RF electrode and Shaver with a tool that enabled comprehensive testing across various system configurations.  

We utilized the GS.Net web application for remote device configuration and data retrieval over USB. This tool was essential for programming new software versions and obtaining diagnostic data from the device throughout the testing process.

The remaining testing consisted of thorough visual inspection of the graphical user interface (GUI), during which we verified the correct display of all elements, their functionality, and user-friendliness.

Agile Collaboration and Test Case Design

As part of the agile development process, we collaborated closely with the client to design comprehensive test cases derived from detailed system requirements.

Our approach involved:

• Requirements Analysis: Our team thoroughly reviewed system requirements for the medical equipment and provided critical feedback to ensure testability. We collaborated with system engineers to refine requirements to prevent contradictions and ensure they could be properly tested at the system level.

• Test Case Development: Creating detailed test scenarios and conditions to thoroughly evaluate device performance under various operational circumstances. Our team employed mainly manual testing methods to cover all aspects of device functionality, including edge cases and stress conditions. We created approximately 150-200 test cases, which was about 40 % of all test cases across the entire project.

• Test Setup Assembly: We assembled test setups that mimic real-world medical conditions to ensure devices perform accurately and reliably using as few resources as possible. Each setup was validated to ensure it met the necessary standards for realism and accuracy.

Testing Process

The testing process included multiple phases:

1. Initial Dry Runs: We conducted preliminary testing to identify issues in both the device and test specifications

2. Test Refinement: Based on dry run results, we improved test cases and provided feedback on requirements

3. Formal Test Execution: Systematic execution of test cases according to established protocols

4. Daily Collaboration: Regular meetings to discuss progress and issues

5. Documentation: Comprehensive documentation of test results for regulatory compliance

Execution and Analysis

• Test Execution: Performing rigorous testing on the devices, ensuring that we simulated real-world conditions to evaluate how the devices performed under various scenarios, capturing data on functionality, reliability, and compliance.

• Test Reviews: During test reviews, we collaborated with teams responsible for design and quality to ensure comprehensive coverage of all scenarios, validate test case effectiveness and if it is up to standard, and discuss identified defects and their resolutions. The primary goal was to ensure that every aspect of the device had been thoroughly tested on the system level and that the test cases met the highest standards of quality and completeness.

• Reporting: Identifying and documenting defects and providing detailed reports to the client's development team for further action. Also documenting positive test results for later FDA submission.

A leading technology partner to medical and advanced industrial original equipment manufacturers.