A doctor uses digital healthcare technologies to check vital signs

Vital Signs: The Rise of Automated Healthcare Software Testing

In the wake of the pandemic, healthcare organizations have been propelled into a digital transformation to stay competitive. With the healthcare software ecosystem contributing to 30% of global data, there is no going back now. However, software testing for this expanding digital frontier presents daunting challenges - from more complex systems to stricter regulations.

The crucial question: How to modernize testing strategies to ensure software quality and patient safety in a time of disruption?

Keysight commissioned a survey with the Healthcare Information and Management Systems Society (HIMSS) to answer this critical question and gain insight into key trends and future perspectives on software testing.

Read the full HIMSS report

Confronting the scalability challenge in manual software testing

Despite automation being embraced across industries, the healthcare industry lags behind. Only 15% of healthcare providers surveyed adopted test automation platforms.

The reality is that today's Electronic Medical Records (EMR) systems are becoming increasingly customized, interfaced, and complex. Their upgrade cycles have quickened, and with every new patch or alteration, there's a risk of introducing bugs or vulnerabilities. The pace of change, coupled with the interconnectivity of system components, surpasses the capacity at which manual testing approaches can keep up, leading to:

●Merely 33% of manual testers are satisfied with their organizations' capacity to scale testing to meet new requirements.

●Only 38% are satisfied with the availability of clear testing analytics.

●Only 38% of manual testers believe their organizations can adequately develop test coverage across all devices and browsers.

The Vital Signs HIMSS study shows a low percentage of manual healthcare software testers who are satisfied with their organization's capabilities

Figure 1. Percentage of respondents using manual approaches who are satisfied with their organizations' capabilities in specific areas

As the shortage of proficient testers increasingly emerges as one of the significant bottlenecks in scale testing, it is imperative to shift towards a more automated approach. Such a transition will free up testers to concentrate on more complicated, value-added tasks.

View all HIMSS insights here

The Dilemmas of DIY Software Testing

The Do It Yourself (DIY) testing approach, favored by over 40% of respondents, has several drawbacks that render it counterproductive and inefficient.

The major challenge is the lack of technical talent to build and maintain in-house testing solutions. According to a recent Forrester report, 63% of organizations following a DIY route use three or more development tools or automation solutions for testing.1

The complexity of healthcare applications, coupled with diverse automation frameworks and programming languages, demands specialized expertise. Consequently, only 38% of the respondents using DIY testing approaches report satisfaction with their organization's ability to enhance QA team job satisfaction and productivity. This satisfaction rate is five percentage points lower than that of teams relying on manual testing approaches.

The second challenge arises when one considers the hidden costs associated with DIY testing. Expenses such as recruiting specialized personnel and investing in extra infrastructure can escalate quickly, making the DIY approach more costly than initially budgeted. For instance, when the time spent on writing scripts, fixing bugs, and updating tests becomes a bottleneck, that delays the release.

Moreover, the time consumed in script writing, debugging, and updating assets can often bottleneck the process, potentially leading to release delays. Only 38% of the respondents using DIY testing approaches were content with their organization's ability to fulfill all prerelease requirements – ten percentage points lower than that of organizations using manual approaches.

Finally, it's important to consider the imperative of interoperability among different healthcare software to stay compliant with emerging regulations. Both manual and DIY testing approaches struggle to keep up with the continuous and simultaneous changes within multiple interconnected system components. This is evident from the survey results, where less than half of respondents using manual or DIY approaches were content with their organization's ability to achieve comprehensive test coverage across multiple platforms.

In conclusion, the mounting complexity of healthcare software requires a strategic shift to more robust testing solutions that enables healthcare organizations to scale their testing capabilities affordably and flexibly.

Embracing Automated Software Testing for Resilience

The silver lining is that 75% of healthcare providers plan to adopt test automation in five years. Early adopters are already experiencing the benefits, with 80% expressing satisfaction with their organization's ability to scale testing to meet emerging requirements.

The Vital Signs HIMSS study shows the benefits of test automation in healthcare

Figure 2. The percentage of respondents satisfied with their organization's ability to scale to meet emerging testing requirements2

If you are venturing into automated software testing for the first time or replacing in-house tools with a commercial platform, I encourage you to download the full report and explore the comprehensive survey findings and best practices. I hope you find helpful insights to future-proof your testing strategies in the Vital Signs report.

Explore more findings and insights from leading industry experts

webinar on healthcare software testing


  1. Look Before Your Leap: Understand the Trade-Offs When Choosing Test Automation Software. Forrester, 2023.
  2. Low base size, insights directional only.