c

The Rise of ChatGPT & What it Means for the Future of AI and Software Testing

Unless you’ve been living under a rock, you can’t have failed to notice how ChatGPT has been stealing the headlines since its launch in November 2022.

The stats are astonishing. It’s estimated the generative AI tool attracted one million users in its first week and had 100 million active users by the end of January 2023.

But ChatGPT is just the tip of the generative AI iceberg. There are already generative AIs for most scenarios, even brain to text. And it’s thought there are more than 500 generative AI startups, which have collectively attracted $11 billion in investment (and that doesn’t include the $10 billion in OpenAI, the company behind ChatGPT).

Examples of Generative AIs: source @ethersiim

There is commentary about how AI is going to transform every area of our work and home lives. The software testing space is no exception. But to understand the impact AI could have on our world, it’s worth going back to basics to understand the generative AI concept that underpins ChatGPT and other tools like it.

What is Generative AI?

ChatGPT is perhaps the most famous example of a generative AI (the GPT of its name stands for Generative Pretrained Transformer).

Generative AIs are trained on Web 2.0 content such as text, images and audio. They are fine-tuned with both supervised and reinforcement learning techniques and can then generate new plausible content.

ChatGPT is one of the largest trained language models because it’s underpinned by 175 billion machine learning parameters.

However, you get a sense of how far we still have to go in this field when you consider that its inputs are dwarfed by the Megatron-Turing Natural Language Generation model (from NVidia and Microsoft), which has the largest dense neural network with 530 billion parameters, and PaLM: Scaling Language Modeling with Pathways, which was trained on 540 billion inputs.

The Opportunities for Generative AI in Software Development

The wealth of inputs means there are few things generative AIs such as ChatGPT can’t do. The fact it can write code unlocks interesting opportunities in the software development process.

I’ve used ChatGPT to undertake the following:

The most exciting possibilities come with the volume at which AI can execute, especially in the Operate phase. In theory, you needn’t have any tests running. Instead, you could have real users acting as the testers, finding the bugs, and letting the generative AI fix them.

The Challenges with ChatGPT and Generative AI

But there are challenges with a scenario in which we rely too heavily – or perhaps too unthinkingly – on the capabilities of generative AI.

In the mainstream media, all the talk has been of how ChatGPT has passed advanced examinations in medicine, law, and finance. You’ve probably seen in our world that it passed the ISTQB® Foundation Level Sample Exam for ISTQB CTFL Certification. So does that mean ChatGPT is a doctor, a lawyer, or a certified tester?

You can spot the problems with this premise when we start to think about how generative AI could become embedded in our everyday lives. What if a generative AI made a faulty medical decision that resulted in a death? Or if one was found to exhibit racial or gender bias?

The ramifications are already being considered by the EU. The proposed EU AI Act proposes that infringements could lead to fines of up to €30,000,000 or up to 6% of total worldwide annual turnover, whichever is higher.

Even on a day-to-day level, there are technical complexities. How do you test AI technologies?

Related Content: Watch our on demand webinar as Paul Gerrard and Jonathon Wright discuss Chat GPT 4 Testers.

The Opportunities for Generative AI in Software Testing

Randy Hesse, Product Solutions Architect at Keysight's Eggplant outlines how this approach reflects the way the use of generative AI is going. Our AI-powered testing platform simulates user behavior, generates test cases, and evaluates the performance of the AI system or model.

This approach is perhaps best summed up with what’s fast becoming a mantra in this space – AI won’t take your job but someone using AI will. Because for me, generative AIs give us augmented intelligence or augmented testing capabilities, helping us with day-to-day tasks, whether it’s exploratory testing, automated testing, or performance security.

For example:

  1. Autonomous Design Automation (ADA) – auto-generating model-based tests (MBT) and therefore making genuine shift right practices a reality.
  2. Test Requirements Engineering (TRE) – providing knowledge and context-specific validation.
  3. Predictive Application Lifecycle Management (ALM) – helping to spot future defects so you can make better decisions about future tests.
  4. Quality Lifecycle Management – providing an executive scorecard for QA.
  5. Autonomous Test Automation (ATA) – auto-generating code and the associated test artifacts.
  6. Transferring knowledge via natural language processing (NLP) and sentiment analysis to provide both domain and context validation.

Keep a Watchful Eye on Generative AI

It’s clear the field of generative AI is evolving – rapidly. It’s also clear it will have profound impacts on the way we work as software testers. For this reason, it’s essential to stay in touch with the debate and start exploring how you can use the tools in your setup.

At the same time, it’s clear there are practical and ethical challenges that need to be assessed. My advice? Watch where the big players in the software landscape are going and follow their lead.

Want to learn more?

In case you missed it, check out our on-demand webinar with Forrester Research titled, Generative AI: The Future Of Software Testing. Principal Analyst, Diego Lo Giudice, explored how generative AI is revolutionizing the way we approach software testing.

Watch on-demand now

limit
3