How AI-Powered Test Automation Eliminates PLM Challenges

Discover how Eggplant's smart test automation helps overcome the challenges of testing complex product lifecycle management software.

Key takeaways:

Product lifecycle management (PLM) software is a vital component required for efficiently orchestrating all product development activities of high-tech manufacturers.
However, the complexity and scope of PLM software also creates an enormous number of challenges that diminish some of its benefits.
Model-based AI-powered test automation effectively solves these challenges.

In high-tech industries like aerospace & defense, consumer electronics and automotive, PLM software facilitates the orchestration of all activities to streamline the entire product development process, achieve high product quality, and reduce time-to-market.

In this article, learn about the common PLM challenges faced by companies and about Keysight's solutions to streamline PLM verification and PLM validation.

Why is PLM essential in high-tech industries?

A chart with yellow rectangular boxes AI-generated content may be incorrect.

Figure 1. Components of PLM and example use cases

PLM software provides a central repository for product development data and activities over a product's entire lifecycle. This includes inception, prototyping, product design, manufacturing process, supply chain management, procurement, project management, process management, quality management, product launch, service, and end-of-life.

It acts as a single source of truth for data related to every stage of the product lifecycle — product information, computer-aided design (CAD) files, bills of materials (BOMs), specifications, test results, regulatory compliance documents, supplier information, pricing data, sustainability, document management, document revisions, and more for a company's entire product portfolio.

PLM integrates closely with other business systems like enterprise resource planning (ERP), manufacturing execution system (MES), and CAD.

Common PLM software solutions include Teamcenter, PTC Windchill, SAP PLM, and Aras — among others.

How is PLM software customized and tested?

A diagram of a diagram AI-generated content may be incorrect.

Figure 2. PLM development and test workflow

PLM deployments are complex undertakings requiring specialized knowledge and experience with the selected PLM solution.

Large companies have in-house PLM teams that know internal business processes best.

Companies also hire specialized third-party PLM service providers who implement customizations and upgrades after understanding customer needs in depth.

What are some critical PLM challenges?

Let's explore some challenges that can potentially reduce productivity and profitability while deploying, implementing, and testing PLM customizations and upgrades.

Accuracy and customization challenges

Robust PLM verification is crucial to ensure the accuracy of all the steps and data in core workflows. All custom workflows and other customizations must be checked after implementation and after each upgrade.

Another common challenge is dealing with user journeys that cross multiple software systems, applications, or device boundaries. End-to-end testing of such user journeys can be very challenging when attempted manually or with conventional tools.

Data management and migration challenges

Connecting the PLM to the siloed document management systems of different departments is a primary challenge for new PLM deployments.

Migration from legacy systems or other PLM systems and version upgrades often require significant data cleaning, transformation, and preparation to address inconsistent, invalid, or missing data.

While moving large volumes of data into or out of a cloud PLM service or related software-as-a-service (SaaS), ensuring data correctness, integrity, and security is difficult.

Integration challenges

The integrations of PLM with other enterprise systems like ERP, CAD, and MES are inherently complex. These points of integrations must be verified for correctness after any upgrade or customization; this is made more difficult as each will have customised business logic.

Large manufacturers often continue to use legacy systems due to organizational inertia or budget shortages. Ensuring PLM interoperability with such brittle legacy systems can be particularly problematic.

BOM accuracy challenges

Avoiding errors in bills of materials — such as inconsistent content, invalid supplier data, or missing information — can be a significant problem.

PLM testing challenges

The PLM validation process itself can face challenges like the following:

Resource shortages: Organizations may deprioritize PLM testing as less critical.
Manual testing problems: Even if resources are allocated, traditional manual testing can be time-consuming, expensive, difficult to scale, and not comprehensive.
Automated testing issues: Conventional automation testing has difficulties testing the quality of PLM user experience, integrating with DevOps, supporting the transition from manual testing, and ensuring that regression tests provide full coverage.
Functional test issues: Verifying that all customizations are working as intended after upgrades is a challenge. Changes in the PLM's core business logic during an upgrade can affect customizations. Robust continuous regression testing of customizations is essential to prevent such issues. Additionally, ensuring the accuracy and consistency of PLM workflows and configurations as per requirements and user expectations is a key challenge.
Scalability of testing: The pace of new releases, customizations, and upgrades to PLM platforms can increase test requirements exponentially.
User experience (UX) challenges: The 2D and 3D graphical nature of PLM applications and the challenges of simulating user actions make UX verification difficult with conventional tools.
Project planning challenges: A PLM software testing tool must be able to complete all required tests within the duration of a sprint.

Performance and scalability challenges

Maintaining optimal performance, usability, and availability for all users across a large organization is crucial. Additionally, the PLM and its customizations must scale as the company grows.

During upgrades, the PLM must remain stable and available until the switchover to the new version and remain so even after. Downtime must be minimized.

Deployment mode challenges

PLM vendors provide both on-premises and cloud-based PLM solutions. Both have various benefits and drawbacks. Some customers may also have both as a hybrid deployment. Seamlessly integrating these different deployments can be challenging.

Additionally, some PLM workflows may be optimized for specific devices, such as production floor managers using mobile devices for seeing real-time part availability.

Security and compliance challenges

Implementing and testing compliance with standards and regulations related to data safety, cybersecurity, and privacy are essential, particularly in critical industries like aerospace, defense, automotive, semiconductors, and consumer electronics. Addressing security vulnerabilities is critical.

How can manufacturers keep up with increasing PLM testing demands?

Manual testing is time-consuming, expensive, not scalable, and not comprehensive. It doesn't facilitate continuous delivery or allow for regular updates. Organizationally, it ties down your product teams in a non-scalable activity and does not identify business insights like productivity blockers.

Conventional test automation can overcome some of these issues but in small pockets and more suited for backend testing rather than end-to-end verification, which is what is needed. This approach continues to rely heavily on manual testing to provide the test coverage needed.

The best alternative to both approaches is intelligent test automation using Keysight’s Eggplant, which delivers several business benefits.

How can PLM testing be automated?

Verifying the correctness of PLM customizations and upgrades can be intimidating if you rely on conventional test methodologies and tools. That's why you need smarter tools that reason like experienced human testers but with even more diligence and efficiency.

Keysight's Eggplant Test is one such smart tool for automated testing that is able to intelligently understand and control the entire PLM landscape.

How can you manage the move from manual to automated PLM testing?

The transition from manual to automated PLM testing requires analysis of the efficiency, return on investment, potential increase in customer satisfaction, stakeholder impacts, and integration with CI/CD pipelines.

The Keysight 30-60-90-day plan is a detailed roadmap to migrate to automated testing based on Eggplant.

What is Keysight Eggplant?

Eggplant automates functional, integration, regression, and performance testing at the user interface (UI) and user experience (UX) levels using advanced computer vision and artificial intelligence (AI). Since a bug in any layer eventually manifests as a functional or performance anomaly in the UX, intelligent UI-level testing is an effective approach for identifying problems using an end user's perspective. Anomalous UI paths can then be further investigated using focused front-end and back-end tests.

How does Keysight Eggplant automate and streamline testing?

Let's look at Eggplant's capabilities in more detail.

Support for large enterprise applications

Eggplant is designed to automate large business systems like PLM, CAD, MES, and ERP that consist of thousands of screens and workflows.

Computer vision techniques

Eggplant sees and understands the screens and controls at a higher abstraction level like a human would by using a number of optical character recognition and intelligent image recognition technologies. This approach means tests are robust against UI changes or system upgrades.

An image search engine of graphical artifacts is also available for more complex image recognition based on 2D or 3D shapes, textures, colors, and other visual characteristics.

Model-based testing

A diagram of a diagram AI-generated content may be incorrect.

Figure 3. Digital twin model for a UI

By creating a model (or digital twin) of the application being tested, Eggplant's machine learning algorithms identify all possible user journeys and interactions, enabling automated exploratory testing.These models replicate different states, representing screens that users visit, and the actions that users might perform within those states or move users between states.

The digital twins are used as a “blueprint” to define all possible user journeys (tests) to check functional correctness and performance, which can be easily defined, managed and/or auto generated with AI.

AI-based test generation

Eggplant applies AI reasoning to generate test cases based on your digital twin model. AI-powered exploratory testing provides a powerful and easy means of generating tests to accelerate coverage and find problems that would otherwise be missed through traditional testing.

If test requirements change, it will automatically regenerate necessary tests to include these changes.

Database and document capabilities

Eggplant can examine databases and access various document formats to read/update their context and use this information to compare and validate what is shown to users is correct.

Assisted test updates

A powerful capability of Eggplant is its ability to automatically and semi-automatically adapt tests to system changes. Since Eggplant relies on image and text recognition to find elements on the screen, if any UI element's visual attributes change after a new install or update, Eggplant can still find it. For UI elements whose appearance change dramatically, Eggplant provides “image sets” to map different appearances to the same UI element. All this makes Eggplant's tests more dynamically adaptable to UI changes, reducing the need for manual intervention.

AI-augmented testing for high-tech industries

Using AI, Eggplant Generator reduces manual effort and improves accuracy by automatically generating detailed test scenarios and conventional test cases based on input documents. It’s trained on various testing standards and content related to industry verticals like aerospace, defense, telecommunications, and others to obtain industry-specific insights.

Application programming interface (API) testing

Eggplant can understand and create end-to-end contract testing for APIs based on their specifications.

Custom test scripts

Eggplant supports a low-code natural language-like scripting approach using SenseTalk with extensive capabilities.

Performance testing

Eggplant can measure and monitor the performance and availability of PLM and other systems. It can provide timing data, responsiveness, and other UX metrics.

DevOps integration

Eggplant facilitates automated continuous testing by integrating with a variety of continuous integration and continuous delivery (CI/CD) tools like Azure DevOps and GitHub Actions.

Test coverage reports

Eggplant provides business insights using dashboards of test automation metrics for better decision-making.

Data security and confidentiality

Eggplant offers security measures for handling sensitive data like passwords.

What does Keysight Eggplant look like in action?

To get a better idea of PLM testing using Eggplant, let’s review some demos and screenshots of it in action.

In a presentation called "How to use Eggplant Functional to test any PLM system" by Keysight's Kieran Leicester, you get practical insights into improving a PLM system's reliability and performance using Eggplant functional tests, performance tests, and custom scripts.

The below screenshot from the presentation shows a functional test script that Eggplant automatically created by observing a user's interaction with a PLM workflow.

Figure 4. Functional test for a PLM workflow

From the same presentation, here's a custom script to manipulate a PLM's CAD integration.

A screenshot of a computer AI-generated content may be incorrect.

Figure 5. A custom script for a PLM workflow

The next screenshot from another demo shows a user journey model that Eggplant created by observing a user's interaction with a PLM software.

A screenshot of a computer AI-generated content may be incorrect.

Figure 6. Eggplant model showing all user journeys of an application

In another presentation called "Harnessing computer vision to automate PLM & CAD testing" by Thierry Zuzel and Vincent Henneuse, you get an in-depth look into Eggplant's AI capabilities for various PLM and CAD use cases.

How does Keysight Eggplant address PLM challenges?

Armed with the above capabilities, here's how Eggplant helps mitigate the PLM challenges we saw earlier.