Automotive AI: Bringing AI Safety Assurance into Engineering Practice
Keysight and the Centre for Assuring Autonomy (CfAA) at the University of York are exploring practical approaches to AI safety assurance for software-defined vehicles
From AI Safety Requirements to Engineering Practice
The automotive industry is entering a new phase of AI adoption. AI is moving beyond research projects and isolated driver assistance features to become part of the software-defined vehicle, influencing functions and decisions that can directly affect vehicle behavior.
As this shift continues, engineering organizations need to demonstrate that AI-enabled systems operate safely, reliably, and as intended throughout their lifecycle. This requires more than evaluating individual machine learning (ML) models or producing isolated test results. It requires evidence that can be interpreted in context and connected to a structured argument about system-level safety.
Standards such as ISO/PAS 8800 provide an important foundation by defining expectations for AI safety in road vehicles. The next step is to develop practical methods that help engineering teams apply these expectations within real development processes. This is the focus of the research collaboration recently announced by Keysight Technologies and the Centre for Assuring Autonomy (CfAA) at the University of York. In this blog, we take a closer look at what that means for engineering practice.
Why This Collaboration Matters
Much of the current discussion around automotive AI safety focuses on standards, regulation, and future certification expectations. For engineering teams, however, the immediate task is to determine how these expectations influence development, validation, and assurance activities in practice.
The collaboration between Keysight and the CfAA brings together complementary perspectives on this question. The CfAA contributes expertise in systems safety and AI assurance, while Keysight brings experience in AI validation, measurement, and engineering workflows. Together, the organizations are exploring how assurance concepts can be translated into methods that are both technically grounded and applicable within automotive development programs.
The Collaboration Scope
The collaboration focuses on four closely connected areas:
- exploring how evidence-driven approaches can support the implementation of ISO/PAS 8800;
- investigating how relevant aspects of AI safety can be made measurable and assessable through methodologies grounded in research and standards;
- examining practical methods for generating and organizing AI safety evidence; and
- understanding how this evidence can support structured safety arguments and justified confidence throughout the lifecycle of an AI-enabled system.
The objective is not simply to produce more validation results. It is to understand how different forms of evidence can be generated, interpreted, and connected to support confidence in AI-enabled automotive systems.
Spotlight: Professor Simon Burton
Portrait of Professor Simon Burton
Few people are better positioned to speak about the future of automotive AI safety than Professor Simon Burton. As Chair in Systems Safety at the University of York and former convenor of the ISO working group that developed ISO/PAS 8800, he has played an important role in shaping current approaches to AI safety in road vehicles. His perspective is also grounded in extensive automotive industry experience, including leadership roles at DaimlerChrysler, Bosch, and Fraunhofer.
This combination of standards development, systems safety research, and industry experience enables him to connect emerging assurance concepts with the realities of engineering programs, development processes, and safety-critical vehicle systems.
Three Questions for Professor Simon Burton
I asked Professor Burton what currently separates AI safety requirements from engineering practice, how the necessary evidence can be justified, and what is needed to make AI safety assurance repeatable at industry scale.
1. Many automotive organizations understand that AI safety is important. What do you see as the biggest gap between AI safety requirements and engineering practice today?
Established approaches to demonstrating the safety of software-intensive vehicle functions as described in standards such as ISO 26262 rely on engineering processes that “design safety into the software” through the use of best-practice software architectures, systematic refinement of requirements to code-level, programming guidelines, static analysis and manual reviews. Various levels of testing are then used to confirm that the desirable properties of the software are indeed achieved. This removes the reliance of software safety on a single test result, allowing an argument to be formulated based on a set of complementary activities and evidence, thus increasing confidence.
Many of these conventional software safety techniques cannot be directly applied to machine-learned (ML) models. In fact, AI safety requirements are often expressed in terms of quantitative metrics, with the primary source of acceptance being test results created against reference datasets. This raises the question of whether it is possible to argue a similar level of safety integrity for AI-based functions as for conventional software, and if so, with which combination of design-time, testing and operation-time evidence?
2. ISO/PAS 8800 describes what organizations need to demonstrate. What do you see as the biggest challenge when it comes to generating and justifying the necessary evidence?
AI models are typically evaluated based on performance metrics evaluated against reference datasets. Whereas safety itself should always be considered an emergent system-level property of the function. Although quantitative risk acceptance targets at the system level have been proposed as a means of evaluating safety by standards such as ISO 21448 and ISO TS 5083, a specific method of deriving measurable properties and target values for AI and, in particular, ML models has yet to be described in the standards. Therefore, methods must be developed to show how different properties of ML models, expressed as metrics, can be combined to justify that system-level behavior will remain safe. Any such quantitative analysis will be subject to uncertainty due to limitations in the testing process and the opaque, non-uniform behavior of the models. This uncertainty must therefore be recognized and addressed within the assurance process.
3. Looking ahead, what needs to happen for AI safety assurance to become a repeatable engineering practice across the automotive industry?
First of all, we need agreement on, and recognition of, the safety assurance challenges introduced by AI. Safety standards such as ISO/PAS 8800 are an excellent first step in this direction, and it was a career highlight of mine to steer the consensus process within the automotive industry as convenor of the working group that created this standard. These standards now need to be anchored into both regulation and industry best practices. As a next step, detailed methods for instantiating the standards for specific application areas, such as ML-based visual perception tasks, are required. These should not only be grounded in safety engineering and computer science foundations, but also need to be validated on industrial scale case studies. Lastly, effective tool-support is essential to allow the generation and evaluation of safety assurance evidence to be directly integrated into the highly iterative engineering processes used to develop AI-based functions. This combination of tool support, best practice methods and reusable safety case patterns is essential to reduce the burden of safety assurance from series development engineering processes and scale the application of AI in safety-critical applications.
What This Means for Automotive Engineering
Professor Burton’s perspective highlights an important shift. AI safety assurance cannot rely on a single metric, dataset, or test result. Confidence must be built from complementary evidence spanning model properties, system behavior, engineering processes, and operation.
For automotive organizations, this means connecting assurance more closely with engineering activities. Evidence needs to be generated throughout development, remain traceable to the relevant requirements and assumptions, and contribute to an argument about system-level safety. It must also remain meaningful as models, data, and operating conditions evolve.
Making these relationships visible and manageable is an important step toward turning AI safety assurance into a repeatable engineering practice.
From Collaborative Research to Engineering Practice
As research findings emerge, they are expected to inform future development directions for the AX1000A AI Software Integrity Builder, particularly in the areas of AI safety evidence, assurance, and lifecycle confidence.
The broader objective, however, extends beyond any individual tool. Standards such as ISO/PAS 8800 establish an important foundation for AI safety in road vehicles. Applying them consistently will also require detailed methods, integrated engineering workflows, and effective tool support.
By combining the CfAA’s expertise in AI assurance and systems safety with Keysight’s experience in AI validation and measurement, the collaboration is exploring how these elements can work together in practice. The aim is to help make AI safety assurance more structured, repeatable, and applicable to real-world automotive development programs.
As AI becomes more deeply integrated into vehicle functions, justified confidence will need to be engineered alongside performance. Developing the evidence, methods, and workflows needed to achieve this is the shared focus of the collaboration between Keysight and the Centre for Assuring Autonomy.