Keysight Pass/Fail Decision Rules - Application Note
Application Notes

Keysight Pass/Fail Decision Rules - Application Note

Show Description

Table of content

  • Measurement Decision Risk Overview
  • Measurement Decision Rules and Why You Should Care
    • Compromised Decision Rules of Third-party Calibration Services
    • Decision rules of Keysight Calibration Services
  • Calibration Services
    • Keysight Calibration
    • Keysight Calibration + Uncertainties
  • Keysight Cal + Uncertainties + Guardbanding
  • Estimating the Level of False-Accept or False-Reject Risk
    • Estimation of specific false-accept risk
    • Test uncertainty ratio

Measurement Decision Risk Overview

In a pass/fail test application, the test system makes a measurement and then compares the result to a published specification. This repeated process ensures the device under test (DUT) conforms to its specified requirements. As much as we would like to trust the accuracy of the pass/fail measurement result, there are always two possible outcomes:

  • A correct pass/fail decision
  • An incorrect pass/fail decision a. Called “false accept”, or “false reject”

This is because all measurements contain errors. The “uncertainty of measurement” includes errors expressed as a standard deviation of all error contributions [1]. Figure 1 shows two identical measurement results but with different measurement uncertainties. The expanded uncertainty1 in the case “A” measurement result occurs within the tolerance limit. The case “B” measurement result has a significantly larger measurement uncertainty. There is a higher risk of falsely accepting a result in case B, because of the larger measurement uncertainty as indicated by “What is the level of risk?” in Figure 1.

Measurement Decision Rules and Why You Should Care

A measurement decision rule “describes how measurement uncertainty is accounted for when stating conformity with a specified requirement” (clause 3.7 of [3] in the ISO/IEC standard).

Different decision rules result in different levels of false-accept or false-reject risk. As a result, the revised ISO/IEC 17025:2017 [3] includes a requirement to provide end calibration customers with the details of the decision rule used. “When the customer requests a statement of conformity to a specification or standard for the test or calibration (e.g., pass/fail, intolerance / out of tolerance) the specification or standard, and the decision rule shall be clearly defined. Unless inherent in the requested specification or standard, the decision rule selected shall be communicated to, and agreed with, the customer” (clause 7.1.3 of [3]).

Compromised Decision Rules of Third-party Calibration Services

Often, a shortcoming of third-party calibration services is the use of performance measurements for only a subset of published specifications. Also, these services may make calibration measurements in reference to out-of-date lab standards which can lead to incorrect pass/fail decisions when the instrument is used in test applications. Additionally, when a third party observes a parameter out of specification, the risk of a false reject is significant. This means that the “true result” maybe within specification. Likewise, a parameter that appears within specification, a false-accept, may actually be out of specification. In any case, the third party is not able to perform the proprietary instrument adjustments needed to return an instrument to specification.