Figure 1 Graph Showing Method 5 Acceptance Limits
Several organizations and publications reference or insist on maintaining a 4:1 Test Uncertainty Ratio (T.U.R.) without understanding the level of risk that they may be subjecting themselves to. The general thought is as long as the lab performing the calibrations has standards at least four times better then what they are calibrating that everything is good. In fact, ANSI/NCSL Z540.3 – Requirements for Calibration of Measuring and Test Equipment in section 5.3 b) allows for use of a test uncertainty ratio (T.U.R.) equal to or greater than 4:1 when it is not practical to estimate the false accept risk of less than 2%. Then goes on to say objective evidence of non-practicability of this determination is expected as in an agreement with the customer T.U.R. use. The 4:1 T.U.R. seems to be a “fall back” position that a lot of the industry has adopted maybe because they did not understand or want to deal with guard bands. The assumption is the higher the T.U.R., the higher the probability the measuring equipment will have a Probability of False Accept (PFA) of less than 2 % as required by the standard assuming the measured reading is closer to the nominal value. However, laboratories who have come to insist on a 4:1 T.U.R. may have to change some of their thinking when they want to comply with ISO/IEC 17025:2017 as the new standard no longer allows for this rationale. Section 184.108.40.206 states “When a statement of conformity to a specification or standard for test or calibration is provided, the laboratory shall document the decision rule employed, taking into account the level of risk (such as false accept and false reject and statistical assumptions) associated with the decision rule employed and apply the decision rule”.
We can think about the risk this way. We have a car and we need to park it between two lines. The lines represent the upper and lower specification limit of our device. The width of our car is the uncertainty and parking lines are our tolerance specification limits. The probability of us getting a ding or denting another vehicle is our PFA depending on how centered we are within the parking lines. If we try to park too close to one side, we may risk not being able to open the door, or if we completely misjudge we may run right into the car in the other lane and cause substantial damage. If we park centered on the line, 50 % of our car will be in the next lane no matter what size our car is. T.U.R. works the same way. No matter what T.U.R. ratio we have, if we are right at the non-guard band specification limit, there will be at least a 50 % chance that our measurement falls outside of the specified tolerance (see figure 2). This paper is going to discuss T.U.R., why the location of the measurement matters, PFA and some common guard banding methods used to assure measurements are compliant and stay within the lines.
Figure 2 31.23:1 T.U.R. with a 50 % PFA at the Upper Specification Limit
T.U.R. or Test Uncertainty Ratio defined in Section 3.11 of ANSI/NCSL Z540.3 as, “The ratio of the span of the tolerance of a measurement quantity subject to calibration, to twice the 95% expanded uncertainty of the measurement process used for calibration." What the T.U.R. tells us is how much space between the lines we have to be “in-tolerance”. NCSLI RP-12 states in section 12.3 “The uncertainty in the value or bias always increases with time since calibration”. The recommendation is to analyze the data and set the tolerances so that the calibration supplier has uncertainties low enough to make the statement of conformity, and that any drift be accounted for between calibration cycles.
NCSLI RP-12 Section 12 suggest when developing equipment tolerance, it may be prudent to include uncertainties due to factors that are not normally included in the list of measurement process errors applicable to calibrations. Several organizations are concerned with setting the system to a certain accuracy and making sure the calibration laboratory adjusts within the specification, however when adjustments occur frequently, it becomes apparent that the tolerances are not appropriate for the device or the cycle time between calibrations is not set appropriately. Setting higher tolerance or specification limits will improve the T.U.R. ratio. Using a calibration provider with low uncertainties will help raise the T.U.R. ratio. The higher T.U.R. will result in wider acceptance (compliance) limits. Wider acceptance limits give more room to account for the bias increase that will occur between calibrations. However, if the uncertainties are not properly accounted for, the probability of the PFA being higher than 2 % will increase. It is important to consider all sources of uncertainty when determining the time between calibration as well as tolerance limits and in some cases, the manufacturer’s tolerance may not be achievable.