How do you know if your device is “In-Tolerance”? Why the location of the measurement is more important than you may realize.
Calling an instrument “In Tolerance” is about location, location, location.
It’s also about the uncertainty of the measurement, but a bad location is going to raise the Probability of False Accept (PFA) significantly. The probability of false accept is the likelihood of a lab calling a measurement “In Tolerance” when it is not. The location we are referring to is how close the measurement is to the nominal value. If the nominal value is 10,000 lbf and the instrument reads 10,004 lbf, the instrument bias is 4 lbf. The larger the bias, the worse the location of the measurement. In previous blogs, we have dealt with measurement risk, statements of compliance, and rules to lessen risk. This blog is a little different as we are going to focus primarily on the location of the measurement and what it means regarding making statements of compliance. Lastly, we are also going to promote our new free to download for Morehouse Force Measurement Insiders excel template for calculating PFA. You will not find a better value elsewhere! Guaranteed as it’s free!
Why do we care about the location of the measurement if the device is within tolerance? If a device has a specification of 0.1 % of full scale and the calibrating laboratory reports a value within 0.1 % the device is “In Tolerance” right? The answer is and always will be it depends on what the uncertainty of the measurement is and if the lab performing the calibration followed the proper guidelines in determining the uncertainty of measurement (UOM) when making the statement of compliance or conformity. For a refresher why this matter, there is a certain standard called ISO/IEC 17025: 2005 wherein section 184.108.40.206 there is this statement “When statements of compliance are made, the uncertainty of the measurement shall be taken into account.” Then ISO/IEC 17025: 2017 clarified this statement in section 220.127.116.11 which states, "when a statement of conformity to a specification or standard for test or calibration is provided, the laboratory shall document the decision rule employed, taking into account the level of risk (such as false accept and false reject and statistical assumptions) associated with the decision rule employed and apply the decision rule". Section 18.104.22.168 The laboratory shall report on the statement of conformity such that the statement clearly identifies –a) to which results the statement applies; and –b) which specifications, standard, or parts thereof are met or not met; –c) the decision rule applied (unless it is inherent in the requested specification or standard) If you need an accredited certificate, this statement can be a big deal, and If the uncertainty of the measurement is significant, the lab performing the calibration is going to have to be very concerned with the location of the measurement. In fact, if their uncertainty of measurement is too high, they may not even be able to perform the calibration at all. You may not even know you have an issue with your measurements until your next audit.
Why we care about risk and making statements of compliance:
2. ANSI/NCSLI Z540.3 -2006 Handbook Section 3.3 paragraph 2 states
3. We care because the probability of an incorrect decision being made about our measurement is quite high if we have a higher than 2 % probability of false accept.
In hopes to simplify things, we are going to provide some examples of why the location of the measurement matters.
Note: The majority of this discussion involves knowing the tolerance required and the standard uncertainty of the measurement process. The formula used for Standard Uncertainty is:
U = standard uncertainty
CMC = Calibration and Measurement Capability of the Laboratory Performing the Calibration, or the Reference Laboratory
K = The Coverage Factor for 95 % confidence
Res = Resolution of the device being calibrated or the UUT (Unit Under Test)
Rep = Repeatability of the Unit Under Test
Don’t worry if some of these examples are hard to follow. We have a new PFA calculator that is free for subscribers to our Morehouse Force Measurement Insider. You can sign up and download the excel file and see if your equipment passes the test. Sign up information is at the end of this blog.
Example 1 above shows an instrument within the specified upper and lower specification limits when the Uncertainty of the Measurement is considered. A good location is often the key to ensuring your devices are within tolerance. In this example, the tolerance was ± 10 lbf, and the instrument read 10,000 lbf when 10,000 lbf was applied. The standard uncertainty for the calibration was 2.89 lbf. The calibration laboratory can say the instrument passes.
Example 2 above shows an instrument outside of the upper specification limits when the Uncertainty of the Measurement is considered. A bad location is going to increase your measurement risk. The above graph shows this risk as 50 %. This means there is a 50 % whatever this instrument is used to measure, will not be “In Tolerance” In this example, the tolerance was ± 10 lbf and the instrument read 10,010 lbf when 10,000 lbf was applied. The standard uncertainty for the calibration was 2.89 lbf. The calibration laboratory cannot make a statement of compliance or pass the instrument.
These two examples are straightforward as both have the same Test Uncertainty Ratio and we have just looked at the risk of moving the location from 10,000 lbf which produced a 0.5 % risk to 10,010 lbf which produced a 0.05 % risk. In this scenario, almost any calibration laboratory is going to have to make sure the instrument being tested reads as close as possible to nominal. When calculating TUR, the resolution of the instrument at 10 lbf becomes the dominant uncertainty contributor.
What happens if we compare two calibration laboratories with two very different Calibration and Measurement Capabilities (CMC’s)?
We have a 10,000 lbf load cell with an accuracy specification of 0.05 % of full scale or ± 5 lbf. Two calibration laboratories calibrate the load cell and report the reading at 10,004 lbf. When nothing else is considered the load cell is within the manufacturer's specified tolerance. However, both laboratories are accredited and need to comply with ISO/IEC 17025. Either the old 17025 from 2005 or the new ISO/IEC 17025 standard are going to require the UOM (Uncertainty of Measurement) to be taken into account. The new version of ISO/IEC 17025 calls out a decision rule as a rule that describes how measurement uncertainty will be considered when stating conformity with a specific requirement.
Let’s look at the risk when Morehouse performs the calibration with deadweight primary standards accurate to 0.0016 % of applied force.
Example 3 Morehouse performs the calibration using deadweight primary standards accurate to 0.0016 %, or 0.16 lbf at 10,000 lbf. The specification is % 0.5 of applied force, and at 10,000 lbf this equates to ± 5 lbf. The measured value is 10,004.0 lbf, and the device has a resolution of 0.1 lbf. The repeatability was very good at 0.2 lbf as a Morehouse shear web load cell was used. The calibration laboratory can say the instrument passes.