In section 4.1 we created a table of key instruments for our measurements that included columns for instrument, purpose, make, model number, range, and uncertainty. Today we dive into the details of instrument uncertainty and traceability.
Many times knowing the precision/uncertainty of your measurement is just as, if not more, important than the value of the measurement itself. The question is: how can you quantify your confidence in the measurement? Two kinds of error will affect your instruments: Precision error and bias error.
For the most part, we can resolve precision error with statistics. We’ll cover that next time. So how do we minimize bias error to the greatest extent possible?
Calibration: determining and documenting the deviation of a measuring instruments indication from the conventional ‘true’ value.
Sure we can calibrate our instruments so that we know they are accurate. But how do we know the calibrations are correct? What value is ‘true’ and how can we trust it?
Traceability: process whereby the indication of a measuring instrument (or a material measure) can be compared, in one or more stages, with a national standard for the measurement in question.
In the United States national standards are maintained and improved by the National Institute of Standards and Technology (NIST). NIST may easily be the most under-appreciated federal agency as a NIST standard is used for nearly every custody exchange in our economy. Formerly known as the National Bureau of Standards (NBS), the organization was mandated by congress after the civil war to standardize the exchange of sugar as a commodity. As it turns out, it’s not easy to standardize the purity of white powdery substance for custody exchange. A full history of NIST and the NBS can be found here. I’ve worked with NIST researchers for my entire career, and spent several summers at NIST-Boulder. It’s a wonderful place!
So if NIST keeps the standards, how do our instruments get compared to this standard? Here’s a diagram of traceability levels established by the European Cooperation for Accreditation of Laboratories (EAL-G12):
This traceability pyramid above is a handy way of understanding the problem of traceability. The wider the base, the more instruments in the world that exist in that level of the pyramid and the cheaper, or lower quality the measurement calibration is. For example THE kilogram block that all other kilograms used to be based on, was kept in a safe under very carefully controlled conditions, I call this the very top of the pyramid, or level 0. Level 0 is used to create the primary standards sent around the world for each country’s National Standards, level 1, that are implemented by the national standard body. Level 2 is a reference standard that is created from the primary standard and distributed to accredited calibration laboratories around the country. These in turn create workplace/factory standards, Level 3, that calibrate the instruments used by the workplace/factory for actually producing things at Level 4. Here’s an example of a length measurement standard applied to the traceability pyramid, again from EAL-G12.
As you can see, an ultimate micrometer or dial gauge isn’t necessarily kept in a safe in Switzerland to calibrate all other dial gauges like in the case of the kilogram. The actual type of instrument can change depending on standard level (0,1,2,3,4). This makes sense as the level of precision goes up significantly with standard level, in many cases you just need different physical paradigms to achieve this. Here’s a handy graphic showing how in late 2018 the SI governing committee moved away from physical standards and to universal physical constants for realizing the SI unit system. A full description of the redefinition is here.
As you can see, the process of traceability can quickly become complex! As the table above shows, most standards bodies will provide a certificate of traceability to describe this chain of calibration on a single convenient document. Here’s an example of such a certificate of traceability for the microphones in the anechoic chamber:
There is an incredible amount of information on this document. Key points include the actual instruments, including serial numbers used in the unbroken chain going back to the primary standard. The dates of calibrations, including environmental conditions are provided, along with the calibration curve itself. In the fine print you’ll also see the obligatory, “whose accuracies are traceable to the National Institute of Standards and Technology.” As you can guess, this document is valuable, sometimes more valuable than the instrument itself. Hang onto these in a safe place!
But even with this document things are not fine and dandy. Is it current enough? How do you know the instrument was not abused before you started use? These are complex questions that there isn’t currently a standard for.
A few years ago I proposed a Standards Traceability Index (STI) to go along with the education of traceability. The STI has three numbers: X.XX
X. — This is the standard level (from the pyramid above) for the instrument that you are using (almost everything in our lab is level 4, although the calibration instruments are level 3).
.X — This number describes the status of the instrument’s traceability. For example, 0 is used for a current unbroken chain, 1 for a current but broken chain, 2 for an outdated unbroken chain, 3 for an outdated broken chain, 4 for no certificate chain.
._X — This number describes the status/condition of the instrument. For example, 0 is brand new and performance is validated, 1 is used by performance is validated, 2 is new but non-validated, 3 is used and non-validated, 4 is for unknown condition.
So for example, a brand new platinum resistance thermometer with a NIST traceable calibration that you just dunked in a liquid nitrogen bath to check would likely have an STI = 4.00.
Using the STI for this class is likely overkill. But if you need to do a very good job, it’s good to think about. In general you should include a statement on the traceability for all your instruments when assessing your measurement confidence/uncertainty. Better yet, check the calibrations once you’re finished with one of our laboratory calibration standards. If you’re uncertain in your results, than how valuable are they?
Traceability is also important for use with property and flow correlations. It’s sadly very common for people to reference a software package as the source of their property data. In reality though the software is just implementing an equation that someone spent a lot of time on. Reference the original equations, not the software. Many people end up embarrassed when they cite some fancy software package, only to find out it’s using the ideal-gas law in a non-ideal situation.
At the minimum, the table of your instruments should include a discussion, specific to each instrument, on the currency and traceability of the calibration, and a justification for the degree it matters.