What is uncertainty?
In ordinary use, the word ‘uncertainty’ does not inspire confidence. However, when used in a technical sense it carries a specific meaning. It is a parameter, associated with the result of a measurement (e.g. a calibration or test) that defines the range of the values that could reasonably be attributed to the measured quantity. When uncertainty is evaluated and reported in a specified way, it indicates the level of confidence that the value lies within the range defined by the uncertainty interval.
How does it arise?
Any measurement is subject to imperfections; some of these are due to external influences, such as short-term fluctuations in temperature, humidity and air-pressure, or variability in the performance of the measurer. Repeated measurements will show variation because of these factors. Other imperfections arise from the practical limitations of how correction can be made for systematic effects, such as offset of a measuring instrument, drift in its characteristics between calibrations, personal bias in reading an analogue scale, or the uncertainty of the value of a reference standard.
Why is it important?
Uncertainty is a quantitative indication of the quality of the result. It gives an answer to the question: “How well does the result represent the value of the quantity being measured?” It also allows users of the result to assess its reliability, for example, in the comparison of results from different sources, or with reference values. For this reason, confidence in the comparability of results can help to reduce barriers to trade.
Often, a result is compared with a limiting value defined in a specification or regulation. In this case, knowledge of the uncertainty shows whether the result is within the acceptable limits or not. Occasionally, a result is so close to the limit that there is a risk, once the uncertainty has been allowed for, that the measured property may not fall within the limit; this must be considered.
Suppose that a customer has the same test done in more than one laboratory, on what they may regard as an identical sample of the same product. Would we expect the laboratories to get identical results? To a degree, however, when the results are close to the specification limit, it may be that one laboratory indicates failure whereas another indicates a pass. From time to time accreditation bodies have to investigate complaints concerning such differences. This can involve much time and effort for all concerned parties, which in many cases could have been avoided if the uncertainty of the result had been known by the customer.
What is done about it?
The standard ISO/IEC 17025:2005 [General requirements for the competence of testing and calibration laboratories] specifies requirements for reporting and evaluating uncertainty of measurement. The problems presented by these requirements vary in nature and severity depending on the technical field and whether the measurement is a calibration or test.
Calibration is characterised as follows;
- Repeated measurements can be made
- Uncertainty of reference instruments is provided at each stage down the calibration chain, starting with the national standard, and
- Customers are aware of the need for a statement of uncertainty in order to ensure that the instrument meets their requirements.
Consequently, calibration laboratories are used to evaluate and report uncertainty. In accredited laboratories the uncertainty evaluation is subject to assessment by the accreditation body and is quoted on calibration certificates issued by the laboratory.
The situation in testing is not as well-developed and particular difficulties may be encountered. For example, in destructive tests the opportunity to repeat the test is limited to another sample, often at significant extra cost and with the additional uncertainty due to sample variation. Even when repeat tests are technically feasible, such an approach may be uneconomic. In some cases, a test may not be defined well enough by the standard, leading to potentially inconsistent application and thus another source of uncertainty. In many tests there will be uncertainty components that need to be evaluated on the basis of previous data and experience, in addition to those evaluated from calibration certificates and manufacturers specifications.
International and accreditation aspects
Accreditation bodies are responsible for ensuring that accredited laboratories meet the requirements of ISO/IEC 17025. The standard requires appropriate methods of analysis to be used for estimating uncertainty of measurement. These methods are based on the Guide to the expression of uncertainty of measurement, published by ISO and endorsed by the major international professional bodies. It is a weighty document and the international accreditation community has taken up its principles and, along with other bodies such as EURACHEM/CITAC, has produced simplified or more specific guidance based on them.
Accreditation bodies are harmonising their implementation of the requirements for expressing uncertainty of measurement through organisations such as the European co-operation for Accreditation (EA) and the International Laboratory Accreditation Co-operation (ILAC).
How is uncertainty evaluated?
Uncertainty is a consequence of the unknown variables and limits to corrections for systematic effects, and is therefore expressed as a quantity, i.e. an interval about the result. It is evaluated by combining a number of uncertainty components. The components are quantified either by evaluation of the results of several repeated measurements, or by estimation based on data from records, previous measurements, knowledge of the equipment and experience of the measurement.
In most cases, repeated measurement results are distributed around the average in the familiar bell-shaped curve or normal distribution, in which there is a greater probability that the value lies closer to the mean than to the extremes. The evaluation of repeated measurements is done by applying a relatively simple mathematical formula. This is derived from statistical theory and the parameter that is determined is the standard deviation.
Uncertainty components quantified by means other than repeated measurements are also expressed as standard deviations, although they may not always be characterised by the normal distribution. For example, it may only be possible to estimate that the value of a quantity that lies within bounds (upper and lower limits) with an equal probability of it lying anywhere within those bounds. This is known as a rectangular distribution. There are simple mathematical expressions to evaluate the standard deviation for this, and a number of other distributions encountered in measurement. An interesting one that is sometimes encountered, e.g. in EMC measurements, is the U-shaped distribution.
The method of combining the uncertainty components is aimed at producing a realistic rather than pessimistic combined uncertainty. This usually means working out the square root of the sum of the squares of the separate components (the root sum square method). The combined standard uncertainty may be reported as it stands (the one standard deviation level), or, usually, an expanded uncertainty is reported. This is the combined standard uncertainty multiplied by what is known as a coverage factor. The greater this factor, the larger the uncertainty interval and, correspondingly, the higher the level of confidence that the value lies within that interval. For a level of confidence of approximately 95%, a coverage factor of 2 is used. When reporting uncertainty, it is important to indicate the coverage factor, state the level of confidence, or both.
What is best practice?
Sector-specific guidance is still needed in several fields in order to enable laboratories to evaluate uncertainty consistently. Laboratories are being encouraged to evaluate uncertainty, even when reporting is not required; which will then mean they will be able to assess the quality of their own results and will be aware whether the result is close to any specified limit. The process of evaluation highlights those aspects of a test or calibration that produce the greatest uncertainty components, thus indicating where improvements could be beneficial. Conversely, it can show whether larger uncertainty contributions could be accepted from some sources without significantly increasing the overall interval. This could give the opportunity to use cheaper, less sensitive equipment or provide justification for extending calibration intervals.
Uncertainty evaluation is best done by personnel who are thoroughly familiar with the test or calibration and understand the limitations of the measuring equipment and the influences of external factors, e.g. environment. Records should be kept that show the assumptions that were made, e.g. concerning the distribution functions referred to above, and the sources of information for the estimation of component uncertainty values, e.g. calibration certificates, previous data, experience of the behavior of relevant materials.
Statements of compliance – effect of uncertainty
This is a difficult area, and when reporting, the context of the client’s needs must be considered. In particular, the possible consequences and risks associated with a result that is close to the specification limit must be examined. The uncertainty may be such as to raise considerable doubt about the reliability of pass/fail statements. When uncertainty is not taken into account, then the larger the uncertainty, the greater are the chances of passing failures and failing passes. A lower uncertainty is usually attained by using better equipment, establishing increased control of the environment, and ensuring consistent performance of the test.
For some products it may be appropriate for the user to make a judgement of compliance, based on whether the result is within the specified limits with no allowance made for uncertainty. This is often referred to as shared risk, since the end user takes some of the risk of the product not meeting specification. The implications of that risk may vary considerably. Shared risk may be acceptable in non-safety critical performance, for example the EMC characteristics of a domestic radio or TV. However, when testing a heart pacemaker or components for aerospace purposes, the user may require that the risk of the product not complying has to be negligible and would need uncertainty to be taken into account. An important aspect of shared risk is that the parties concerned agree on the acceptable uncertainty, otherwise disputes could arise.
Conclusion
Uncertainty is an unavoidable part of any measurement and it begins to matter when results are close to a specified limit. A proper evaluation of uncertainty is good professional practice and can provide laboratories and customers with valuable information about the quality and reliability of the result. Although common practice in calibration, there is some way to go with expression of uncertainty in testing; however, there is growing activity in the area and, in time, uncertainty statements will be the norm.
This main article can be found here