Introduction to Measurement Uncertainty
1. Measurement Uncertainty
Measured values vary randomly, often without apparent cause, and hence the term ‘Random errors’. The values tend to cluster around a central value, and the best estimate of this central value is the mean or average of the measured values. However, the mean may not be the 'true' value. It may be biased away from the 'true' value by a combination of errors that are essentially constant. If their values were able to be accurately determined then they could be applied as corrections to the mean, thereby bringing it into better agreement with the true value.
Whilst precise determination is frequently not possible, there will usually be extreme limits that can be estimated or measured and that will encompass the possible value of such errors. These errors are called ‘Systematic errors’. Repeated measurements will not reduce the size of a systematic error or, in general, improve the knowledge of its magnitude or sign.
The total error of a measurement consists of at least one systematic component and at least one random component.
The ISO 17025 Standard
The ISO ‘Guide to the Expression of Uncertainty in Measurement’ is the first internationally recognised statement of the methodology for metrological specification and evaluation of measurement uncertainty. It is the result of over a decade of work by the leading international organisations in legal and technical metrology and science.
Specifically the ISO/IEC 17025 standard states in Section 5.4.5.2, Note 2, that ‘The techniques used for the determination of the performance of a method should be one of, or a combination of the following:
- Calibration using reference standards or reference materials,
- Comparison of results achieved with other methods,
- Interlaboratory comparisons,
- Systematic assessment of the factors influencing the result, and
- Assessment of the uncertainty of the results based on scientific understanding of the theoretical principles of the method and practical experience.'
Section 5.4.5.3, Note 3 states that validation is always a balance between costs, risks and technical possibilities. There are many cases in which the range and uncertainty of the values can only be given in a simplified way due to a lack of information.
Section 5.4.6.2 states that testing laboratories shall have and shall apply procedures for estimating uncertainty of measurement. In certain cases, the nature of the test method may prevent rigorous, and metrologically and statistically valid calculation of uncertainty of measurement. In these cases, the laboratory shall at least attempt to identify all the components of uncertainty and make a 'reasonable estimation', and shall ensure that the form of reporting of the result does not give a wrong impression of the uncertainty. Reasonable estimation shall be based on knowledge of the performance of the method and on the measurement scope, and shall make use of, for example, previous experience and validation data. This process is described in detail in the Eurachem/CITAC Guide 'Quantifying Uncertainty in Analytical Measurement' (See http://www.vtt.fi/kit/eurachem).
An understanding of basic statistics is also required to determine the uncertainty of measurement.
Sources of uncertainty:
(a)Incomplete definition of the measurand (analyte)
(b)Imperfect realisation of the definition of the measurand(c)Non representative sampling – the sample measured may not represent the defined measurand(d)Inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditions(e)Personal bias in reading analogue instruments(f)Finite instrument resolution or discrimination threshold(g)Inexact values of measurement standards and reference materials(h)Inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithm(i)Approximations and assumptions incorporated in the measurement method and procedure(j)Variations in repeated observations of the measurand under apparently identical conditions
Uncertainty estimation in chemical analysis
RANDOM ERRORS
The estimation of the random components may be done using a Type A analysis. This requires that the measurements be repeated a number of times. A simple manipulation of the data produces a mean value and a standard deviation. To fully account for all random sources of error, the measurement system should be disassembled and reassembled between each measurement. This is usually not practical and so it is necessary to carefully assess all possible error components and make approximate allowances.
Where multiple measurements on an instrument are not practicable, a component of uncertainty should be included to allow for short term instrument instabilities. The estimate of this component must be based on prior experience of instabilities of the particular instrument or other instruments of the same type. This is a Type B analysis of the random error contribution.
Large random variations in a measurement process should be investigated, as this may indicate that the measurement process is not under control.
SYSTEMATIC ERRORS
Systematic errors may be considered to be corrections that are not applied because their actual magnitude and sign are not precisely known. Because systematic errors are essentially constant under fixed conditions, they can be very difficult to detect. It is possible, and indeed common, for a systematic error to be many times greater than either the random effects, or the resolution and repeatability of the measurement.
A distinction should be drawn between known corrections that are not applied and systematic errors. We are concerned here with unknown errors. Corrections can and should always be applied.
In general, except for uncertainties given on calibration certificates supplied by another laboratory, systematic uncertainty components are estimated from measurements, manufacturers’ specifications and calculations based on established laws of engineering, physics and chemistry.
Operator errors are generally systematic, but may also be random in nature.
MEASURING UNCERTAINTY
The EURACHEM/CITAC Guide for Quantifying Uncertainty in Analytical Measurement (First draft April 2000) describes the following process:
(1) Specify the measurand
(2) Identify uncertainty sources:
- For example, sampling, storage conditions, instrument effects, reagent purity, assumed stoichiometry, measurement conditions, sample effects, computational effects, blank correction, operator effects and random effects.
(3) Quantify uncertainty components:
- Simplify by grouping sources covered by existing data.
- Quantify grouped components.
- Quantify remaining components.
- Convert remaining components to standard deviations.
(4) Calculate the combined uncertainty:
- Review and if necessary re-evaluate large components.
- Calculate expanded uncertainty (usually a factor of 2).
(5) Reporting uncertainty:
- Standard uncertainty, for example:
Total nitrogen 3.52 % m/m, Standard Uncertainty 0.07 % m/m
- Expanded uncertainty, for example:
Total nitrogen (3.52 + 0.14) % m/m
Reference
‘Sources of Uncertainty and Their Estimation’, NATA News, September, 1999, p4-9.
2. uncertainty estimate moisture in cocoa
2.1. INTRODUCTION
The first step described in the Eurachem Guide is to write down a clear statement of what is being measured and the parameters on which it depends.
Moisture in cocoa is determined by placing a 2 g test portion into an air oven at 100 oC and drying to constant weight (note that the measurand is an empirical quantity moisture; this is not the same as water, which is a well defined chemical entity).
This measurement process can be expressed mathematically by the equation:
This calculation could be set out on a work-sheet such as this:
Table 1: Example calculation
(g) (g) (g) (g) (g)17.017219.043818.91272.02661.89556.47%
2.2. Identification of sources of uncertainty
The next step in the Eurachem approach is to list sources of uncertainty for each part of the process or each parameter. For clarity, the measurement process is presented as a flow chart (Figure 1). The steps of the test method are detailed in the left hand column. Sources of uncertainty are highlighted in the right hand column
Figure 1: Flow chart with sources of uncertainty identified.
An alternative presentation of the measurement process is the cause and effect diagram (Ishikawa diagram). This presentation highlights the contributions to uncertainty of each component or step in the measurement process, and makes it easier to judge whether all sources of uncertainty have been accounted for, and that none have been double counted:
Figure 2: Cause and effect diagram
.
2.3. Data
Weighings
A standard mass (2.00008 ( 0.00003 g) was used to calibrate the balance. Results were as follows.
Table 2: Balance calibration
Weight (g)12.000122.000132.000142.000152.000062.000172.000182.000192.0001102.0001Mean2.0001SD0.000026
The mean weight is within the uncertainty of the calibration mass, so the balance has no bias.
The standard uncertainty in the calibration mass is 0.00003 g/(3 = 0.000017 g.
The relative standard uncertainty is 0.000017g/2g = 0.000008.
This is a very small amount and can be neglected.
Precision
Replicate measurements were made on three lots of cocoa having low, moderate and high moisture content. Results were as follows.
Table 3: Precision
Low (%moisture)Medium (%moisture)High (%moisture)12.995.017.5122.974.977.4933.005.027.5043.035.027.4953.025.027.5263.035.047.5072.964.987.5483.005.017.4993.025.017.47102.985.007.47Mean3.005.017.50SD0.0260.0200.022
The standard deviation is fairly constant over the range of moisture contents. It is appropriate to take an average of the three standard deviations as representative of the standard uncertainty of precision over the scope of the method.
So uprec = (0.026 + 0.020 + 0.022)% moisture / 3 = 0.0228% moisture.
Temperature
The temperature of the working area (middle two shelves) of the oven was measured and found to be within ( 3 oC of the set temperature. The standard uncertainty of the oven temperature is thus 3 oC / (3 = 1.732 oC.
A cocoa sample of moderate moisture content was measured with the oven set at 95 oC and at 105 oC. Results were as follows.
Table 4: Effect of temperature
95 oC
(%moisture)105 oC (%moisture)14.895.1424.895.0534.945.0944.905.1454.875.1764.965.1174.855.1184.885.1394.935.10104.855.14Mean4.895.12
So the effect of temperature is:
This quantity, 0.0224 %moisture oC-1, is an example of a sensitivity coefficient.
The standard uncertainty attributable to temperature is:
utemp = 0.0224 %Moisture oC-1 x 1.732 oC = 0.039 %Moisture.
2.4. Calculation of the combined uncertainty
Referring back to the cause and effect diagram (Figure 2), we now have the effects of weighings, temperature and precision expressed as standard uncertainties. The standard uncertainty of the weighing is small and can be neglected. The effects of temperature and precision are combined to obtain the standard uncertainty of the result.
The effects of temperature and precision are additive. So their uncertainties are combined as the square root of their standard uncertainties.
The moisture content for the example in Table 1 would be reported as:
"6.5 % ( 0.1 % moisture.
The reported uncertainty is an expanded uncertainty having a coverage factor of 2, which gives a level of confidence of approximately 95%."
Interested readers may wish to work through the process of adding in the uncertainty due to the weighings. The maths is a little more complicated because the uncertainty in weighings (umass) has units of g. In this example, umass is very small, so omitting it has no practical effect.
2.5. REFERENCE
W. Horwitz Ed. (2000) Official Methods of Analysis of AOAC International, 17th Edition. AOAC, Gaithersburg. USA, Sect 31.1.02.
Measured values vary randomly, often without apparent cause, and hence the term ‘Random errors’. The values tend to cluster around a central value, and the best estimate of this central value is the mean or average of the measured values. However, the mean may not be the 'true' value. It may be biased away from the 'true' value by a combination of errors that are essentially constant. If their values were able to be accurately determined then they could be applied as corrections to the mean, thereby bringing it into better agreement with the true value.
Whilst precise determination is frequently not possible, there will usually be extreme limits that can be estimated or measured and that will encompass the possible value of such errors. These errors are called ‘Systematic errors’. Repeated measurements will not reduce the size of a systematic error or, in general, improve the knowledge of its magnitude or sign.
The total error of a measurement consists of at least one systematic component and at least one random component.
The ISO 17025 Standard
The ISO ‘Guide to the Expression of Uncertainty in Measurement’ is the first internationally recognised statement of the methodology for metrological specification and evaluation of measurement uncertainty. It is the result of over a decade of work by the leading international organisations in legal and technical metrology and science.
Specifically the ISO/IEC 17025 standard states in Section 5.4.5.2, Note 2, that ‘The techniques used for the determination of the performance of a method should be one of, or a combination of the following:
- Calibration using reference standards or reference materials,
- Comparison of results achieved with other methods,
- Interlaboratory comparisons,
- Systematic assessment of the factors influencing the result, and
- Assessment of the uncertainty of the results based on scientific understanding of the theoretical principles of the method and practical experience.'
Section 5.4.5.3, Note 3 states that validation is always a balance between costs, risks and technical possibilities. There are many cases in which the range and uncertainty of the values can only be given in a simplified way due to a lack of information.
Section 5.4.6.2 states that testing laboratories shall have and shall apply procedures for estimating uncertainty of measurement. In certain cases, the nature of the test method may prevent rigorous, and metrologically and statistically valid calculation of uncertainty of measurement. In these cases, the laboratory shall at least attempt to identify all the components of uncertainty and make a 'reasonable estimation', and shall ensure that the form of reporting of the result does not give a wrong impression of the uncertainty. Reasonable estimation shall be based on knowledge of the performance of the method and on the measurement scope, and shall make use of, for example, previous experience and validation data. This process is described in detail in the Eurachem/CITAC Guide 'Quantifying Uncertainty in Analytical Measurement' (See http://www.vtt.fi/kit/eurachem).
An understanding of basic statistics is also required to determine the uncertainty of measurement.
Sources of uncertainty:
(a)Incomplete definition of the measurand (analyte)
(b)Imperfect realisation of the definition of the measurand(c)Non representative sampling – the sample measured may not represent the defined measurand(d)Inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditions(e)Personal bias in reading analogue instruments(f)Finite instrument resolution or discrimination threshold(g)Inexact values of measurement standards and reference materials(h)Inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithm(i)Approximations and assumptions incorporated in the measurement method and procedure(j)Variations in repeated observations of the measurand under apparently identical conditions
Uncertainty estimation in chemical analysis
RANDOM ERRORS
The estimation of the random components may be done using a Type A analysis. This requires that the measurements be repeated a number of times. A simple manipulation of the data produces a mean value and a standard deviation. To fully account for all random sources of error, the measurement system should be disassembled and reassembled between each measurement. This is usually not practical and so it is necessary to carefully assess all possible error components and make approximate allowances.
Where multiple measurements on an instrument are not practicable, a component of uncertainty should be included to allow for short term instrument instabilities. The estimate of this component must be based on prior experience of instabilities of the particular instrument or other instruments of the same type. This is a Type B analysis of the random error contribution.
Large random variations in a measurement process should be investigated, as this may indicate that the measurement process is not under control.
SYSTEMATIC ERRORS
Systematic errors may be considered to be corrections that are not applied because their actual magnitude and sign are not precisely known. Because systematic errors are essentially constant under fixed conditions, they can be very difficult to detect. It is possible, and indeed common, for a systematic error to be many times greater than either the random effects, or the resolution and repeatability of the measurement.
A distinction should be drawn between known corrections that are not applied and systematic errors. We are concerned here with unknown errors. Corrections can and should always be applied.
In general, except for uncertainties given on calibration certificates supplied by another laboratory, systematic uncertainty components are estimated from measurements, manufacturers’ specifications and calculations based on established laws of engineering, physics and chemistry.
Operator errors are generally systematic, but may also be random in nature.
MEASURING UNCERTAINTY
The EURACHEM/CITAC Guide for Quantifying Uncertainty in Analytical Measurement (First draft April 2000) describes the following process:
(1) Specify the measurand
(2) Identify uncertainty sources:
- For example, sampling, storage conditions, instrument effects, reagent purity, assumed stoichiometry, measurement conditions, sample effects, computational effects, blank correction, operator effects and random effects.
(3) Quantify uncertainty components:
- Simplify by grouping sources covered by existing data.
- Quantify grouped components.
- Quantify remaining components.
- Convert remaining components to standard deviations.
(4) Calculate the combined uncertainty:
- Review and if necessary re-evaluate large components.
- Calculate expanded uncertainty (usually a factor of 2).
(5) Reporting uncertainty:
- Standard uncertainty, for example:
Total nitrogen 3.52 % m/m, Standard Uncertainty 0.07 % m/m
- Expanded uncertainty, for example:
Total nitrogen (3.52 + 0.14) % m/m
Reference
‘Sources of Uncertainty and Their Estimation’, NATA News, September, 1999, p4-9.
2. uncertainty estimate moisture in cocoa
2.1. INTRODUCTION
The first step described in the Eurachem Guide is to write down a clear statement of what is being measured and the parameters on which it depends.
Moisture in cocoa is determined by placing a 2 g test portion into an air oven at 100 oC and drying to constant weight (note that the measurand is an empirical quantity moisture; this is not the same as water, which is a well defined chemical entity).
This measurement process can be expressed mathematically by the equation:
This calculation could be set out on a work-sheet such as this:
Table 1: Example calculation
(g) (g) (g) (g) (g)17.017219.043818.91272.02661.89556.47%
2.2. Identification of sources of uncertainty
The next step in the Eurachem approach is to list sources of uncertainty for each part of the process or each parameter. For clarity, the measurement process is presented as a flow chart (Figure 1). The steps of the test method are detailed in the left hand column. Sources of uncertainty are highlighted in the right hand column
Figure 1: Flow chart with sources of uncertainty identified.
An alternative presentation of the measurement process is the cause and effect diagram (Ishikawa diagram). This presentation highlights the contributions to uncertainty of each component or step in the measurement process, and makes it easier to judge whether all sources of uncertainty have been accounted for, and that none have been double counted:
Figure 2: Cause and effect diagram
.
2.3. Data
Weighings
A standard mass (2.00008 ( 0.00003 g) was used to calibrate the balance. Results were as follows.
Table 2: Balance calibration
Weight (g)12.000122.000132.000142.000152.000062.000172.000182.000192.0001102.0001Mean2.0001SD0.000026
The mean weight is within the uncertainty of the calibration mass, so the balance has no bias.
The standard uncertainty in the calibration mass is 0.00003 g/(3 = 0.000017 g.
The relative standard uncertainty is 0.000017g/2g = 0.000008.
This is a very small amount and can be neglected.
Precision
Replicate measurements were made on three lots of cocoa having low, moderate and high moisture content. Results were as follows.
Table 3: Precision
Low (%moisture)Medium (%moisture)High (%moisture)12.995.017.5122.974.977.4933.005.027.5043.035.027.4953.025.027.5263.035.047.5072.964.987.5483.005.017.4993.025.017.47102.985.007.47Mean3.005.017.50SD0.0260.0200.022
The standard deviation is fairly constant over the range of moisture contents. It is appropriate to take an average of the three standard deviations as representative of the standard uncertainty of precision over the scope of the method.
So uprec = (0.026 + 0.020 + 0.022)% moisture / 3 = 0.0228% moisture.
Temperature
The temperature of the working area (middle two shelves) of the oven was measured and found to be within ( 3 oC of the set temperature. The standard uncertainty of the oven temperature is thus 3 oC / (3 = 1.732 oC.
A cocoa sample of moderate moisture content was measured with the oven set at 95 oC and at 105 oC. Results were as follows.
Table 4: Effect of temperature
95 oC
(%moisture)105 oC (%moisture)14.895.1424.895.0534.945.0944.905.1454.875.1764.965.1174.855.1184.885.1394.935.10104.855.14Mean4.895.12
So the effect of temperature is:
This quantity, 0.0224 %moisture oC-1, is an example of a sensitivity coefficient.
The standard uncertainty attributable to temperature is:
utemp = 0.0224 %Moisture oC-1 x 1.732 oC = 0.039 %Moisture.
2.4. Calculation of the combined uncertainty
Referring back to the cause and effect diagram (Figure 2), we now have the effects of weighings, temperature and precision expressed as standard uncertainties. The standard uncertainty of the weighing is small and can be neglected. The effects of temperature and precision are combined to obtain the standard uncertainty of the result.
The effects of temperature and precision are additive. So their uncertainties are combined as the square root of their standard uncertainties.
The moisture content for the example in Table 1 would be reported as:
"6.5 % ( 0.1 % moisture.
The reported uncertainty is an expanded uncertainty having a coverage factor of 2, which gives a level of confidence of approximately 95%."
Interested readers may wish to work through the process of adding in the uncertainty due to the weighings. The maths is a little more complicated because the uncertainty in weighings (umass) has units of g. In this example, umass is very small, so omitting it has no practical effect.
2.5. REFERENCE
W. Horwitz Ed. (2000) Official Methods of Analysis of AOAC International, 17th Edition. AOAC, Gaithersburg. USA, Sect 31.1.02.
Komentar