Determination Risk Level Associated to Decision of Conformity to Specification for Environmental Laboratories
Before 2017, there aren’t standards obligated analytical laboratories even accredited to take measurement uncertainty into account in the decision for acceptation or rejection the samples result to a specification, so the decision of conformity for all results closed to specification limits became doubtable. Now in the recent edition of ISO Standard (ISO/IEC 17025:2017) decision rules become obligatory for both testing and calibration laboratories, a clause number 7.8.3 titled “Reporting statement of conformity†stated the laboratories shall document the decision-rule employed and taking into account the risk level. The environmental measurements are critical and mistakes occur in decision lead to a disaster, so this paper concerned to explain different methods of decision-rules used and calculate the risk level relevant to each decision through the practical implementation these decisions by Kafe EL Seikh accredited water laboratory.
Introduction
Previous version of ISO standard (ISO/IEC 17025:2005) recommended only to testing laboratories to write a statement of conformity where necessary for the interpretation of the test results. Statement of conformity not obligatory in this version because this version didn’t confirm the implementation in terms of the shall and should but it was mentioned as where relevant (ISO 17025:2005, Clause 5.10.3.1b) [1]. and there was lake in references that explain that, especially for testing laboratories.
On the other hand, recent ISO Standard (ISO/ IEC:17025/2017) obligated both testing and calibration laboratories to write the statement of conformity in case of requested by the customers or the competent authority and mentioned that at different positions such as resources and processes related to personnel, contract review and reporting.
Clause 3.7, illustrated the definition of the decision- rules with conformity, which is taking the uncertainty of measurement into account to stat the conformity with a specified requirement [2]. Clause 6.2.6 obligate the laboratory to authorized personnel for results analyses including the writing of the conformity statement, opinion and interpretation. Clause 7.1.3 requires when the customer needs lab to write a statement of conformity to a specification or standard a binary decision shall be taken ( pass/fail, in- tolerance/out-of- tolerance), the specification or standard and the decision-rule shall be defined unambiguously in test report or in calibration certificate unless mentioned clearly in the requested specification or standard, the decision-rule selected shall be discussed , and agreed with, the customer before the lab initiated customer requests [2, 3, 4]. Clause 7.8.3. (1c) where applicable, the uncertainty of measurement written in the same unit as absolute value or a relative percentage to the measurand, when the value of uncertainty affects the validity of the test results, when requested by a customer, or when the uncertainty of measurement affects statement of conformity to specifications. Clause 7.8.6.1 When a statement of conformity to a specification or standard for test or calibration is required, the laboratory shall mentioned the decision-rule used also the risk level (like false of acceptation and false of rejection and statistical assumptions used) which related to the decision-rule used [2] .Clause 7.8.6.2 the laboratory shall report the statement of conformity clearly to identify which results that statement of conformity implemented; which specifications and requirements are compiled or not; the decision-rule applied (unless it is stated in the specification or standard used) [2].
Some analytical laboratories working in the environmental field, acceptance and rejection decisions are inaccurate due to they haven’t taken uncertainty in their decisions due to lack in statistical knowledge. So in this paper we explain different methods of decision-rules used and associated risk levels with a practical example for each method.
Methods for Determination Risk Levels of Decision Rules
When performing a measurement and subsequently making a statement of conformity, there are two possible outcomes; correct decision regarding to specification or incorrect decision is regarding to specification [5].
As shown in Figure 1 The expanded measurement uncertainty in the lower result (A) totally lies within the tolerance limit. The upper result (B) has significantly larger measurement uncertainty. The risk of falsely accepting a result in case B is higher due to the larger measurement uncertainty, so using of a guard bands (W) can reduce the probability of making an incorrect conformance decision. A safety factor built into the measurement decision to reduce the acceptance limit below that of the specification/ tolerance limit [5]. There are four cases of decision-rules that often encountered to the workers in field of environmental laboratories; simple acceptance, upper tolerance limits, lower tolerance limits and double-sided tolerance limit [6, 7].
Figure 2a: Guard band for lower tolerance limit decision.
Figure 2b: Guard band for lower tolerance limit decision.
Figure 2c: Guard band for double side tolerance limit decision, (1) Tolerance/ specification Zone, (2) Acceptance Limit, (3) Acceptance Zone, (4) Rejection Zone, (5) Guard band(W) for lower Tolerance Limit, (6) Guard band(W) for Upper Tolerance Limit, (LTL) Lower Tolerance/ Specification Limit and (UTL) Upper Tolerance/ Specification Limit.
This document refers to Guard Bands, where the Guard Band (W) equal the Tolerance/specification Limit (TL) minus the Acceptance Limit (AL) W = TL-AL Eq. (1) This means that if the measurement result is below the Acceptance Limit (AL), the measurement is accepted as conforming to specification. If measurement exceed Acceptance Limit (AL) by the amount less than specific risks provided in the following table are also accepted as conforming to specification, reject otherwise [8, 9].
| Decision rule | Guard Band W(r*U) | Specific risk | Conformance probability (pc) | |
|---|---|---|---|---|
| 1 | 6 Sigma | 3U | < 1ppm PFA | ≥ 99.99999% |
| 2 | 3sigma | 1.5U | <0.16 PFA | ≥ 99.84% |
| 3 | ILAC G8:2009 rule | 1U | <2.5% PFA | ≥ 97.5% |
| 4 | ISO 14253-1:2017 [5] | 0.83U | <5% PFA | ≥ 95.0% |
| 5 | Simple acceptance | 0 | <50%PFA | ≥ 50% |
| 6 | Uncritical | -U | Item reject for measured value greater than AL=TL+U <2.5 PFR | ≥ 97.5% |
| 7 | Customer defined | r U | Customers may define arbitrary multiple of r to have applied as guard band | Defined by customers |
Table 1: Specific risk of decision rules used in environmental laboratories. PFA-Probability of False Accept and PFR- Probability
Case (1): Simple Acceptance
Customers ask a laboratory to make a conformity decision that “ignores uncertainty”. In case of uncertainty ignorance, the guard band length equal to zero, W = 0, infers that acceptance occur when a measurement result is below a tolerance limit. This is called simple acceptance or shared risk because the probability to be outside the tolerance limit may be as high as 50% (risk level is 50%) in the case when a measurement result is exactly on the tolerance limit (assuming a symmetric normal distribution of the measurements) [10]. According to the accreditation requirements this decision is not permitted by ISO/IEC 17025:2017, which require that uncertainty should be taken into account (directly or indirectly) when conformity decisions are made [11].
Case (2): Single Sided Upper Tolerance Limit
Conformance probability for an upper tolerance limit is therefore pc= 100- NORM. DIST (TL, ym, SD, TRUE) Eq. (2) Where: TL is specification limit, ym is lab result, SD standard deviation which equal standard uncertainty(u) of the result with the same unit * factor specified in different types of decision rules (r) Table (1).
Probability false acceptance PFA (risk level) is PFA = 100 – pc Eq. (3)
Possible decision rules for this conformity decision might therefore be defined in terms of conformance probability(pc) or probability false acceptance (PFA) [11, 12].
Example: What is the lab decision of compliance with the WHO specification for measuring iron in drinking water sample, while the sample result is 0.28 mg/l, WHO specification limit is 0.3 mg/l, expanded uncertainty associated with measuring of iron in this lab is ±8%, lab using ILAC-G8 decision-rule Table (1).
Answer: Decision-rule lab used is ILAC-G8, where guard band length equal 1* Expanded uncertainty(U) and the accept risk level is <2.5% so the result ACCEPT when 𝑝𝑐 ≥ 97.5 %; REJECT otherwise or equivalently, also the result ACCEPT when 𝑃𝐹𝐴 < 2.5 %; REJECT otherwise” Frist of all calculate standard uncertainty(u) with the same unit of measurand by applying the following equation [9, 10, 11].
( ) ( ) ( )
(u) 2 m measured value y Expanded uncertinity Standard uncertinity u K average value K × = = Eq (4) Standard uncertinity (u)=±0.0112mg/l Standard deviation (SD)= r*u=1*0.0112=±0.0112 mg/l By applying the values of: specification limit TL, lab result 𝑦𝑚, Standard deviation SD in Conformance probability (pc) Eq. (2), using Excel program pc = 100-NORM. DIST (0.3, 0.28, 0.0112, TRUE) =96.29% Calculation probability false acceptance using Eq. (3).
PFA = 100 – pc = 3.71% This result might therefore be reported as:
“REJECT, with a conformance probability of only 96.29% which does not meet acceptance criterion of 𝑝c ≥ 97.5%” or “REJECT with a probability false acceptance of 3.71%, unable to meet acceptance criterion of PFA < 2.5%”
Case (3): Single Sided Lower Tolerance Limit
Conformance probability(pc) for lower tolerance limit is therefore [11, 12].
pc = NORM. DIST (TL, ym, SD, TRUE)Eq. (6)
Probability false acceptance PFA (risk level) is PFA = 100 – PFA Eq. (7)
Example: Dissolved oxygen(O2) of treated sewage plant was 4.2 mg/L as O2, specification limit lab follows to determine statement of conformance is O2 not less than 4 mg/l, expanded uncertainty associated with O2 measuring is ±6% and the decisions rule lab always uses is ISO 14253- 1:2017[5] Table (1).
Answer: Decision-rule which lab use is ISO 14253- 1:2017[5] where guard band length equal 0.83* expanded uncertainty(U) and the accept risk level is < 5.0% so the result ACCEPT when 𝑝𝑐 ≥ 95.0 %; REJECT otherwise or equivalently, also the result ACCEPT when 𝑃𝐹𝐴 < 5.0 %; REJECT otherwise Calculate standard uncertainty(u) as the previous example u= ± 0.126 mg/l Standard deviation (SD)= 0.83 * u= 0.83 * 0.126= ±0.104 mg/l pc = NORM. DIST (4, 4.2, 0.104, TRUE) = 97.21% PFA = 2.79% This result might therefore be reported as: “Accepted, with a conformance probability (𝑝c) of only 97.21% which meet acceptance criterion of 𝑝c ≥ 95.0%” or “accepted with a probability false acceptance of 2.79%, meet acceptance criterion of PFA < 5.0%”
Case (4): Double Sided Tolerance Limit
Conformance probability for a two-sided tolerance limit is therefore. pc = NORM. DIST (TL(max), ym, SD, TRUE) − NORM. DIST (T(min), ym, SD, TRUE) Eq. (8) PFR= 100-Pc Eq. (9) Decision Rules for this conformity might therefore be defined in terms of 𝑝c or 𝑃𝐹𝐴 [12, 13, 14]. Example: Dissolved oxygen(O2) of a river was 2.2 mg/L as O2, dissolved oxygen needed for fish living ranges from 2 - 6 mg/l, expanded uncertainty associated with O2 measurement is ±6% and the decisions rule lab always uses is 3 Sigma Table (1).
Answer: Decision-rule which lab used is 3 Sigma where guard band length equal. 1.5* expanded uncertainty(U) and the accept risk level is <0.16% so the result ACCEPT when 𝑝𝑐 ≥ 99.84 %; REJECT otherwise or equivalently, also the result ACCEPT when 𝑃𝐹𝐴 < 0.16 %; REJECT otherwise” Calculate standard uncertainty as the previous examples u= ± 0.066 mg/l Standard deviation (SD) = 1.5 * u= 1.5 * 0.07= ±0.10 mg/l pc = NORM. DIST (6, 2.2, 0.10, TRUE) −NORM. DIST (2, 2.2, 0.10, TRUE) = 97.83% PFR= 2.17% This result might therefore be reported as: “REJECT, with a conformance probability of only 97.83% which does not meet acceptance criterion of 𝑝c ≥ 99.84%” or “REJECT with a probability false acceptance of 2.17%, unable to meet acceptance criterion of PFA < 0.16 %”
Conclusion
Laboratories working in the environment field must accurately determine the uncertainty values associated with the tests results due to its extreme importance in the different decision-rules and also, they must apply decision- rules methods whether or not they are internationally accredited unless it is inherent in the requested specification or standard.
Highlights
This paper concerned to explain different methods of decision-rules used and calculate the risk level relevant to each decision through the practical implementation these decisions by Kafe EL Seikh accredited water laboratory.
References
-
Ricci U (2014) Establishment of an ISO 17025:2005 accredited forensic genetics laboratory in Italy, Accred Qual Assur 19: 289-299.
-
(2017) ISO/IEC 17025 General requirements for the competence of testing and calibration laboratories. International Organization for Standardization/ International Electro-Technical Committee, Geneva.
-
(2001) ASME B89.7.3.1 (2001) Guidelines for Decision Rules: Considering Measurement Uncertainty in Determining Conformance to Specifications. The American Society of Mechanical Engineers.
-
(2003) ISO 10576-1 (2003) Statistical methods- Guidelines for the evaluation of conformity with specified requirements.
-
(2019) ILAC-G8:09 (2019) Guidelines on Decision Rules and Statements of Conformity.
-
(2017) BS EN ISO 14253-1 (2017) Geometrical product specifications (GPS)-Inspection by measurement of workpieces and measuring equipment Part 1: Decision rules for verifying conformity or nonconformity with specifications.
-
UKAS (2019) M3003, The Expression of Uncertainty and Confidence in Measurement, 4th (Edn.), United Kingdom Accreditation Service.
-
UKAS (2020) LAB 48, Decision Rules and Statements of Conformity, 2nd (Edn.), United Kingdom Accreditation Service.
-
(2012) ISO/IEC Guide 98-4, Uncertainty of measurement-part 4: role of measurement uncertainty in conformity assessment. International Organization for Standardization, Geneva.
-
(2008) ISO/IEC Guide 98-3, Guide to the expression of uncertainty in measurement.
-
ASME (2005) B89.7.4.1, Measurement Uncertainty and Conformance Testing: Risk Analysis. American Society of Mechanical Engineers.
-
Tsimillis KC (2018) Measurement uncertainty: requirements set in the accreditation standards, Accredit Qual Assur 23(253): 109-114.
-
Ricci U (2014) Establishment of an ISO 17025:2005 accredited forensic genetics laboratory in Italy. Accred Qual Assur 19: 289-299.
-
Silva GMP, Nogueira R, Faria ACO (2014) the lead assessor role in the ISO/IEC 17025:2005 accreditation of Brazilian calibration and testing laboratories by the General Coordination of Accreditation (Cgcre), Accred Qual Assur 19: 127-132.
- Genetic Improvement of Nile Tilapia (Oreochromis niloticus): Advances in Selective Breeding and Genomic Approaches for Sustainable Aquaculture
- Microplastics, Contaminants, and Waste Hotspots: Divergences and Faults in Prioritizing Control Efforts
- Creating a Healthier, More Vibrant Open and Closed Aquatic Environment. A Submersible, Centrifugal Magnetically Affixed Current Changing Aquarium Pump
- An Attempt to Assess Alpha Diversity and Sample Size: Using the Ostracod Assemblages off Kumamoto Port, Japan
- Assessment of the Efficiency of Common Fishing Gears and Crafts Used at Mohananda River of Chapai Nawabganj, Bangladesh
- Fish Productivity and Biodiversity Status of Sundarban Mangrove in Bangladesh