Accreditation and Quality Assurance (v.20, #5)

Qualitative PT data analysis with easy-to-interpret scores by Steffen Uhlig; Christian Bläul; Kirstin Frost; Susann Sgorzaly; Bertrand Colson; Kirsten Simon (347-353).
Numerical values called L-scores, with properties similar to those of z-scores, can be calculated for proficiency tests involving qualitative test methods. These scores do not require the use of replicates and are defined in such a way as to reflect both the Level of Competence of the Laboratory (LCL) and the Level of Difficulty of the Task (LDT). On the basis of the “logit” statistical model, the parameters corresponding to the LCL and the LDT values are determined via maximum likelihood estimation. With p denoting the corrected binary response, the model can be interpreted as $$ln left( {p/left( {1 - p} ight)} ight) = LCL - LDT.$$ ln p / 1 - p = L C L - L D T . The estimated parameters are then the quantities used to compute the L-scores for the participating laboratories. In the case of the laboratory-specific L-scores across tasks, the score for a particular laboratory can be computed as the deviation of the individual LCL from the average LCL relative to the standard error. The interpretation of the L-scores is slightly different from that of z-scores If |L| ≤ 2, the competence of the participant is not significantly different from the average competence. If L > 2, the competence is significantly higher, while if L < −2, the competence is significantly lower.
Keywords: Qualitative test method; Proficiency test (PT); Binary data; Rate of detection (ROD); Probability of detection (POD); z-score; L-score; Logit model

Introduction of z-score arrow ranges (ZSAR) in proficiency testing by Kees J. van Putten; Robert-Jan A. N. Lamers (355-358).
Most proficiency testing (PT) providers do not calculate the performance of participants that report results that are below the limit of quantification, so-called truncated results ("< results"). The ISO FDIS 13528 standard describes three approaches for the analysis of censored data. DUCARES leaves out truncated results from the statistical evaluation. Although these results are removed from the statistical dataset, it is possible, however, to give a performance indication with z-score ranges. For this, DUCARES has developed the graphical z-score arrow range (ZSAR) method which considers the full interval of potentially possible results as described by the truncated value <x reported by a laboratory, i.e., from the limit value x down to zero. The ZSAR method translates this into a z-score interval and displays it graphically with a double-headed arrow in the distribution plot. This presentation gives the participating laboratory a good indication whether its reported truncated value potentially includes satisfactory scores |z| ≤ 2 or not, and to decide on corrective actions. In this paper, we present the ZSAR method and discuss its advantages in PT.
Keywords: ZSAR method; Truncated values; Proficiency testing; ISO 13528

Honey as a material for proficiency testing by L. Ciaralli; A. C. Turco; M. Ciprotti; A. Colabucci; M. Di Gregorio; A. Sorbo (359-365).
Honey is widely analysed by many European Union control laboratories for heavy metals in foodstuffs. No maximum levels are set for cadmium and lead in this matrix, but most of the member states include this analytes–matrix combination in their National Residue Monitoring Plans setting specific action levels. The need to harmonize these values is under consideration within the European Commission. The lack of suitable certified reference materials as well as the scarcity of appropriate proficiency testings imply a difficulty for the official and National Reference Laboratories in the assessment of their method performance. The European Union Reference Laboratory for Chemical Elements in Food of Animal Origin organized the 19th PT on the determination of cadmium and lead in honey giving its network the possibility of checking the performances of the analytical methods used. An adequate procedure for the preparation of honey material spiked with cadmium and lead at values of mass fraction of interest (roughly 20 and 100 µg kg−1 for cadmium and lead, respectively) was developed. A commercial wildflower honey was spiked at proper concentration levels, achieving suitable fluidity and homogeneity by mild heating and stirring. Homogeneity and stability were investigated using an ad hoc validated analytical method and the PT material was found homogeneous and stable, covering the time span of the interlaboratory comparison. These outcomes as well as the good agreement between the assigned values and the values obtained from the homogeneity test confirmed the effectiveness of the procedure for the production of the material and the fitness of the material for the purpose of this PT.
Keywords: European Union Reference Laboratory; Cadmium; Lead; Honey; Proficiency testing material; Interlaboratory comparison

Ad hoc material for proficiency testing: freeze-dried liver by Andrea Colabucci; Alessandra Sepe; Maria Ciprotti; Laura Ciaralli (367-372).
One of the tasks of European Union Reference Laboratories (EURLs) for feed and food is the organization of proficiency testing (PTs) for the network of National Reference Laboratories (NRLs) of EU member states according to the area of competence. This crucial assignment leads the EURL for Chemical Elements in Food of Animal Origin (EURL-CEFAO) to plan PTs that include various chemical element/matrix/concentration combinations in order to organize a profitable exercise for its network. The purpose of this paper was to describe the “step-by-step” preparation of a PT material and the efforts of an EURL to provide proper test items to national laboratories (NRLs). The exercise was tailored to train participants and give them the opportunity to practice around the maximum levels (MLs) set in EU legislation, values at which sample acceptance is the critical issue. The EURL strategy has been chosen considering the role of the NRLs that could be summoned as third party in legal controversies. A PT on the determination of cadmium and lead in freeze-dried liver was then scheduled. The goal was to achieve a sample with a mass fraction (w) of the elements around the ML (0.50 mg/kg for both). Preliminary studies allowed the EURL to calculate the amount of spike to reach the target mass fractions. The target Cd and Pb mass fractions were in good agreement with the contents found in the frame of the homogeneity test as well as with the consensus values derived from the results reported by the PT participants. This proved the suitability of the production procedures and the quality of the delivered PT test items.
Keywords: EU legislation; Cadmium; Lead; Offal; Proficiency tests; European Union Reference Laboratory

Preparation of an infant formula proficiency testing material and assessment of its homogeneity and stability by A. Sorbo; M. Ciprotti; A. Colabucci; C. Zoani; M. Di Gregorio; A. C. Turco; L. Ciaralli (373-380).
The European Union Reference Laboratory for Chemical Elements in Food of Animal Origin (EURL-CEFAO) is a proficiency testing (PT) provider accredited according to ISO Guide 43-1 and later to ISO/IEC 17043:2010. It organizes exercises devoted to expert laboratories of the EU Member States. PTs are proposed taking into account matrices, analytes, maximum levels (MLs) ruled in Commission Regulation (EC) 1881/2006 (and following amendments) and discussions about new or modified MLs. Following this approach, an exercise was organized in 2012 on the measurement of the cadmium (Cd) and lead (Pb) mass fraction in powdered infant formula based on animal proteins (17th PT). One of the goals was to prepare a material containing Cd around the probable new ML so as to allow participants to test their performance at such a level. The procedure to produce the PT material was set on the basis of the EURL long experience in preparing freeze-dried samples and was tested on a preliminary batch. Homogeneity and stability of the PT material were evaluated through procedures based on international standards. For all tests, a standard deviation for proficiency assessment (σ pt), more restrictive than that from the Horwitz equation (σ H), was used. The mass fraction of Cd and Pb in the material was proven to be homogeneous and stable for 15 months. These results are extremely satisfactory taking into account the low mass fraction proposed and the restrictive standard deviation for proficiency assessment used to perform the tests. Furthermore, the PT was particularly useful considering that Commission Regulation (EC) 1881/2006 has been recently amended [CR (EU) 488/2014] with the introduction of the new ML for Cd in infant formulas consistent with the mass fraction value proposed in the 17th PT.
Keywords: European Union Reference Laboratory; Cadmium; Lead; Infant formula; Proficiency testing material; Interlaboratory comparison

The proficiency testing program of the European Union Reference Laboratory for E. coli by Clarissa Ferreri; Fabio Galati; Laura Grande; Antonella Maugliani; Valeria Michelacci; Fabio Minelli; Rosangela Tozzoli; Alfredo Caprioli (381-385).
Since 2006, the European Union Reference Laboratory for E. coli (EU-RL) has developed and evaluated standard operating procedures for the identification and typing of verocytotoxin-producing Escherichia coli (VTEC) and for their detection in food, mainly based on PCR detection of virulence genes. In particular, the EU-RL coordinated the development of the Technical Specification ISO TS 13136:2012 on the detection of VTEC in food and animal feed, based on the real-time PCR screening of food enrichment cultures. A total of 13 proficiency testing (PT) rounds were organized: 7 PTs on the identification and typing of VTEC and other pathogenic E. coli strains (detection of virulence genes by PCR and identification of the VTEC serogroups by serological and molecular methods) and 6 PTs on the detection of VTEC in different matrices (carcass swabs, milk, spinach, water, seeds and sprouts), analyzed by using the ISO TS 13136 method. A Web-based platform for submitting the PT results was developed and used since 2010. Between 2007 and 2014, a positive trend was observed in both the number of participants and their performance. Through its PT program, the EU-RL has consolidated a network of reference laboratories for the control of pathogenic E. coli in the EU Member States. Such a network can contribute to gather harmonized data on the prevalence of VTEC in food, to improve the knowledge of the epidemiology of VTEC infections in Europe, and to provide the EC with more standardized operative structures and tools to face possible emergencies in this field of food safety.
Keywords: Proficiency testing; Escherichia coli ; Verocytotoxin; Reference laboratory; Real-time PCR

Proficiency testing of wastewater sampling: What did we learn? by Magda Cotman; Albin Pintar (387-394).
The proficiency testing of wastewater sampling is organized by the Laboratory for Environmental Sciences and Engineering at the National Institute of Chemistry, Slovenia (NIC). Preliminary tests started in 2010 at two candidate sites to choose for a trial the most convenient one. In 2012, we organized the first collaborative field trial on wastewater sampling and such trials continued in 2013 and 2014. Between 16 and 20 sampling teams were selected from those that (1) participate in the national wastewater monitoring program and (2) perform analytical activities in laboratories. The participants used a 6-h time proportional sampling approach. They used sampling equipment with different principles of sample collection which take a series of discrete samples at fixed time intervals. Subsequent analyses involved both field and laboratory measurements. The participants, with only one exception, exhibited required technical skills for measuring field parameters (temperature and pH value). All samples taken were also analyzed at NIC. The variability between reported values of participants in the trials, expressed as CV, was found to be 8.5 %, 11.1 % and 9.5 % for chemical oxygen demand. Of all parameters, the largest variability was found for total organic carbon concentration amounting to 23.7 % and 35.7 %. Particular attention was given to separate the measurement uncertainty contributions arising from sampling and chemical analyses.
Keywords: Monitoring parameters; Proficiency testing; Sampling uncertainty; Scoring results; Wastewater sampling

The laboratory for precious metals within the Institute of Metrology of Bosnia and Herzegovina acts as a state reference laboratory in the field of quality control of articles manufactured of precious metals, e.g. jewellery. It disseminates reference values through the organisation of proficiency testing (PT) schemes. Two separate PT schemes for the determination of mass fraction of gold in typical jewellery alloys were conducted in 2013 and 2014. Laboratories taking part in these PT schemes are affiliated to the National Metrology Institutes of several countries and two private goldsmith laboratories. Different approaches in the design of the PT schemes and the evaluation of the PT results were used with the purpose of exploring their features regarding the interpretation of the results. For PT.13, z-scores were used as laboratory performance indicators. The assigned value and the standard deviation for proficiency assessment were calculated from the results of the participating laboratories by applying robust statistic. z-score and z′-score were used as performance indicators for the scheme PT.14, and the assigned value of the test sample and its measurement uncertainty were determined by the PT provider. The standard deviation of the testing method of 0.5 mg/g as prescribed in ISO 11426 as fit for purpose in the field of interest is used as standard deviation for proficiency assessment σ p. By comparing the z-scores and z′-scores assigned to the PT.14 results, it is evident that the measurement uncertainty of the assigned value provided by the PT provider does not affect the overall successfulness of the participants. The Q-score based on relative deviation was used as an alternative approach to identify future partners in PT schemes with the purpose of reference value assignment (Q ≤ 0.05).
Keywords: PT schemes; Assigned value; z-score; z′-score; Q-score; Gold alloy

Nitrofuran residues in food, particularly intensively farmed prawns, were identified as an important trade and health issue. No certified reference materials were available to allow laboratories to demonstrate satisfactory performance of their analytical methods. Freeze-dried prawn reference materials MX012A and MX012B were prepared at the National Measurement Institute, Australia, from blank and contaminated prawn tissue. A high-accuracy, exact-matching isotope dilution mass spectrometry method was developed for the analysis of two nitrofuran metabolites in the prawn materials. Residues were extracted by acid hydrolysis and derivatisation with 2-nitrobenzaldehyde, extracts were purified by liquid–liquid extraction followed by solid-phase extraction, and instrumental analysis was by ultra-high-pressure liquid chromatography with electrospray tandem mass spectrometry (UHPLC-ESI-MSMS). The method was used to certify the reference materials for mass fraction of 3-amino-2-oxazolidinone (AOZ) and semicarbazide (SEM), including the preparation of detailed measurement uncertainty budgets. MX012A contains AOZ at (137.5 ± 8.5) ng/g of freeze-dried material, and MX012B contains AOZ at (30.2 ± 1.8) ng/g and SEM (fortified) at (70.3 ± 3.1) ng/g. When reconstituted as directed on the certificate, the materials contain AOZ at (20.6 ± 1.3) ng/g and (4.53 ± 0.27) ng/g (MX012A and MX012B, respectively) and SEM at (10.5 ± 0.5) ng/g (MX012B). The individual components contributing to the measurement uncertainty estimates were the mass fractions assigned to calibration standards, gravimetric mass measurements, precision of the analytical method, reproducibility, long-term storage stability of the material at −20 °C, stability of the material during transportation, and potential bias due to extraction efficiency of the incurred analyte and matrix interference.
Keywords: Reference material certification; High-accuracy reference measurement procedure; Traceability; Isotope dilution mass spectrometry (IDMS)

Validation of a GC–IDMS method for the metrologically traceable quantification of selected FAMEs in biodiesel by Valle Morales; Ahmet C. Goren; Andrea Held; Mine Bilsel; Simay Gündüz; Hasibe Yilmaz (411-419).
Current methods for the analysis of fatty acid methyl esters (FAMEs) in rapeseed oil-based biodiesel refer to operationally defined measurands, which is a practical solution for routine analysis. In this paper, a new method for the SI-traceable quantification of selected FAMEs in biodiesel and its validation are described. This method has the potential to be a reference method for applications requiring structurally defined measurands and traceability to the SI as it allows direct comparisons to well-characterised calibrants through the use of isotopically labelled analogues of the analytes as well as establishing a full uncertainty budget. The method is based on gas chromatography–isotope dilution mass spectrometry. Its performance is demonstrated through its implementation and validation in two independent laboratories and is shown to provide reliable and traceable results for selected FAMEs in biodiesel test samples.
Keywords: Biofuels; Biodiesel; GC–IDMS; Traceability; Fatty acid methyl ester

Reliable measurements are needed for the verification measures of States’ declarations of their nuclear activities in line with international agreements and the EURATOM Treaty. Laboratories carrying out measurements of nuclear material need to follow stringent quality control concepts and are required to demonstrate their measurement capabilities on a regular and timely basis to legal and safeguards authorities. This includes participation in interlaboratory comparisons (ILCs). In the frame of the Regular European Interlaboratory Measurement Evaluation Programme (REIMEP), a new ILC (REIMEP-17) was jointly organized by the EC—Joint Research Centre—Institute for Reference Materials and Measurements (JRC-IRMM) and EC—Joint Research Centre—Institute for Transuranium Elements (JRC-ITU) for EURATOM and IAEA safeguards laboratories, nuclear plant operators and nuclear material laboratories. The focus in REIMEP-17 was on measurements of the uranium and plutonium amount contents and isotope amount ratios in synthetic dissolved spent nuclear fuel solutions. Participants received two test samples, REIMEP-17A and REIMEP-17B, with different uranium and plutonium amount contents. Laboratories were requested to report the results with associated uncertainties applying their standard measurement procedures and had the possibility to benchmark those results against the independent assigned (reference) values and the ones listed in the International Target Values for Measurement Uncertainties in Safeguarding Nuclear Materials (ITV2010). It can be concluded that the participants in REIMEP-17 performed well for the measurements of uranium and plutonium amount content in compliance with the respective ITV2010 values. In particular, the measurement performance for the isotope amount ratios was very satisfactory for both REIMEP-17 test samples. This confirms the measurement capabilities of laboratories in the field of nuclear material analysis and demonstrates that the stringent ITV2010 values are achievable targets under state-of-practice conditions. On the other hand, the spread of results for the minor uranium isotope amount ratios was larger. Moreover, for some of the measurands, differences in the measurement uncertainty estimations provided by laboratories were observed even when using the same instrumental technique. A summary of the participant results is presented and discussed in this paper.
Keywords: REIMEP-17; Quality control; Nuclear measurements; Interlaboratory comparison; ITV2010

Application of indicators and quality index as a tool for critical analysis and continuous improvement of laboratories accredited against ISO/IEC 17025 by Raquel Helena Catini; Fernando José Pires de Souza; Maria de Fátima Martins Pinhel; André de Oliveira Mendonça; Vitor Hugo Polisél Pacces; Igor Renato Bertoni Olivares (431-436).
In this work, a model for developing indicators and quality index was developed and applied, meaning to evaluate laboratories which have implemented the quality management system in accordance with the ISO/IEC 17025 requirements. Firstly, the chosen analysis tool (such as “brainstorming” and PDCA) to create a development of indicator and quality index model is discussed. This model was applied at the National Agricultural Research Laboratory of Sao Paulo—Laboratório Nacional Agropecuário do estado de São Paulo (Lanagro-SP). As a result, a set of indicators was developed and applied which were grouped to obtain an indicator of unique quality for ten different laboratory units evaluated. This proved to be an essential tool for critical analysis and continual improvement of the quality management system in laboratories exercising the ISO/IEC 17025.
Keywords: Indicator; Index; Quality; Continuous improvement; ISO/IEC 17025

Method comparisons are routinely performed in hospital laboratories as part of the regulatory approval process. These trials are often biased in several ways. There is often a conflict of interest bias because the hospital is paid for the trial by the manufacturer of the assay. Moreover, the trial itself is often conducted in a way that is different from routine use. The reagents selected for the trial cannot be randomly selected from all possible reagents since future reagents do not exist. Yet, future reagents often have new lots of raw materials and/or changed manufacturing procedures, which can affect assay trueness. User error is often minimized because the user chosen to perform the study receives more training then would be routinely provided. All of these factors can lead to better performance in the regulatory evaluation then performance seen after release for sale as shown by examples for glucose meters. The above biases are largely unavoidable. Steps are suggested to reduce the above biases.
Keywords: Method comparison; Clinical trial; Bias; Regulatory approval; FDA