Talk:Interlaboratory Comparisons

1. Introduction

In this talk, information from EN ISO/IEC 17043 are selected and reported.
An interlaboratory comparison consists of organization, performance and evaluation of measurements or tests on the same or similar items by two or more laboratories in accordance with predetermined conditions.
Among the main goals of an interlaboratory comparison are:
1) evaluation of the performance of laboratories for specific tests or measurements;
2) monitoring laboratories' performance, identification of possible problems and initiation of actions for improvement;
3) establishment of the effectiveness and comparability of test or measurement methods;
4) identification of interlaboratory differences;
5) education of participating laboratories based on the outcomes of such comparisons;
6) validation of uncertainty claims;
7) evaluation of the performance characteristics of a method – often described as collaborative trials;
8) assignment of values to reference materials and assessment of their suitability for use in specific test or measurement procedures;
9) support for statements of the equivalence of measurements of National Metrology Institutes through “key comparisons” and supplementary comparisons conducted on behalf of the International Bureau of Weights and Measurement (BIPM) and associated regional metrology organizations.

Proficiency testing involves the use of interlaboratory comparisons for the determination of laboratory performance.
The need for ongoing confidence in laboratory performance is not only essential for laboratories and their customers but also for other interested parties, such as regulators, laboratory accreditation bodies and other organizations that specify requirements for laboratories.

2. Types of proficiency testing schemes

Proficiency testing schemes vary according to the needs of the sector in which they are used, the nature of the proficiency test items, the methods in use and the number of participants. However, in their simplest form, most proficiency testing schemes possess the common feature of comparison of results obtained by one laboratory with those obtained by one or more different laboratories. The nature of the test or measurement performed in proficiency testing schemes governs the method of comparing performance. There are three basic types of laboratory examinations: quantitative, qualitative and interpretive.
⎯ The results of a quantitative measurement are numerical and are reported on an interval or a ratio scale. Tests for quantitative measurement may vary in their precision, trueness, analytical sensitivity, and specificity. In quantitative proficiency testing schemes, numerical results are usually analysed statistically.
⎯ The results of qualitative tests are descriptive and reported on a categorical or ordinal scale, e.g. identity of micro-organisms, or by identification of the presence of a specific measurand (such as a drug or a grading of a characteristic). Assessment of performance by statistical analysis may not be appropriate for qualitative examinations. ⎯ In interpretive tests, the “proficiency testing item” is a test result (e.g. a descriptive morphology statement), a set of data (e.g. to determine a calibration line) or other set of information (e.g. a case study), concerning an interpretative feature of the participant's competence.

2.1 Sequential participation schemes

Sequential participation schemes involve the proficiency test item being circulated successively from one participant to the next (i.e. sequential participation), or occasionally circulated back to the proficiency testing provider for rechecking.

2.2 Simultaneous participation schemes

Simultaneous participation proficiency testing schemes usually involve randomly selected sub-samples from a source of material being distributed simultaneously to participants for concurrent testing.
In some schemes, participants are required to take samples, which are then considered to be the proficiency test items for analysis. After completion of the testing, the results are returned to the proficiency testing provider and compared with the assigned value(s) to give an indication of the performance of the individual participants and the group as a whole. Examples of proficiency test items used in this type of scheme include food, body fluids, agricultural products, water, soils, minerals and other environmental materials. In some cases, separate portions of previously established reference materials are circulated. Advice or educational comments are typically part of the report returned to participants by the proficiency testing provider with the aim of promoting improvement in performance.

3. Statistical methods for proficiency testing

Proficiency test results can appear in many forms, spanning a wide range of data types and underlying statistical distributions. The statistical methods used to analyse the results need to be appropriate for each situation. ISO 13528 describes preferred specific methods for common situations, but also states that other methods may be used as long as they are statistically valid and are fully described to participants. The methods discussed in ISO/IEC 17043:2010 cover the fundamental steps common to nearly all proficiency testing schemes, i.e.

1. determination of the assigned value;
2. calculation of performance statistics;
3. evaluation of performance;
4. preliminary determination of proficiency test item homogeneity and stability.

4. References on interlaboratory comparisons

The following referenced documents must be considered for a comprehensive view on interlaboratory comparisons.

4.1 Normative references

ISO/IEC 17043:2010, Conformity assessment - General requirements for proficiency testing.
ISO/IEC 17000:2004, Conformity assessment — Vocabulary and general principles.
ISO/IEC Guide 99:2007, International vocabulary of metrology — Basic and general concepts and associated terms (VIM)
ISO/IEC 17025:2017, General requirements for the competence of testing and calibration laboratories
ISO 13528:2015, Statistical methods for use in proficiency testing by interlaboratory comparison
ISO 21748:2017, Guidance for the use of repeatability, reproducibility and trueness estimates in measurement uncertainty evaluation

4.2 Publications

1. Metrological traceability of measurement results in chemistry: Concepts and implementation (IUPAC Technical Report) - Paul De Bièvre, René Dybkær, Aleš Fajgelj, D. Brynn Hibbert - Pure Appl. Chem., 2011, Vol. 83, No. 10, pp. 1873-1935. http://dx.doi.org/10.1351/PAC-REP-07-09-39

2. The International Harmonized Protocol for the proficiency testing of analytical chemistry laboratories (IUPAC Technical Report) - THOMPSON M., ELLISON S.L.R., WOOD R. - Pure and Applied Chemistry, Vol. 78, No. 1, pp. 145-196, 2006.

5. References on NMR interlaboratory comparisons

Help us to improve the list of publications. Please, email your suggestions to vito.gallo@poliba.it

5.1 Series

1. “Validation of a 1D 1H-NOESY experiment for fingerprinting of grape juices” - Vito Gallo, Nicola Intini, Piero Mastrorilli, Mario Latronico, Stefano Todisco, Antonino Rizzuti, Rosa Ragone, Pasquale Scapicchio, Paolo Dambruoso, Mauro Andrea Cremonini, Francesca Benevelli, Stefano Ghelli - Volume 3, “NMR Inter-Laboratory Comparisons” Series, (July 2017), ISBN: 9788885494060, Ed.: NeP Edizioni, Rome

2. “Validation of NMR fingerprinting methods: effects of processing on measure reproducibility and laboratory performance assessment” - Vito Gallo, Nicola Intini, Piero Mastrorilli, Mario Latronico, Pasquale Scapicchio, Alessandra Milella, Antonino Rizzuti, Stefano Todisco, Antonello Pascazio, Maurizio Triggiani, Vitoantonio Bevilacqua, Carlos Cobas, Silvia Mari - Volume 2, “NMR Inter-Laboratory Comparisons” Series, (September 2016), ISBN: 9788899259709, Ed.: NeP Edizioni, Rome

3. “Validation of a 1D 1H-NOESY experiment for fingerprinting of wheat and flour” - Vito Gallo, Nicola Intini, Piero Mastrorilli, Mario Latronico, Pasquale Scapicchio, Mauro Andrea Cremonini, Francesca Benevelli, Alessandra Milella, Antonino Rizzuti, Isabella Cafagna - Volume 1, “NMR Inter-Laboratory Comparisons” Series, (September 2015), ISBN: 97888999259112, Ed.: NeP Edizioni, Rome

5.2 Journals

1. “Performance assessment in fingerprinting and multi component quantitative NMR analyses” - Vito Gallo, Nicola Intini, Piero Mastrorilli, Mario Latronico, Pasquale Scapicchio, Maurizio Triggiani, Vitoantonio Bevilacqua, Paolo Fanizzi, Domenico Acquotti, Cristina Airoldi, Fabio Arnesano, Michael Assfalg, Francesca Benevelli, Davide Bertelli, Laura R. Cagliani, Luca Casadei, Flaminia Cesare Marincola, Giuseppe Colafemmina, Roberto Consonni, Cesare Cosentino, Silvia Davalli, Sandra A. De Pascali, Virginia D’Aiuto, Andrea Faccini, Roberto Gobetto, Raffaele Lamanna, Francesca Liguori, Francesco Longobardi, Domenico Mallamace, Pierluigi Mazzei, Ileana Menegazzo, Salvatore Milone, Adele Mucci, Claudia Napoli, Thelma Pertinhez, Antonino Rizzuti, Luca Rocchigiani, Elisabetta Schievano, Fabio Sciubba, Anatoly Sobolev, Leonardo Tenori, Mariacristina Valerio - Analytical Chemistry, 87 (2015) 6709-6717.

2. "Validation of quantitative NMR" - Frank Malz, Harald Jancke - Journal of Pharmaceutical and Biomedical Analysis 38 (2005) 813–823.

6. NMR proficiency test providers

Help us to improve the list of providers. Please, email your suggestions to vito.gallo@poliba.it

1. Innovative Solutions S.r.l. (Spin off company of the Polytechnic University of Bari), Italy, www.innovative-solutions.it (contact person: Vito Gallo, vitogallo@poliba.it)
2. Spectral Service AG, Germany, www.spectralservice.de, (contact person: Bernd Diehl, diehl@spectralservice.de)
3. Sigma-Aldrich Production GmbH, Switzerland, www.sigma-aldrich.com (contact person: Markus Obkircher, markus.obkircher@sial.com)