General principles adopted by CompaLab for the organisation of ILC

The aim of this document is to outlines to participants the principles selected by CompaLab for the organisation of its ILC (inter laboratory comparisons), and the reasons for which these principles were selected.

Aim of ILC organised by CompaLab

The aim of the ILC organised by CompaLab is to assess the reliability of the test results of the participating laboratories. The reference standards for the organisation are ISO/IEC 17043 and ISO 13528.

Their aim is not to:

• Validate a test method and determine the uncertainties of results, via the determination of standard deviations of repeatability and reproducibility (reference standard for this ISO 5725-1);
• Determine the characteristics of a product to be used as reference standard (CRM, certified reference material, reference document for this ISO Guide 34 y ISO Guide 35).

The results of ILC include:

• The assessment of the bias of the results of the participant (results being too low or too high);
• The assessment of the scatter of the test results;
• A qualitative assessment of the uncertainties provided by participants;
• A computation of the uncertainties of the test results of each of the participants, using an alternate method;
• When possible, a comparison of results obtained by using different test methods.

The participation is free willing for all the participants.

Reference method for the ILC

The tests submitted to ILC shall be covered by a method similar for all participants. In most of cases, the method is described in a reference document, which is generally speaking an ISO or ASTM standard. When the test is not destructive, possibility is given for the participants to provide results obtained by several methods concerning the same characteristic (for example, chemical analysis against different methods).

Participants are requested to use their usual procedures and methods, in order to avoid:

• To generate an artificial anomaly in its results of ILC, which would be related to the use of a method that he does not control very well;
• Not to detect an anomaly in his usual results of ILC, which would be related to his implementation of his usual method;
• To generate inaccuracies in the mean values and standard deviations computed from the totality of results.

For the same reasons, participants are requested to use the parameters and tolerances given by defect in the reference standard, excepted if his quality system includes specific provisions, which then are to be applied.

When the reference document proposes several possibilities for methods and/or test parameters, regarded as comparable, but having to be reported in the test report, the participant shall then use his method by defect and report in his result sheet the method that he used for the ILC.

To enable each potential participant to check whether the methods submitted to ILC meet his needs, these methods are listed in the document of presentation of each ILC.

Characteristics submitted to tests

The characteristics submitted to tests are those described in the reference standard. They are shared into:

• Principal characteristics, for which a large number of participants are expected to provide results;
• Secondary characteristics, in most cases less important for the use of the product, and for which a significant number of participants are expected to provide no result.

Also to avoid artificial anomalies and inaccuracies, participants are not at all required to provide results for the totality of the proposed characteristics.

To enable each potential participant to check whether the characteristics submitted to ILC meet his needs, these characteristics are listed in the document of presentation of each ILC.

Products submitted to tests

The products submitted to tests are industrial products, as representative as possible of those usually tested by participants (this consequently implies that their level of internal homogeneity is not as good as those of CRM). However, each laboratory has got his own particularities with regard to his customers and of his equipment. To propose to each laboratory products exactly corresponding to those he is used to, is then impossible. The proposed programs consequently try,, as far as possible:

1. To propose a selection of products enabling a maximum number of laboratories to find levels of characteristics and type of difficulties close to those he is used to manage, even if these products may be show differences for some other respects;
2. To modify the type of products, levels and testing parameters along the years within their family (for example, for tests at high temperature, cycles at 400°C then 600°C then 800°C);
3. For products and/or tests performed by a low number of laboratories, to propose a frequency of ILC reduced to once each 3 or 5 years.

To enable each potential participant to check whether the products submitted to ILC meet his needs, these products are described in the document of presentation of each ILC.

Preparation of the test specimens

When a preparation of the test specimens is necessary:

• It has, in most cases, an important impact on the test results;
• The participants are not equipped for all the possible types and all the possible dimensions of test specimens.

Participants are then let in charge of this operation, to include this step in the assessment of the test results and make the ILC accessible for a maximum number of participants.

To enable each potential participant to check whether the provided samples are compatible with his equipment, their shape and dimensions are described in the document of presentation of each ILC.

Homogeneity

To assure the homogeneity of samples, provisions are taken concerning:

1. The preparation of samples (risk analysis of the possible causes of homogeneity defects and actions to avoid them);
2. The check of samples using appropriate non destructive tests, taking into account the intra-sample homogeneity, intra-origin bar of intra-origin strip homogeneity, inter-origin bar of inter-origin strip homogeneity;
3. When necessary, the introduction of specific directions for participants to compensate possible intra sample gradients that might be impossible to avoid (for example, gradient of hardness on a radius of a bar), especially concerning the selection of test specimens from the samples;
4. Taking into account an inter sample homogeneity standard deviation when it can be estimated from a repetition of tests on a same sample.

Statistical evaluation

Case of results which are numerical

A logarithmic transformation of the results may be applied when the distribution of the results is not symmetrical. With regard to the tests submitted to ILC, the determination of the assigned values is, in almost all cases, made by consensual value of participants. The mean value and the interlaboratory standard deviation are computed by using the A algorithm described in the ISO 13528 standard. The standard deviation of repeatability is computed by using the S algorithm described in this same standard. These algorithms are said “robust” because they show low sensitivity to the extreme values of the distribution. A prior selection of the values may be achieved on the basis of the competence of the laboratory (accredited or not), of the method (when a reference method is identified in the standard), results that can be regarded as not valid with regard to the reference standard, probable collusion between participants, gross outliers (for example unit mistakes or typing mistakes).

The evaluation of the performance is made by using z-scores, generating warning signals (limits corresponding to a probability less than 2,5%) and action signals (limits corresponding to a probability less than 0,13%). For the mean value, scores are computed with regard to Gaussian distribution of results. For standard deviations of repeatability, scores are computed with regard to a Khi2 distribution.

Cases of non numerical results

For the tests providing non numerical results, samples each other different are provided or different test conditions are requested for the tests. A scoring for each of the samples or each of the test conditions may be defined. The assessment of the non numerical results is achieved either using a transformation into Gower indexes or with regard to appropriate distribution laws, binomial or Poisson in most cases.

Uncertainties

For each participant, are achieved:

• A determination of the ζ score as described in the ISO 13528;
• A determination of standard uncertainties, computed by application of the GUM recommendations to the general model of gap between an individual test result and the “true value”, as described in ISO 5725-1;
• A qualitative comparison of the uncertainties declared by participants.

The ζ score is designed to check whether the uncertainty provided by the participant is consistent with the deviation of its results to the assigned value.

In the case of an ILC, we may consider that the deviations between any individual results of the laboratory and the assigned value include all the known and unknown contributions to the uncertainty. Hence, the uncertainty can be split into the following individual contributions:

• Uncertainty on the assigned value;
• Bias of the laboratory, representing the totality of the systematic unknown errors attached to the laboratory, its environment, its equipment, the competence of its personnel, the adequacy of its methods;
• Uncertainty on the mean value of the laboratory, caused by the finite number of determinations;
• Uncertainty related to repeatability, representing the totality of the random errors attached to the laboratory.

This approach enables to use the whole information made available by the organisation of an ILC and generally not available when a laboratory estimates its uncertainty alone. That is why it is chosen by CompaLab to provide estimations of uncertainty to each of the participants.

The collection of standard uncertainties estimated by each of the participant enables to provide a qualitative assessment of these estimations. No score was computed even if this is mathematically possible. Indeed, the ways of calculation of this parameter are too different to be comparable, and a serious risk exists to generate alerts that are irrelevant.
This qualitative evaluation enables each participant to know where it lies with regard to uncertainty (in the middle or significantly lower or significantly upper the mean value), which enables it to judge whether this is consistent with its own test conditions.

Comparison of methods

When an enough number of results is available for a specific testing method:

• The results of the participants are assessed for both this single method and for results of all methods altogether;
• The mean values and repeatability standard deviations per method and all methods altogether are provided and compared.

More details concerning: What inter laboratory comparisons are proposed

Click here to register now