APRIL 1979 The llnalyst Vol. 104 No. 1237 Approach for Achieving Comparable Analytical Results from a Number of Laboratories A. L. Wilson Water Research Centre Henley Road Medmenham Marlow Buckinghamshire SL7 2HD In the field of water analysis growing importance is being attached to the ability to compare with confidence the results from different laboratories. However the errors in the results can invalidate such comparisons. One approach to ensuring results of adequate accuracy is described in this paper. Examples of its successful application to river-water analysis within the UK will be presented in subsequent papers. Keywords Water analysis ; accuracy of results ; inter-laboratory comparability ; analytical quality control 1 Introduction Increasing importance is being attached to the measurement and control of the quality of many types of water.This in turn has led to a growing need to control the accuracy* of analytical results so that valid conclusions can be drawn when results are compared either with each other or with water-quality standards. Such control generally presents problems because of the many factors that can adversely affect analytical accuracy. In the author’s experience a critical and well co-ordinated approach to such problems is essential if there is to be any chance of reliable achievement of a specified accuracy in routine analysis. The purpose of this paper is therefore two-fold (i) to describe the approach that is being used within the UK to ensure adequately accurate results for the Harmonised Monitoring of River Water Quality3; and (ii) to introduce a series of papers on the application of the approach to and the results obtained for a series of determinandst within that scheme.Hence this paper is concerned with a topic (analytical quality control) about which much has already been written. However to the author’s knowledge there has been little or no integrated discussion of all aspects of the topic relevant to water analysis although other publications5-12 deal with one or more of the important aspects. It is hoped therefore, that the present account of a complete analytical quality control scheme will be of general interest. The scheme described below was evolved about 15 years ago for a survey of feed-water quality in power stations of the Central Electricity Generating Board.The successful applications to river and other waters suggest that the approach may be of general value. It is interesting therefore to note that the scheme is very similar in fundamental concepts and practical aspects to an excellent and authoritative approach recently recommended for clinical chemistry.l 2 Approach to Analytical Quality Control 2.1 General Concepts The approach consists for each determinand of sequential completion of a number of individual but closely linked stages as summarised in Fig. 1. These activities and their sequence have been chosen so that generally important sources of error are eliminated progressively or controlled at adequately small values. In this way a permanently sound * Accuracy is used here with the sense of “total error” (h.the sum of random and systematic errors); accuracy is said to improve as the total error decreases. The total error of a result is the difference between the result and the “true value,” it being assumed here that a true value exists for every sample.’#* t Determinand is used here with the sense* “that which is to be determined.” 27 274 WILSON APPROACH FOR ACHIEVING COMPARABLE Analyst VoZ. 104 basis for control is ensured with a minimum chance of wasted effort. It is important, therefore that no stage is started until the preceding stage has been completed satisfactorily. The reasons for and activities involved in the individual stages are described in the following sections. Activity I 1 Establish working group I Define determinands and required accuracy Purpose To plan and co-ordinate all subsequent activities.To ensure clear specification of analytical requirements. Choose analytical methods To select methods capable of the required accuracy. Ensure unambiguous r- description of methods + Estimate within-laboratory precision To ensure that the chosen methods are properly followed. To ensure that each laboratory achieves adequate precision. Ensure accuracy of standard solutions t Set up qua1 ity-control charts + To eliminate this source of bias in each laboratory. To maintain continuing check of precision in each laboratory. Check between-laboratory bias To ensure that each laboratory achieves adequately small bias tests to be repeated a t regular intervals to main-tain a continuing check on bias.Fig. 1. Sequence of activities for analytical quality control. The detailed considerations and work necessary in each stage are governed by the particular determinand and the required accuracy. The aim here therefore has been to stress the principles involved ; subsequent papers will give the information relevant to particular applications. Emphasis has also been placed on analytical aspects and no attempt has been made to explain and give full computational details of all the statistical tests involved. Each such test is however stated and details can be obtained either from the references quoted or from general statistical texts. I t is also worth stating that many of the points made below are not novel.Nevertheless their inclusion is considered necessary because they are sometimes overlooked and more importantly they should be seen in the context of an integrated approach to analytical quality control. In particular as the literature has dealt with the early stages of Fig. 1 to a smaller extent than the latter the former are treated here in greater detail especially as they fonn the basis of all subsequent experimental work. One aspect of the scheme in Fig. 1 should be stressed at the outset i.e. everything possibl April 1979 ANALYTICAL RESULTS FROM A NUMBER O F LABORATORIES 275 is done to eliminate sources of error before making tests of between-laboratory bias. There are two main reasons for this. Firstly it is often difficult in water analysis to obtain a direct experimental estimate of this bias for all of the many types of sample usually analysed by laboratories.Secondly if such a bias is found considerable difficulty is often experienced in eliminating it particularly when the laboratories are widely dispersed geographically. 2.2 Establishment of a Working Group To ensure an efficient and uniform approach in all laboratories throughout the stages of Fig. 1 thorough co-ordination of all the work is necessary. At least one laboratory (see below) with the effort and resources necessary to provide advice and to plan and co-ordinate the work is essential; this laboratory is referred to as the co-ordinating laboratory. The effectiveness of any control scheme rests ultimately on the competence of individual laboratories.Hence the maximum possible understanding of all aspects of the work should be sought in all laboratories. This point is of special importance because the statistical procedures required in estimating errors may sometimes be unfamiliar to analysts. Under-standing and efficiency tend to be improved when laboratories are parties to decisions rather than having particular approaches imposed on them. For these and other reasons it is of value to establish a Working Group to plan and agree both the general approach and also the detailed procedures for each determinand. Whenever possible this Group should be composed of a representative from each of the participating and co-ordinating laboratories; other interested organisations may of course also be represented.In addition to the particular purposes mentioned above such Working Groups also act as a useful means of exchange of analytical information. The number of laboratories and/or their geographical distribution may sometimes be so large that it is impracticable for one co-ordinating laboratory to deal with all laboratories. When this is so a hierarchical scheme is useful i.e. the laboratories are divided into a number of groups each of which can be adequately serviced by a co-ordinating laboratory. All the co-ordinating laboratories then form one level in the hierarchy and they follow the scheme in Fig. 1. On satisfactory completion of all stages of the work for a particular determinand each co-ordinating laboratory then initiates an identical approach within its own group of laboratories.2.3 Definition of Determinands and Required Accuracy Unambiguous definition of the determinands of interest89ll and numerical definition of the accuracy required of the analytical results are essential for three main purposes in any control scheme (i) to allow the choice of analytical methods appropriate to the intended uses of the results; (ii) to provide clear criteria for deciding whether or not the errors observed in particular laboratories warrant corrective action ; and (iii) to ensure that adequate numbers of tests are made to allow errors of the magnitudes of interest to be detected as statistically significant. The first task of the Working Group is to ensure that these definitions of the requirements are available because they will form the basis of almost all subsequent work.Much could be written on their formulation but it suffices here to mention that they often appear to be accorded little or no attention and to summarise certain generally important points in sect ions 2.3.1-2.3.6. 2.3.1 results rather than by those who provide them. of course joint discussions are generally essential.8 Responsibility f o r definition of required determinands and accuracy Point (i) above implies that the requirements should be defined by those who will use the In principle this is correct but in practice, 2.3.2 Dejnition of determinands Many of the substances of interest in water analysis can exist in a variety of chemical and physical forms to each of which a given analytical method often responds to different extents.I t follows that an unambiguous definition of determinands is essential so that appropriate analytical methods can be selected. Examples of this and related points are discussed elsewhere.sJ 276 WILSON APPROACH FOR ACHIEVING COMPARABLE Analyst VoZ. 104 Many determinands in water analysis are non-specific,13 i.e. they do not correspond to one or more particular chemical species (chloride iron etc.) but rather express certain properties of samples e.g. colour biochemical oxygen demand which are governed by undefined species. In general the true values for such determinands in any sample are defined by the analytical methods used for their measurement. Therefore the determinands can be defined unambiguously only by defining a particular method to be used (see also section 2.4).The same approach may also be necessary for certain determinands that cannot be measured analytically by available methods e.g. “dissolved” iron. In this instance as a complete separation of dissolved and undissolved forms is not generally possible the procedures used for this separation can have markedly different efficiencies. Hence the separation procedure (e.g. the filtration system) may need to be specified as a means of defining the determinand. 2.3.3 Sources of error Inaccurate results can be caused by errors occurring during sampling (e.g. contamination), between sampling and analysis (e.g. through instability of the determinand) and during analysis.* In water analysis the first two sources can be very important for many deter-minands if appropriate precautions are not taken.Clearly attention must be paid to such errors in addition to those occurring during analysis. However control of the errors before analysis commonly involves different factors than those relevant to analysis. Further, analysts are often not directly involved with sampling. For these reasons the author favours two separate but closely linked approaches to the control of sampling and analytical errors and this is used in the Harmonised Monitoring Scheme. A similar suggestion has been made in the context of quality control in clinical chemistry.l However procedures suitable for ensuring constancy of the determinand concentrations between sampling and analysis (e.g. the addition of a chemical preserving reagent) can affect the performance of analytical methods ; close co-ordination of these aspects is therefore essential.On the above basis the remainder of this paper is concerned primarily with the control of errors arising during analysis. A subsequent paper will consider the other sources of error. 2.3.4 Method of exj5ressing the required accuracy This can be done in a number of ways and each scheme should choose the most appropriate for its purposes. However the approach described below is thought to be of general value in environmental monitoring and in similar applications where the determinand concentrations may cover a relatively wide range. For each determinand the smallest concentration? of interest is decided and this is equated to the limit of detection L required of the analytical method.The limit of detection is defined statistically1*J5 with the assumption that the random errors of analytical results follow a normal distribution. The maximum tolerable total error of an analytical result for any sample is then defined numerically by a statement in the form “The maximum tolerable total error of a result is 9% of the concentration of the determinand or L mg 1-1, whichever is the greater.” This method of expression recognises that relative errors increase markedly as the concentration of a deterrninand approaches the limit of detection. Appropriate numerical values of 9 and L are chosen for each determinand and scheme. For the Harmonised Monitoring Scheme p is 20 for most but not all determinands; the value of L varies with the determinand.3y11 Of course simpler expressions are possible when only a narrow range of determinand concentrations occurs or is of interest.Finally there is the question of how to express the required accuracy. 2.3.5 Dejnition of tolerable random and systematic errors It is also important to define separately the random and systematic errors that can be tolerated because they have different effects on the validity of results and because they are estimated in different ways. For the Harmonised Monitoring Scheme the random error * Errors arising through clerical errors and/or through the use of different units in reporting results, though important are not considered here. t Certain determinands are not expressed in concentration units.For simplicity here the term con-centration is used throughout April 1979 ANALYTICAL RESULTS FROM A NUMBER OF LABORATORIES 277 of a result is taken as twice the standard deviation of individual results and the maximum tolerable total error is divided equally between random and systematic errors. This division of total error between random and systematic errors is arbitrary but experience suggests it is often useful. Other divisions may be used if desired but experimental problems in checking bias can arise as the ratio of the tolerable random and systematic errors is increased (see section 2.9.3). Thus numerical values for the maximum tolerable standard deviation and bias of analytical results can be deduced for any concentration of determinand. For example the maximum tolerable standard deviation is 0.259% of the concentration of the determinand or 0.25L whichever is the greater.It is of interest to note that recent German proposals9 on monitoring drinking water have recommended maximum tolerable standard deviations for a number of determinands. For each determinand only one fixed value is given but the recommendations are concerned mainly with narrow concentration ranges around values specified in water-quality standards. The use of the above target values for random (precision) and systematic (bias) error is described in the following sections. 2.3.6 Provisos to the approach in sections 2.3.1-2.3.5 In the above approach a few points should be noted. (i) There is a slight numerical inconsistency between the values of L and the maximum tolerable standard deviation at low concentrations.Thus if results are normally distributed with a standard deviation uR independent of concentration at low concentrations the limit of detection is given by 3.290 (probabilities of errors of the first and second kinds1*J5 are both 0.05). Hence at low concentrations the maximum tolerable standard deviation (specified as 0.25L) is 0.82aR i.e. it is smaller than the standard deviation achieved aR. This inconsistency could be eliminated for example by selecting slightly different probability levels to define the limit of detection. However such complication is considered unnecessary because of the uncertainties involved in estimating uR and the fact that the targets for errors should not usually be regarded as precisely fixed constants.If the target values are interpreted with common sense no difficulty is caused by the above inconsistency. (ii) The limit of detection may not be directly relevant to certain deterrninands e.g. pH value. When this is so the expressions for tolerable errors will usually need to specify only fixed percentage or absolute errors. (iii) As mentioned above the numerical values of p and L should be chosen appropriately for each scheme. However as those values are reduced the analytical effort and cost needed to achieve them will generally increase and at some point will become impracticably large (see also section 2.9.3). Care is required therefore in specifying errors that are both tolerable and achievable. The target values in the Harmonised Monitoring Scheme may seem rather large but experience of the approach in this paper suggests that they are both useful and realistic.I t is interesting that McFarren et aZ.,16 in considering the results from many inter-laboratory studies of different analytical methods concluded that for only a relatively small fraction of methods studied were total errors* of 25% or less achieved. (iu) In specifying a particular method in order to define a non-specific determinand care is also necessary to ensure that the method selected is capable of the required accuracy and limit of detection. 2.4 Choice of Analytical Methods This is the most important stage of all because an inappropriate choice will not only prevent achievement of the required accuracy but is also likely to cause much wasted time and effort.Accordingly the choice of methods is considered in some detail below. It sometimes appears to be thought that comparable results can simply be achieved by ensuring that all laboratories use the same method for a particular determinand. However, innumerable experimental studies have shown that this approach does not necessarily control the errors of all laboratories; see for example the compilation of inter-laboratory tests given by McFarren et aZ.16 In addition use of the same method by all laboratories penalises * The definition of “total error” used by McFarren et al. is similar to but not identical with that in this paper; see also reference 17 278 WILSON APPROACH FOR ACHIEVING COMPARABLE Analyst VoZ. 104 those able to take advantage of analytical advances e g .by using new methods or instru-ments. This approach is not therefore gene]-ally recommended except when required by the definition of a determinand (see section 2.3.2). In principle the approach recommended here is simple in that each laboratory merely has to select a method capable of the required accuracy. Of course other factors such a< the speed and simplicity of a method are frequently important and should be consider d carefully when choosing a method from those capable of the required accuracy. In applying this approach several aspects need careful consideration and those of general importance are summarised below. 2.4.1 Definition of determinands by specifying particular analytical method The methods to be used for certain deternninands may already have been defined (see preceding section).When this is so there is in principle no choice to be made and it is necessary only to ensure that all laboratories follow essentially the same procedure (see section 2.5). This approach has been used in the Harmonised Monitoring Scheme for suspended solids and biochemical oxygen demand. It is also possible to allow the use of alternative methods for a particular determinand if they are first shown to give results in satisfactory agreement with the specified method. This latter approach is basically sound, but can lead to problems because of the experimental difficulties involved in proving satis-factory agreement particularly when the samples of interest are of markedly differing com posit ions.2.4.2 For any determinand other than those in section 2.4.1 the method should be selected from among those known to be capable of the required performance and for which a thorough investigation of the effects of relevant experimental parameters has been made.5918J9 If only one such method exists it is technically desirable for all laboratories to use that method, but see section 2.4.4. When several suitable methods are available for a particular deter-minand each laboratory is free to choose the one that it prefers. In making that choice it is useful to select methods capable of rather smaller errors than the maximum tolerable errors defined above. This will tend to ensure a safety margin and to reduce the need for frequent corrective action to reduce intolerably large errors.Particular attention should be paid to sources of bias when selecting a niethod5,11J8120 because in many situations the choice of a suitable method is the main control on bias in the analytical results. Notwith-standing the specified tolerable bias it is recommended that whenever possible the aim should be to select methods with negligible sources of bias. Determinands that are defined chemical sjhecies 2.4.3 In comparing the published performances of analytical methods with the required limit of detection and maximum tolerable standard deviation and bias care is necessary because the definitions of these characteristics and the methods used to estimate and report them often differ from one publication to another.1.8-22 This potential problem is readily elimi-nated if authors make clear the definitions and procedures relevant to their quoted perform-ance characteristics and if they give unambiguous information on the magnitudes of errors.Within the UK one particular approach to the definition and tabulation of performance characteristics has been developed by the Central Electricity Research Laboratories.5Jg-22 This approach has been followed by the Water Research Centre and recently by a committee* concerned with the selection evaluation and publication of methods for the analysis of a wide range of waters effluents and other materials.23 An example of such a tabulation is given in reference 24. Essentially the same approach is followed in the Harmonised Moni-toring Scheme. In comparing published and required values of standard deviations and limits of detection, careful consideration of the sensitivity and discrimination of the analytical-measurement system is also essential.For example a published method may have used an instrument Comparing the required and Published performances of analytical methods * The Standing Committee of Analysts to Review Standard Methods for Quality Control of the Water Cycle established by the Department of the Environment and the National Water Council April 1979 ANALYTICAL RESULTS FROM A NUMBER OF LABORATORIES 279 of better discrimination (ability to make finer readings of the instrument scale) than that available in another laboratory. If instrumental errors govern precision the latter labora-tory will be unable to achieve the published precision.Thus in selecting methods each laboratory should ensure that the sensitivity and discrimination of its preferred method will allow the required precision to be achieved. 2.4.4 Availability of suitable analytical methods The approach in section 2.4.2 cannot always be followed in the Hannonised Monitoring Scheme for two reasons. Firstly thoroughly evaluated and characterised methods are not at present available for all determinands. In the absence of such methods there is much to be said for delaying attempts to achieve comparable results but the need for information on water quality commonly requires some attempt to control analytical errors. Fortunately, this problem is gradually being eliminated by the work of the many laboratories and organisa-tions concerned with the development of analytical methods.Secondly the staff and/or equipment of a laboratory may necessitate the use of a method whose capability of achieving the required accuracy is at least partially in doubt. Therefore the use of methods not known to be suitable requires consideration. If a laboratory has to select an analytical method that has not been thoroughly evaluated, critical assessment of its likely errors is d e ~ i r a b l e . ~ * ~ ~ s ~ ~ Certain sources of bias can some-times be readily eliminated e.g. in the blank and calibration pro~edures.~~~*~~0 Other likely sources of bias and the method's ability to achieve the desired precision should then be assessed. To aid uniformity among laboratories in such appraisals it is useful for the co-ordinating laboratory to make similar assessments of all such methods; this approach is followed for all determinands in the Harmonised Monitoring Scheme.Whenever possible, each laboratory concerned should then make any preliminary tests needed to estimate the unquantified errors.195 This seldom presents great difficulty if only the precision is in doubt. However if as is common little is known of interference effects,ll the task is much more difficult because of the large amount of work needed for a reasonably thorough investigation of such errors,20 particularly for samples whose composition is as variable as river waters, Other sources of bias20 may also not be simple to determine. Such uncertainties and the effort needed to resolve them are a powerful argument for the approach in section 2.4.2.In the present context however experience shows that routine laboratories cannot usually make all the required tests. Hence all that can be done in the short term is for a laboratory to make as many tests as possible and then to proceed to subsequent stages of Fig. 1; the latter stages may detect certain sources of bias e.g. in the tests described in section 2.9. In the longer term the aim should be either to complete any tests on errors or to replace the method with one whose performance is better established. 2.4.5 Importance of sample preservation 9rocedures As mentioned above stability of the concentration of a determinand between sampling and analysis must be ensured but the procedures used to that end can affect the performance of a subsequent analytical method.It is desirable therefore to select only methods that include as an integral part of the total procedure all aspects relevant to sample preservation; e.g. details of sample containers and preserving reagents. 2.5 Written Descriptions of Analytical Methods Virtually all publications concerned with analytical quality control stress the importance of detailed written descriptions of analytical methods. In addition to the clear need for such descriptions in each participating laboratory they are also essential in assessing likely sources of error (see section 2.4.4). There are many possible formats that such descriptions can take and an international standard is a~ailable.~5 It suffices here to suggest that the aim should be to specify the entire experimental procedure in such detail that if it were faithfully followed an inexperi-enced analyst would be able reliably to achieve adequate accuracy.Relevant points have been reviewed by many workers; see for example references 18 and 19. The type of format favoured by the author is illustrated in references 5 and 24 280 WILSON APPROACH FOR ACHIEVING COMPARABLE AnaZyst VoZ. 104 It is also essential to do everything possible to ensure that the method is followed exactly in all subsequent work at least until tests have demonstrated the validity of any contem-plated changes. In considering such changes great caution is generally desirable ; even apparently minor procedural details can sometimes have unsuspectedly large effects particu-larly in trace analysis.One of the aims of subsequent stages of Fig. 1 is to detect any deterioration in the accuracy of routine analytical results caused by unsuspected changes in procedure. 2.6 Ensuring Adequate Within-laboratory Precision Three main considerations lead to this and the three subsequent stages of Fig. 1. (i) Many factors other than the written description of a method can affect the precision of analytical results. Such factors include the purities of reagents the reliabilities of instru-ments the abilities of analysts and the degree of contamination problems. Therefore the fact that one or more laboratories have achieved adequate precision when using a given method does not guarantee this for another laboratory. Experimental estimates of precision should therefore be obtained by each laboratory.(ii) Direct tests of between-laboratory bias are usually not easy to arrange on a frequent basis. There is an advantage therefore in attempting to control as many sources of error as possible by tests that can be made independently by each laboratory. (iii) The number of replicate tests needed to detect bias of a given magnitude is governed by the standard deviation of results.16p26 It i!j useful therefore to ensure that all labora-tories achieve adequately small standard deviations before making tests of bias. On this basis estimation of within-laboratory precision by all laboratories is the first of the experimental stages in Fig. 1. 2.6.1 In deciding an appropriate experimental design for such tests several desiderata are generally relevant.1s6s21 (i) Precision commonly depends on the concentration of the determinand.Therefore, except when interest is attached only to a narrow concentration range,9 estimates of standard deviation should be obtained for at least two concentrations.21 Further whenever the limit of detection is of interest the within-batch standard deviation of blank determinations should usually be e ~ t i m a t e d . ~ ~ s ~ 5 $ ~ ~ (ii) Precision commonly worsens as the time period over which the tests are made is increased. As it is the precision of routine analytical results that is of interest the tests should be spread over a number of occasions rather than being made on only one occasion. (iii) Precision may depend on the nature oi the sample analysed and in particular real samples (e.g.river waters) may give worse precision than standard solutions. However, standard solutions are of general value in analytical quality contr01~~~J1 and tests of precision €or both standards and real samples are therefore useful (see section 2.8). (iv) Precision is assessed from replicate results for a given sample; these results should be independent of each other.21 The determinand concentration in the sample must also be the same on each occasion it is analysed. (v) As the number of tests and samples is increased more information is obtained and the uncertainties of the estimated standard deviations tend to decrease. This has generally to be balanced against the amount of effort required and available for the tests.Many experimental designs for estimating precision have been described and any may be used provided valid estimates of the required information are obtained. The design normally followed in the Harmonised Monitoring Scheme aims to meet the above de~iderata,~,~l and is summarised in section 2.6.2; experience to date suggests the design to be of general value. Important factors to be considered in designing tests of precision 2.6.2 Typical experimental design In each of m batches of analyses (no more than one batch in any one day) n portions of each of the following are analysed in random order (i) blank determination; (ii) standard solution of concentration 0.1 C (C = upper concentration limit of the analytical method); (iii April 1979 ANALYTICAL RESULTS FROM A NUMBER OF LABORATORIES 281 standard solution of concentration 0.9 C,; (iv) a sample of river water; and (v) the same as (iv) but with an accurately known addition (equivalent to 0.5 C,) of the determinand.All analyses are made exactly as for normal samples by the same experienced analyst. The values of rn and n are usually 10 and 2 respectively but other values can be used.5921 Special consideration of the value for n may be needed if the analytical method specifies analysis of more than one portion of a sample and/or blank in order to produce an analytical result for a sample. The value n = 2 mentioned above applies to the usual situation where only one portion of a sample and a blank is specified in the method. Other details are given elsewhere.5~~1 Precautions are necessary when the determinand is unstable and these are summarised in the Appendix.2.6.3 At the end of the tests for each of the solutions (ii)-(v) the 20 analytical results are analysed ~tatistically~,~~ to obtain estimates of the within-batch (sw) between-batch (sb) and total (st) standard deviations where st2 = sw2 + sb2. The results for the blanks are used to provide an estimate of its within-batch standard deviation (in concentration units). For each of the solutions (ii)-(v) the value of st is compared (variance-ratio test) with the maximum tolerable standard deviation at the corresponding concentration of the deter-minand. The precision is then accepted as adequate if st is not significantly greater (0.05 probability level) than the target value (see Appendix).Similarly the limit of detection can be calculated from the value of sw for the blank and then compared with the target value (see Appendix). If all targets are achieved it can be concluded (but see the Appendix) that the precision is adequate. If one or more targets are not achieved sources of imprecision should be sought reduced and the tests then repeated. When this is necessary the values of sw and s b for a particular solution may help to indicate the likely source of e r r ~ r ~ ~ ~ and this is one of the main reasons for choosing n = 2 i.e. this allows separate estimations of sw s b and st. When n = 1 only st can be estimated. In addition to the above estimates of precision the recovery of the determinand added to solution (v) is also calculated.This recovery will generally differ from 100yo even in the absence of bias as a result of random errors and a criterion is therefore needed to decide whether or not the observed recovery is to be considered as adequate. In the Harmonised Monitoring Scheme this criterion is that the recovery should not differ significantly (t-test, 0.05 probability level) from the range 95-105%. Again if the results are unsatisfactory, the cause should be sought reduced and the tests then repeated. Finally the results from solutions (i)-(iii) allow the analytical sensitivity* to be com-pared with that expected for the method. Other aspects such as the linearity of the cali-bration graph should also be checked. If such parameters depart appreciably from those expected for the method it is as well to seek and if necessary eliminate the reasons for such differences as they may indicate mis-application of the method or other problems.When all the above aspects can be regarded as satisfactory the next stage in Fig. 1 (section 2.7) is started. Calculations and com9arison of required and achieved performances 2.6.4 Co-ordination of tests To ensure that all laboratories carry out the agreed tests arid calculations uniformly the co-ordinating laboratory should prepare (i) detailed descriptions of all aspects of the tests for each determinand and (ii) suitably designed forms on which laboratories can enter the results from the tests. These forms can also and in the Harmonised Monitoring Scheme do, include detailed instructions on how to make the various statistical calculations involved, Experience shows that if such details and forms are not supplied to all laboratories some participants may make inappropriate tests and calculations and this may invalidate their work.The co-ordinating laboratory is also likely to meet problems in attempting to check and collate all results if they have not been calculated and presented in a uniform format. One general point on these and subsequent tests in laboratories is worth noting here. * The term sensitivity is used here to denote the rate of change of the analytical response with deter-Thus when the response is directly proportional to concentration the sensitivity minand concentration. is the slope of the calibration graph 282 WILSON APPROACH FOR ACHIEVING COMPARABLE Analyst VoZ.104 2.7 Accuracy of Standard Solutions This stage is intended to ensure that the standard solutions (used for calibration purposes) of all laboratories are in satisfactory agreement and hence do not cause important between-laboratory bias. In principle this aim could be achieved by the co-ordinating laboratory providing the necessary standard solutions for all laboratories. In practice however this approach is not favoured because it is cumbersome liable to practical difficulties and very demanding of effort from the co-ordinating laboratory. A better approach and one which is used in the Harmonised Monitoring Scheme is as follows. All laboratories prepare their standards as prescribed in their analytical methods and the co-ordinating laboratory distributes a portion of its own concentrated standard solution for the determinand under study to each laboratory.Of course considerable care is essential to ensure that the distributed standard has a negligible error. In water analysis particular care is often required to ensure that the concentration of the determinand in the distributed solution is stable for an adequate time period and that the containers used for the standard do not cause contamination. Such problems are usually minimised by distributing relatively concentrated standard solutions ; this also aids transport arrangements because relatively small volumes of such solutions suffice for the subsequent tests. Each laboratory then compares the concentrations of its own standard with the distributed standard in the following way.Both standard solutions are diluted accurate1.y to the same concentration this concentra-tion being that for which the tests in section 2.6.2 have indicated the smallest within-batch relative standard deviation*; this concentration is usually at or near C,. Then (I portions of each diluted standard solution and one blank determination are analysed in one batch of analysis in a prescribed order? and the mean results for each standard compared using a t-test. The value of q is calculated by each laboratory to allow the experiment to have the desired statistical power of detecting a difference of d% between the two standards.26 In the Harmonised Monitoring Scheme d = 2 and the probabilities of errors of the first and second kinds are both set at 0.05.If any laboratory finds that its standard solution is in error by more than d% the cause is sought and reduced and the tests are repeated. When a laboratory’s standard is in satisfactory agreement with the distributed standard the next stage in Fig. 1 (section 2.8) is started. 2.8 Analytical Quality-Control Charts Even though a laboratory has satisfactorily completed the two preceding stages this by no means guarantees that its accuracy will remain permanently satisfactory. It is essential, therefore for each laboratory to maintain continuing checks that its errors remain adequately small. A widely recommended and used is to make special control tests in each batch of the normal analyses and to plot the results immediately they are obtained on statistical quality-control charts.If the results from the control tests indicate that the accuracy has worsened appreciably further analysis of samples is stopped until the source of the increased error has been found and eliminated. This use of quality-control charts is recommended here and is followed in the Harmonised Monitoring Scheme. Several important aspects in using such charts for analytical quality control are summarised in sections 2.8.1-2.8.6. It will be seen that various uncertainties arise and this emphasises the desirability of laboratories seeking to obtain accuracy rather better than that expressed by the maxi-mum tolerable errors. 2.8.1 Basis of q.uality-control charts For those who may be unfamiliar with the concepts underlying statistical quality-control charts most general statistical texts deal with the topic; see for example references 27 and 28.The use of these charts in analysis has also been described by many worker~.5~6,*,12,29 * The relative standard deviation signifies here th,e standard deviation for a particular concentration t A random order is usually suitable but systematic orders may be preferable if the analytical response expressed as a percentage of that concentration. can drift continually in one direction April 1979 ANALYTICAL RESULTS FROM A NUMBER OF LABORATORIES 283 2.8.2 Choice of type of control test For example if a standard solution is analysed in each batch of analyses this gives information on precision and certain sources of bias but clearly gives no direct information on the accuracy of sample analysis.Information on the precision for samples can be obtained by analysing two or more portions of a sample in each batch but this gives no information on bias. Recovery tests using samples may be useful in providing information on certain though not all sources of bias but are likely to present problems in assessing precision. Ideally all these and any other relevant control tests would be made routinely but a compromise with the effort required is usually necessary. Each situation should therefore be assessed individually to decide the most appropriate control test(s) for a given determinand. This has been dis-cussed in more detail elsewhere,30 and the results from the preceding tests of within-laboratory precision are of value in deciding which control tests to use.For most determinands the single best control test is usually considered to be the analysis of a standard solution particu-larly when the initial tests of precision have shown essentially the same precision for standard solutions and samples. Of course when this control is used it is essential to ensure that the concentration of the standard solution has negligible error on each occasion it is analysed. Another factor frequently relevant to the choice of the control test(s) is the extent to which precision depends on the concentration of the determinand. If for example the standard deviation of results increases markedly with concentration there is some problem in deciding the best single concentration to use for a control standard.Of course if only a narrow range of concentrations exists or is of interest the mid-point of that range will usually be a suitable value for a control standard. Otherwise it is preferable to use two control standards one near the lower limit of the concentration range of the method and the second near the upper limit. If the effort necessary for both standards is not available the use of the upper concentration is on balance favoured. It should be noted that problems also arise in the construction and interpretation of control charts for the two other types of control test mentioned in the preceding paragraph whenever precision depends on concentra-tion and the concentrations in samples cover a wide range.30 Various devices can be used to reduce such problems.For example one control chart for each of a number of relatively narrow concentration ranges can be used for each type of control test the results being plotted on the appropriate chart. In general no single control test can check all possible sources of error. 2 3.3 Frequency distribution followed by results To allow exact interpretation of the results of control tests the nature of the probability distribution followed by the results must be known; it is convenient and common to assume the normal distribution. To ensure that the results of control tests closely follow this distribution statistical texts usually recommend that for example the control test is replicated in each batch and the mean of the results is plotted as the control point for each batch.This approach is desirable but again the effort available may only allow single control tests in a batch. When this is so it is still of value to use control charts but it must be accepted that the control charts cannot be exactly interpreted without further information on the nature of the probability distribution of results. In practice this is not considered an important objection. 2.8.4 Types of quality-control chart In the Harmonised Monitoring Scheme the results from control tests are plotted on Shewhart-type control ~ h a r t s . ~ ~ - ~ ~ In recent years another type of chart the CUSUM quality-control chart has come into common use and is said to be advantageous in allowing more rapid detection of changes of a c ~ u r a c y . ~ ~ ~ ~ ~ This latter chart has been recommended for use in clinical chemistryl~~~ and has been included in one manual on analytical quality control in water analysk6 An investigation of the use of CUSUM charts in water analysis is to be made in the author’s laboratory.At present however it is considered that there is little to choose between the two types of chart for analytical quality control given the various uncertainties involved in the practical use of control charts 284 WILSON APPROACH FOR ACHIEVING COMPARABLE Analyst Vol. 104 2.8.5 Before a control chart can be constructed an estimate of the standard deviation of the results of control tests is required to allow insertion of the warning and action limits. Ideally this estimate would have a large number of degrees of but insistence on this would delay construction of the chart for a relatively long period while the necessary tests were being made.In the Harmonised Monitoring Scheme therefore a preliminary control chart is constructed as soon as a 1abora.tory’s standard solution has been shown to be adequately accurate. The estimates of standard deviation obtained from the tests of precision are used for this purpose. Such estimates have at least 9 and usually more, degrees of freedom (see the Appendix) but revised estimates should be obtained (and the warning and action limits on the chart adjusted accordingly) as the results from control tests accumulate.30 Particular care is needed in using the chart initially but it is of value for laboratories to have a sequential record of the control tests from the outset especially as an appreciable time period may elapse before the next stage (section 2.9) of Fig.1. Construction of Shewhart-type quality-control charts 2.8.6 Anonymity of control solutions There is value but commonly difficulty in arranging that the analysts concerned are unaware of which solutions are control^.^ In the Harmonised Monitoring Scheme individual laboratories decide whether or not to adopt this approach. The importance of this aspect is much reduced if all analytical staff are brought into a quality-control scheme properly, with full understanding of its purposes. 2.9 Checking Between-laboratory Bias As soon as all laboratories have satisfactorily completed the previous stages this final stage is begun. Portions of one or more standard solutions and/or samples are distributed to each laboratory which then makes sufficient replicate determinations on each solution to allow detection of a bias equal to the maximum tolerable value.The bias is assessed separately for each solution because both bias and precision may depend on the concentration of the determinand and on the nature and general composition of the solution. Other approaches to between-laboratory tests have been described which may allow some economy of effort or a greater amount of information for the same effort ; such approaches have been briefly reviewed.’ However these advantages are gained by making assumptions such as equal precision in all laboratories. For present purposes it is considered essential not to make such assumptions and the approach described below is followed.The co-ordinating laboratory prepares and distributes appropriate solutions to all labora-tories; only a broad indication of the concentrations of these solutions is given to the laboratories. Each laboratory then analyses w portions of each solution the normal control test@) (see section 2.8) being included in each batch of analysis. The results obtained are reported to the co-ordinating laboratory for assessment of the bias for each laboratory and solution. Any laboratories with an unacceptably large bias are informed so that they can seek and reduce the cause before undertaking further tests of bias. Further details relevant to this approach are given below. The approach adopted is very simple.2.9.1 Solutions to be used The main sources of bias needing to be checked are those arising from the analysis of samples rather than standard solutions. Emphasis should whenever possible therefore be given to the distribution of the former. Nevertheless it is useful to include at least one standard solution so that the true concentration of at least one distributed solution is known. The distributed solutions should also cover the range of determinand concentrations of interest and in general the more types of sam.ple that can be included the better In the Harmonised Monitoring Scheme the general aim is to distribute one standard solution near the middle of the concentration range of interest and two samples of river water of different types with concentrations near the lower and upper concentrations of interest.Whenever possible (see section 2.9.2) these solutions are distributed at the above concentrations so that their dilution before analysis is not necessary; such dilution may prevent detection of certain sources of bias April 1979 ANALYTICAL RESULTS FROM A NUMBER OF LABORATORIES 285 2.9.2 Stability of distributed solutions It is essential that the co-ordinating laboratory takes great care to ensure that each laboratory receives essentially identical portions of each of the distributed solutions. Of particular importance in water analysis are the cleanliness of solution containers and the stability of the deterrninand. Preliminary tests in the co-ordinating laboratory are often desirable to ensure this identity of the distributed solutions.In water analysis many determinands are so unstable that distribution of samples without special treatment is impossible. When this is so the use of preserving reagents can be of value but the co-ordinating laboratory should first ensure that a reagent it proposes to use is compatible with the analytical methods used by all laboratories. One approach commonly recommended to overcome such problems is to distribute a stable standard solution with a sample the former being used to spike a portion of the latter before analysis. The difference between the results for the spiked and unspiked portions is then used to assess the bias. This procedure can be of value but should be used with caution because it may not detect those sources of bias whose magnitudes are independent of the concentration of the determinand (see section 2.9.4).Therefore the best approach needs to be decided individually for each study; the procedures used in the Harmonised Monitoring Scheme for particular determinands will be described in subsequent papers. 2.9.3 Number of analyses required on each solution The number of replicate analyses w required on each solution depends on the ratio B/a, where B is the maximum tolerable bias and a is the standard deviation of results. DaviesZ6 describes the calculation of w for different values of B/a for various probability levels and when a is not known exactly. In the Harmonised Monitoring Scheme an approximate value for w to be used by all laboratories is obtained by setting B and at equal to their maximum tolerable values.The ratio B/at is therefore 2.0 and the corresponding value of w (0.05 probability of errors of the first and second kind) is approximately 3. However it is necessary to increase this value somewhat because only an estimate of at is available.26 At the same time there will be a tendency for at to be less than the maximum tolerable value. Hence for convenience the approximation w = 5 is used for each solution; experience shows this to be a reasonable approach. The precision achieved by certain laboratories may be substantially better than the maximum tolerable value so that a value for w smaller than 5 could be used. However the use of w = 5 throughout is preferred because it provides better estimates of precision from these tests (see section 2.9.5) and also ensures better ability to detect bias by those laboratories with better precision.I t is also worth noting that the values of the maximum tolerable bias and standard devi-ation have a marked effect on the number of tests needed to detect such bias. For example, suppose that B is 5% at is 5% and that a laboratory just achieves the required precision. The number of tests required on each solution would on the above basis be approximately 15 a number that for several reasons would often be impracticable. Whenever possible one portion of each of the distributed solutions is analysed on each of five days. This approach makes some allowance for the possibility that bias can vary from day to day and thus provides an estimate of the average bias; see also section 2.9.5.How-ever if the stability of a distributed solution or the convenience of the laboratories require, all replicate analyses may be made on one day without great effect on the value of the informa-tion. 2.9.4 In the Harmonised Monitoring Scheme for each solution and laboratory the mean and its 90% confidence limits 5 4 h are calculated from the five individual results. The upper limit for the true bias (95% confidence level) is then calculated by the co-ordinating labora-tory as 5 + h - T if 5 > T (where T is the true value for the distributed solution) or as 5 - h - T if 3 < T. These expressions apply when the value of T is such that the maxi-mum tolerable bias is a fixed concentration (see section 2.3). For greater values of T where the target is a fixed percentage of T the corresponding expressions for bias become lOO(3 + h - T)/T and lOO(5 - h - T)/Tyo.If the estimated bias is greater than the maximum tolerable value laboratories are informed as soon as possible so that in general, Calculation of bias for each laborator 286 WILSON APPROACH FOR ACHIEVING COMPARABLE Analyst VOl. 104 possible causes of the bias can be sought and eliminated. In practice this assessment of bias also includes some subjective consideration. For example a laboratory whose bias marginally exceeded the maximum tolerable value for one solution but was well within the target for other solutions is usually considered acceptable. On the other hand a laboratory only just within the target for all solutions may well have need to check possible sources of bias.In this approach T is equated to the known concentration of the distributed standard solution. For samples of river water the value of T is not generally known and an approxi-mate value must be used. In the Harmonised Monitoring Scheme this value has generally been equated to the mean of results from all laboratories. This approach is considi.red reasonable given the preceding stages of Fig. 1 and the number of laboratories (11) involved,* but it is of course not necessarily correct. Careful consideration of the results from each test and of possible sources of bias in the analytical methods used is generally essential before assigning a value to T.' This problem can sometimes be much reduced if a sample can be obtained with a negligible concentration of the determinand but typical in all other respects.An accurately known amount of the determinand can then be added by the co-ordinating laboratory before distribution of portions of the spiked solution. In this situation the value of T can be determined with reasonably small error and this approach has been useful in connection with the determination of lead in drinking water.34 Similarly T would be known if bias is assessed from the difference between the results for spiked and unspilced portions of a distributed solution (see section 2.9.2). 2.9.5 Estimates of precision The results from these tests also provide estimates of the standard deviation st of analytical results for each of the distributed solutions. These estimates are poor in that they have only four degrees of freedom but it is useful to compare these estimates with the target values as in the tests of within-laboratory precision.If there has been any deteriora-tion of precision not revealed by the control chart(s) the upper limit for bias may be largely governed by the imprecision (&h) of the mean iresult rather than by bias. 2.9.6 Need for further inter-laboratory tests of bitzs On satisfactory completion of this final stage of Fig. 1 it is provisionally taken that laboratories have achieved the required accuracy. However precision and bias at this stage have been estimated for few sample types and both parameters may also deteriorate subsequently. As explained above the use of control tests will help to ensure that precision and certain sources of bias remain satisfactory but it is also considered essential to repeat the tests of between-laboratory bias regularly.As always the more frequent such tests and the greater the number of samples included in them the better will be the control. The effort involved is however substantial and at present in the Harmonised Monitoring Scheme, the tests are repeated at approximately 6-monthly intervals for each of the determinands studied to date. Further only one sample is distributed in each repeat test but the aim is to use samples of types different to those distributed in previous tests. 3 Conclusions The discussion above has shown that close attention must be paid to many different aspects of analysis if the accuracy of routine analytical results is to meet a specified and reasonable value.I t is worth stressing that virtually all the many points and suggestions mentioned in this long paper have been included because they have caused problems in one or more groups of laboratories with which the a.uthor has been involved. To overcome such problems permanently a systematic approach is essential and the scheme described here (summarised in Fig. 1) represents one such approach that has been found successful in the analysis of river and other waters. It is considered that the principles of this approach are of general value and validity but of course many detailed modifications are possible. The fact that the scheme is invol.ved and unlikely to achieve rapid progress is * The accuracy of the value used for T will all other things being equal improve as the number of laboratories increases April 1979 ANALYTICAL RESULTS FROM A NUMBER OF LABORATORIES 287 a disadvantage.However it seems to the author that it is only by a scheme of this nature that any progress will be achieved on a permanent and sound basis. Clearly there is no substitute for detailed consideration of each particular application and this may well suggest simplifications of value. Nevertheless such simplifications should only be undertaken after very careful consideration; experience shows that lack of success and wasted effort can result if short cuts are taken on insufficient evidence. Clearly effort is required to implement analytical quality control and the amount depends on the required accuracy the analytical methods and the degree of control considered appropriate.As a broad indication suggestions have been made that 10-20%11 or 15-20~0s of an analyst’s time should be devoted to quality-control work. This proportion may seem rather large and may be difficult to achieve in initially establishing an analytical quality-control scheme in a laboratory. Nevertheless such effort is considered a reasonable goal though initiation of control schemes should not be prevented if only a smaller amount of effort is available initially. Finally it is worth making the obvious point that it is not generally possible to obtain direct estimates of precision and bias for every sample that laboratories analyse. Hence, though in the author’s opinion analytical quality control should be an integral part of the work of a laboratory it cannot prove the accuracy of every result.Rather it provides valuable confirmatory evidence of the adequacy of all the various means adopted by the analyst to ensure the required accuracy. In other words the continuing and critical assess-ment by the analyst of all that he does remains as always of prime importance. In general a little control is better than none. Many colleagues have contributed to many aspects of the approach described in this paper. The author is grateful to them all but particular thanks are due to Messrs. R. V. Cheeseman D. J. Dewey I. R. Morrison and W. J. Wyse and for much guidance and advice on statistical aspects to Messrs. W. J. Allum J. C. Ellis R. E. Fry and R.F. Lacey. Thanks are also due to the author’s many colleagues in the Department of the Environment Water Authorities and River Purification Boards who operate the Harmonised Monitoring Scheme. Finally the author is also grateful to the Director of the Water Research Centre for per-mission to publish this paper. Appendix A few points of detail concerning the tests of within-laboratory precision are mentioned below. (a) The degrees of freedom of the estimates sw and s b are m(n - 1) and (m - l) respec-tively. The concept of degrees of freedom is not strictly applicable to the combined estimate st but a reasonable approximation is to assign f degrees of freedom to st where f is the integer nearest to the value of using the notation in reference 21. ( b ) In comparing an experimental estimate of a standard deviation with a target value by the variance-ratio test the target standard deviation has an infinite number of degrees of freedom.(c) For a given solution the precision of a laboratory is considered acceptable if st is not significantly greater (0.05 probability level) than the target value. Such values of st can be obtained when the population standard deviation ut is greater than the target value, i.e. the probability of an error of the second kind is much greater than 0.05. If desired this probability can be decreased by increasing the number of replicate analyses used to estimate st26S27 but the number corresponding to a probability of 0.05 will normally be impracticable. An estimate st is obtained for each of several solutions and this provides an opportunity to judge whether or not there is a general tendency for all s t values to approach or exceed the targets.Three points reduce though they do not eliminate this problem. (i 288 WILSON APPROACH FOR ACHIEVING COMPARABLE Analyst VoZ. 104 (ii) Subsequent tests (see section 2.8) will provide more precise estimates of at. (iii) By aiming to use methods capable of rather better precision than required there will be a tendency to obtain at values less than the targets. ( d ) To ensure valid estimates of ab and at the concentration of the determinand in a given solution must be the same for all batches of analyses. When determinands are unstable, this condition can be satisfied for standard solutions by using freshly prepared solutions for each batch However for samples no direct estimates of ab and at are possible and the following approximate approach is followed.A freshly collected sample is used for each batch these m samples being selected so that the determinand concentration is approxi-mately the same in each. Alternatively the same sample can be used for all batches provided its concentration of determinand changes by no more than approximately 20%. On completion of the m batches sw s b and st are calculated for standard solutions and sw is calculated for samples. The assumption is then made that the between-batch random error is due only to uncorrected variations in the calibration graph from batch to batch. On this basis and when the analytical response is directly proportional to the concentration of the determinand (Tb is independent of the type of solution and is also directly proportional to concentration ie.ab = kC. An approximate value of k can therefore be obtained from values of s b for standard solutions. Thus an approximate value for s b for a sample can be obtained from its mean concentration and the estimated value of k . Finally st for a sample can then be calculated from .st = .\/(s + ~ 2 ) . (e) The limit of detection (0.05 probability of errors of the first and second kinds) is given by 4.65aw where ow is the within-batch standard deviation of blank determinations (in concentration units) ,14915 provided that : (i) an analytical result is obtained by making one measurement each of a blank and a sample the apparent concentration of the blank being subtracted (in fact or in effect) from that of the sample; (ii) the apparent concentrations of any solution of low concentration follow the normal distribution; and (iii) the within-batch standard deviation of the apparent concentrations for any solution of low concentration is independent of the concentration of the determinand and is the same for blanks and samples.If any one of these conditions is not satisfied other expressions for the limit of detection are req~ired.1~ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. References Biittner J. Borth R. Boutwell J. H. and Broughton P. M. G. Clinica Chim. Acta 1976 63, Eisenhart C. J . Res. Natn. Bur. Stand. 1963 67C 161. Simpson E.A. J . Inst. Water Engrs Scient. 1978 32 45. Wilson A. L. Talanta 1965 12 701. Research and Development Department Central Electricity Generating Board “Methods of Sampling and Analysis Volume 1. Analytical Quality Control Laboratory “Handbook for Analytical Quality Control in Water and Wastewater Laboratories,” US Environmental Protection Agency Cincinnati 1972. Cheeseman R. V. Technical Memorandum TM 96 Water Research Centre Medmenham Bucks., 1974. Wilson A. L. “The Chemical Analysis of Water,” Analytical Sciences Monograph No. 2 Society for Analytical Chemistry London 1974. Gans I. and Sonneborn M. in Aurand K. Hasselborth U. Miiller G. Schumacher W. and Steuer W. Editors “Die Trinkwasser-Verordnung,” Erich Schmidt Verlag Berlin 1977 pp. 181-195.Green A. C. and Naegele R. Report EPA-600/4-77-031 US Environmental Protection Agency, Cincinnati 1977. Wilson A. L. J . Inst. Water Engrs Scient. 1978 32 57. Ekedahl G. Rondell B. and Wilson A. L. “Analytical Errors” in “Manual on Analysis for Water Allen H. E. and Mancy K. H. in Ciaccio L. Id. Editor “Water and Water Pollution Handbook, Roos J . B. Analyst 1962 87 832. Currie L. A. Analyt. Chem. 1968 40 586. McFarren E. F. Lishka R. J. and Parker J. H. Analyt. Chem. 1970 42 358. Eckschlager K. Analyt. Chem. 1972 44 878. F25. Steam and Water,” 1966. Pollution Control,’’ World Health Organization Geneva in the press. Volume 3,” Marcel Dekker New York 1972 pp. 971-1020 Apri1 1979 ANALYTICAL RESULTS FROM A NUMBER OF LABORATORIES 289 18. 19. 20. 21.22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 34. Biittner J. Borth R. Boutwell J. H. Broughton P. M. G. and Bowyer R. C. Clinica Chim. Wilson A. L. Talanta 1970 17 21. Wilson A. L. Talanta 1974 21 1109. Wilson A. L. Talanta 1970 17 31. Wilson A. L. Talanta 1973 20 725. “Standing Committee of Analysts to Review Standard Methods for Quality Control of the Water Cycle. Standing Technical Committee Reports Number 7 Department of the Environment/National Water Council London 1977. Department of the Environment/National Water Council “Lead in Potable Waters by Atomic Absorption Spectrophotometry 1976,” Methods for the Examination of Waters and Associated Materials HM Stationery Office London 1977. ISO/R78-1969(E) International Standardization Organization Geneva 1969. Davies 0. L. Editor “Design and Analysis of Industrial Experiments,” Second Edition Oliver and Boyd Edinburgh 1956. Davies 0. L. and Goldsmith P. L. Editors “Statistical Methods in Research and Production,” Fourth Revised Edition Oliver and Boyd Edinburgh 1972. Bennett C. A. and Franklin N. L. “Statistical Analysis in Chemistry and the Chemical Industry,” Chapman and Hall London 1954. Nalimov V. V. “The Application of Mathematical Statistics to Chemical Analysis,” Pergamon Press Oxford 1963. Wilson A. L. Technical Memorandum TM 56 Water Research Association Medmenham Bucks., 1970. Woodward R. H. and Goldsmith P. L. “Cumulative Sum Techniques,” Oliver and Boyd Edin-burgh 1964. Griffin D. F. Am. J. Med. Technol. 1968 34 644. Ranson L. and Wilson A. L. Technical Report TR 28 Water Research Centre Medmenham, Bucks. 1976. Ada 1976 69 F1. First Report May 1973- January 1977.” Received September 1 lth 1978 Accepted November 2nd 197