ASTM C1067-00
(Practice)Standard Practice for Conducting A Ruggedness or Screening Program for Test Methods for Construction Materials
Standard Practice for Conducting A Ruggedness or Screening Program for Test Methods for Construction Materials
SCOPE
1.1 This practice covers a procedure for detecting and reducing sources of variation in a test method early in its development and prior to an interlaboratory study. One of the main purposes of the procedure, by means of a ruggedness or screening program, is the elimination of test methods that exhibit poor precision even after vigorous efforts to reduce their variability.
1.2 This practice covers, in very general terms, techniques for planning, conducting, and analyzing results from a few laboratories to establish whether a well written test method is good enough to justify an interlaboratory study. Annex A1 provides the details of the procedure with an example and Annex A2 gives the theoretical background.
1.3 This practice is intended to be used before spending the time and money required to conduct a full scale interlaboratory study, such as those in Practices C670, C802, or E691.
1.4 This standard may involve hazardous materials, operations, and equipment. This standard does not purport to address all of the safety problems associated with its use. It is the responsibility of whoever uses this standard to consult and establish appropriate safety and health practices and determine the applicability of regulatory limitations prior to use.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation:C1067–00
Standard Practice for
Conducting A Ruggedness or Screening Program for Test
Methods for Construction Materials
This standard is issued under the fixed designation C1067; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (e) indicates an editorial change since the last revision or reapproval.
1. Scope E456 Terminology Relating to Quality and Statistics
E691 Practice for Conducting an Interlaboratory Study to
1.1 Thispracticecoversaprocedurefordetectingsourcesof
Determine the Precision of a Test Method
variationinatestmethod.Theprocedureshouldbeusedduring
E1169 Guide for Conducting Ruggedness Tests
the development of a test method, before the interlaboratory
study is executed, such as those in Practices C670, C802, and
3. Terminology
E691. Interlaboratory studies can be expensive to execute.
3.1 Definitions:
Resources will probably be more efficiently used if sources of
3.1.1 determination value, n—numericalquantitycalculated
variation in a test method are eliminated prior to performing
as directed in the test method using direct measurements
the interlaboratory study. The procedure also is useful for
obtained in accordance with the procedures given in the test
determiningsourcesofvariationinanexistingtestmethodthat
method.
has been found to have poor precision.
3.1.2 replication, n—the act of obtaining two or more
1.2 This practice covers, in very general terms, techniques
determination values under specified conditions. The number
for planning, collecting data, and analyzing results from a few
of replications must be finite and the scope of the replication
laboratories. Annex A1 provides the details of the procedure
operation may be narrow or broad, but must be specified.
with an example and Annex A2 gives the theoretical back-
3.1.3 For definitions of other statistical terms used in this
ground.
standard, refer to Terminology E456.
1.3 The practice does not give information pertinent to
3.2 Definitions of Terms Specific to This Standard:
estimating within- or between-laboratory precision.
3.2.1 factor, n—an element in the test procedure or labora-
1.4 This standard does not purport to address all of the
tory environment that is a potential source of variation in test
safety concerns, if any, associated with its use. It is the
results.
responsibility of the user of this standard to establish appro-
3.2.2 ruggedness, adj—the characteristic of a test method
priate safety and health practices and determine the applica-
that produces test results that are not influenced by small
bility of regulatory limitations prior to use.
differences in the testing procedure or environment.
2. Referenced Documents 3.2.3 screening, n—the detection of significant sources of
variation as compared to chance variation.
2.1 ASTM Standards:
3.2.4 variable, n—a number or quantity that varies.
C670 Practice for Preparing Precision and Bias Statements
for Test Methods for Construction Materials
4. Summary of Practice
C802 Practice for Conducting an Interlaboratory Test Pro-
4.1 The practice requires that the user develop, from theo-
gram to Determine the Precision of Test Methods for
2 retical or practical knowledge, or both, a list of factors that
Construction Materials
plausibly would cause significant variation in test results if the
factors were not controlled. The technique is limited to the
analysis of the effects seven factors and requires considerably
This practice is under the jurisdiction of ASTM Committee C09 on Concrete
less effort than would be required to collected data for seven
and ConcreteAggregates . This practice was developed jointly byASTM Commit-
tees C-1, C-9, D-4, and D-18, and is endorsed by all four committees. factors in a full factorial study. Procedures exist for analysis of
Current edition approved June 10, 2000. Published August 2000. Originally
published as C1067–87. Last previous edition C1067–87 (1993).
2 3
Annual Book of ASTM Standards, Vol 04.02. Annual Book of ASTM Standards, Vol 14.02.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.
C1067–00
smaller and larger numbers of factors (see Guide E1169), but additional laboratories will probably contribute fresh critical
seven is a convenient number for many test methods for reviewofthevalidityofthetestmethodandprovideassistance
construction materials. The seven-factor analysis requires 16 in clarifying the instructions of the test method when needed.
determinations by each laboratory. The procedure can be
6. Materials
usefully executed by a single laboratory, but sometimes addi-
6.1 The number and types of material shall cover the range
tionalinformationcanbeobtainedifitisrepeatedinoneortwo
additional laboratories. of material properties to which the test method is applicable.
The test method does not apply to material types or property
4.2 The procedure requires that two levels of each factor be
identified, then 16 determinations be done on a prescribed values outside the range evaluated.Three to five materials will
usually be sufficient.
combinations of factor levels. The levels assigned to a factor
may be quantitative or qualitative (for example, brass versus 6.1.1 Some preliminary testing may help the laboratories
involved determine the materials that shall be used in the
steel).
4.3 The disadvantage of this type of analysis is that the screening program.
method only estimates simple effects of each factor and does
7. Procedure
not detect interactive effects among factors.
7.1 Determine the number of laboratories that will partici-
5. Significance and Use
pate in the program and which materials each will use in the
5.1 The purpose of a ruggedness evaluation is to determin-
program. The maximum amount of information is obtained if
ing how sensitive the test method is to changes in levels of
all laboratories include all materials in their part of the
pertinent operating factors. Normally, operating conditions for
program, however cost can be reduced by each laboratory
a test method are defined along with an allowable tolerance.A
using a different material. Caution must be exercised in
ruggedness analysis determines that effect of worst-case varia-
interpreting the results since laboratory-dependent cannot be
tion in operating conditions within this tolerance range. The
separated from material-dependent effects.
method then can be revised with smaller tolerances on operat- 7.2 Factors that are likely to have the greatest effect on the
ing conditions to improve the precision.
variability in the test results are selected for study. Levels of
5.2 Amajor reason for poor precision in test methods is the these factors are determined, selecting the minimum and
lackofadequatecontroloverthesourcesofvariationintesting
maximumlevelsthatwouldplausiblyoccurintheexecutionof
procedures or testing environments.These sources of variation
the test method if there were no particular efforts to control
often are not controlled adequately because they were not
them. Only two levels are allowed. Levels often represent
identified during the development of the test procedures.
quantitative properties, such as temperature, pressure, etc, but
5.3 All new test methods must be subjected to an interlabo-
they may also represent nonquantitative values, such as old vs
ratoryprogramforpurposesofdevelopingaprecisionandbias
new,wetvsdry,etc.Inthisstandard,factorsareassignedletter
statement. These programs can be expensive and lengthy, and
designations, A– G, and the two levels of each factor are
the result may be that the determination is made that the
designated with upper and lower cases of these letters, as in
methodistoovariabletobepublishedwithoutfurtherrevision.
Table 1.
Interlaboratory studies typically give the subcommittee an
7.3 Assign combinations of factor levels to experimental
indication that the method is too variable, but they do not
determinationsaccordingtoTable1.The8determinationswill
usually give a clear picture of what is causing the variation.
be done in duplicate, therefore, the full study on each material
Application of this ruggedness practice using one or a few
will require 16 determinations.
laboratoriesmaybeamuchmoreeconomicalwaytodetermine
7.4 Constructa16rowby16columnresultsmatrixfromthe
these causes.
16 determinations values (d – d ) as shown in Table 2. The
1 16
5.4 Many existing test methods were published before there
absolutevaluesofthedeterminationsineachrowareidentical,
was a requirement that precision and bias statements be onlythesignsvary.Calculate Zand Wstatisticsasshowninthe
developed. Since this became a requirement, most of these test
equations below.
methodshavedevelopedprecisionandbiasstatements,andthe
Z 5 d,where d 8sarethe16resultsineachrow ~r!. (1)
r (1 i i
result is that many have been found to suffer from relatively
large amount of variation. Use of this practice represents a
Z
r
relatively simple way to investigate the causes of variation in
W 5 (2)
r
test methods, so that a subcommittee will have some guidance
as to which parts of the test method need to be studied further
TABLE 1 Pattern of Assigning Levels to Seven Factors
for revision.
Determination Number
5.5 Theprocedurecanbeusedforaprogramwithinasingle
Factor 1 2 3 4 5 6 7 8
laboratory, but involvement of at least three laboratories is
A aaa a A A A A
recommended, particularly if the single laboratory were to be
B b bBB b bB B
the one in which the test method was developed. This is
C C cC c C cCc
particularly important for new test methods. The originating DD D d d d d D D
Ee E e E E e E e
laboratory is so much a part of the development of the test
F F ff F F ff F
method that it is difficult for it to be objective in spotting any
GG g g G g G G g
problems in the clarity of the test method directions. Two
C1067–00
TABLE 2 Results Matrix of 16 Determinations (d – d )
1 16
Eight Determinations for Replicate Set 1 Eight Determinations for Replicate Set 2
row 1234567812345678 ZW
1 d d d d d d d d d d d d d d d d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 1
2 d d d d –d –d –d –d d d d d –d –d –d –d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 2 2
3 d d –d –d d d –d –d d d –d –d d d –d –d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 3 3
4 d –d d –d d –d d –d d –d d –d d –d d –d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 4 4
5 d d –d –d –d –d d d d d –d –d –d –d d d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 5 5
6 d –d d –d –d d –d d d –d d –d –d d –d d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 6 6
7 d –d –d d d –d –d d d –d –d d d –d –d d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 7 7
8 d –d –d d –d d d –d d –d –d d –d d d –d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 8 8
9 d d d d d d d d –d –d –d –d –d –d –d –d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 9 9
10 d d d d –d –d –d –d –d –d –d –d d d d d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 10 10
11 d d –d –d d d –d –d –d –d d d –d –d d d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 11 11
12 d –d d –d d –d d –d –d d –d d –d d –d d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 12 12
13 d d –d –d –d –d d d –d –d d d d d –d –d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 13 13
14 d –d d –d –d d –d d –d d –d d d –d d –d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 14 14
15 d –d –d d d –d –d d –d d d –d –d d d –d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 15 15
16 d –d –d d –d d d –d –d d d –d d –d –d d Z W
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 16 16
TABLE 3 Summary of Statistics for Seven Factors and Random
7.5 The W statistic for row 1 represents the simple sum of
Error
the determinations and are not used in this analysis. Statistics
FactorWF
for rows 2–8 (W – W ) represent the effects of the seven
2 8
A W W /X
factors.Thestatisticforrow9(W )representthetotalvariation
2 2
B W W /X
3 3
between the two replicate sets and is not used in this analysis.
C W W /X
4 4
Statistics for rows 10 through 16 (W – W ) are used to
10 16 E W W /X
5 5
calculatetheerrorvariance(X),whichthenisusedtocalculate F W W /X
6 6
G W W /X
7 7
the test criterion (F) for each factor, as shown by the equations
H W W /X
8 8
below. Calculations are summarized in Table 3.
W
W
W
X 5 ~ W !/7 (3)
( r
r510 W X = ((W )/7, for W
13 10-16
W
2 14
W
r W
F 5 , where F is the F statistic for the effect of factor
f f
W
X 16
f (1–7, represented by W – W , respectively)
2 8
7.6 A F value of$5.59 represents a significant effect for
factor f at a probability of 5% for drawing an erroneous
8. Keywords
conclusion.
7.7 An example of an analysis of data representing results
8.1 precision; ruggedness; test method; variation
on 4 materials from 3 laboratories is shown in Annex A1.
ANNEXES
(Mandatory Information)
A1. EXAMPLE OF A RUGGEDNESS PROGRAM
A1.1 This annex describes the procedure for conducting a A1.2.3 Topickmaterialsthatcovertherangeoftheproperty
ruggednessevaluationusingasanexampleadescriptionofthe ofinterestfortherangeofphysicalformsofthematerialstobe
ruggednessevaluationonatestmethodforthemeasurementof tested, and
the viscosity of asphalt. A1.2.4 To determine the proper levels to be evaluated for
each of the chosen variables.
A1.2 As the first step in the ruggedness evaluation, each of
A1.3 In this example, representatives of the three labora-
the laboratories critically examined the procedure in the
tories, after familiarizing themselves with the test method as
proposed test method. The objectives of the examination were
specified inA1.2, met and tried to improve the instructions for
as follows:
the viscosity method. They selected variables, materials, and
A1.2.1 To determine if the instructions are clear, concise,
levels that showed the effect of the variation. One of the
and complete,
laboratories measured viscosity at 24°C, 25°C, and 26°C and
A1.2.2 To decide which factors are likely to influence test found that there was about a 10% variation with a change of
results and therefore should be included in the study, 1°C. This was considered too large so 24.6 and 25.4°C were
C1067–00
selected as the lower and upper temperature levels for the row, Z , is the algebraic addition of the second row in Table
ruggedness test. In the same manner, the effect of the other A1.5 and measures the effect of temperature. In the same
variables were evaluated and the two levels to be evaluated manner, the third row, Z , measures the effect of the age of the
were determined. viscometer. The fourth row, Z , measures the effect of vacuum
level. The fifth row, Z , measures
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.