ISO/TR 22914:2020

Statistical methods for implementation of Six Sigma - Selected illustration of analysis of variance

This document describes the necessary steps of the one-way and two-way analyses of variance (ANOVA) for fixed effect models in balanced design. Unbalanced design, random effects and nested design patterns are not included in this document. This document provides examples to analyse the differences among group means by splitting the overall observed variance into different parts. Several illustrations from different fields with different emphasis suggest the procedure of the analysis of variance.

Méthodes statistiques pour la mise en œuvre du Six Sigma - Exemples choisis d'application de l'analyse de la variance

General Information

• Status: Published
• Publication Date: 11-Oct-2020
• ICS: 03.120.30 - Application of statistical methods
• Technical Committee: ISO/TC 69/SC 7 - Applications of statistical and related techniques for the implementation of Six Sigma
• Drafting Committee: ISO/TC 69/SC 7/WG 1 - Design of experiments
• Current Stage: 6060 - International Standard published
• Start Date: 12-Oct-2020
• Due Date: 03-Jul-2020
• Completion Date: 12-Oct-2020

Overview

ISO/TR 22914:2020 - "Statistical methods for implementation of Six Sigma - Selected illustration of analysis of variance" provides a practical, example-driven guide to using analysis of variance (ANOVA) in Six Sigma projects. The Technical Report focuses on one-way and two-way ANOVA for fixed-effect models in balanced designs, illustrating how to partition observed variance to test differences among group means and guide process-improvement decisions. Note: unbalanced designs, random effects and nested designs are explicitly excluded.

Key topics and technical requirements

• Scope and model type: Fixed-effect ANOVA for balanced data (one-way and two-way classifications).
• Data planning and measurement: Guidance on data collection plans, variable descriptions and measurement-system considerations to ensure valid ANOVA inputs.
• Assumption checks: Steps to verify ANOVA assumptions - independence, normality (residuals), and homogeneity of variances (homoscedasticity) - plus outlier detection and strategies for non‑standard cases.
• ANOVA procedure: Practical workflow including stating hypotheses, graphical analysis, computing sums of squares, F-tests, p-values, and residual analysis.
• Further analysis & conclusions: Guidance on post-hoc interpretation and next steps for process improvement.
• Illustrative annexes: Multiple domain examples (e.g., bond strength; effect of script and training on income per sale; welded joint strength; water consumption in a petroleum enterprise; hub total hours) and Annex F containing ANOVA formulae.
• Normative references: Uses ISO 3534-1:2006 and ISO 3534-3:2013 for terms and symbols.

Practical applications and users

ISO/TR 22914:2020 is designed for practitioners who apply statistical methods within Six Sigma and continuous improvement programs:

• Six Sigma Black/Green Belts and project teams using ANOVA to identify factors that significantly affect process performance.
• Quality engineers, industrial engineers, and statisticians seeking standardized, example-based guidance for fixed-effect ANOVA in balanced experiments.
• Data analysts and researchers in manufacturing, services, petroleum, welding, sales and other sectors where comparing group means and allocating variance across factors informs decisions.
  Practical uses include testing process changes, comparing supplier batches, evaluating training or script effects, and prioritizing improvement actions based on statistically significant factor effects.

Related standards

• ISO 3534-1:2006 - Statistics: Vocabulary and symbols (general statistical terms)
• ISO 3534-3:2013 - Statistics: Vocabulary and symbols (design of experiments)

Keywords: ISO/TR 22914:2020, ANOVA, analysis of variance, Six Sigma, one-way ANOVA, two-way ANOVA, fixed effects, balanced design, F-test, residual analysis, homoscedasticity.

Standards Content (Sample)

TECHNICAL ISO/TR
REPORT 22914
First edition
2020-10
Statistical methods for
implementation of Six Sigma —
Selected illustration of analysis of
variance
Méthodes statistiques pour la mise en œuvre du Six Sigma - Exemples
choisis d'application de l'analyse de la variance
Reference number
©
ISO 2020
© ISO 2020
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: [phone]
Email: [e-mail]
Website: www.iso.org
Published in Switzerland
ii © ISO 2020 – All rights reserved

Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Symbols and abbreviated terms . 4
5 General description of one-way and two-way classifications . 4
5.1 General . 4
5.2 Stating objectives . 5
5.3 Data collection plan . 6
5.4 Variables description . 6
5.5 Measurement system considerations . 6
5.6 Performing data collection . 6
5.7 Verification of ANOVA assumptions . 7
5.7.1 General. 7
5.7.2 Test of normality . 7
5.7.3 Test of homogeneity of variance . 7
5.7.4 Test of independence . 7
5.7.5 Outliers identification . 7
5.7.6 How to deal with non-standard cases . 8
5.8 Undertaking ANOVA analysis . 8
5.8.1 State hypotheses H and H . 8
0 1
5.8.2 Graphical analysis . 8
5.8.3 Generate analysis results . 8
5.8.4 Residual analysis . 8
5.9 Further analysis . 9
5.10 Conclusion . 9
6 Description of Annexes A through E . 9
Annex A (informative) Bond strength .11
Annex B (informative) Effect of script and training on income per sale .19
Annex C (informative) Strength of welded joint .30
Annex D (informative) Water consumption in a petroleum enterprise .38
Annex E (informative) The hub total hours used on a task .45
Annex F (informative) ANOVA formulae .51
Bibliography .56
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www .iso .org/ directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www .iso .org/ patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www .iso .org/
iso/ foreword .html.
This document was prepared by Technical Committee ISO/TC 69, Applications of statistical methods,
Subcommittee SC 7, Applications of statistical and related techniques for the implementation of Six Sigma.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html.
iv © ISO 2020 – All rights reserved

Introduction
Analysis of variance (ANOVA) is a collection of statistical models used to analyse the differences
among group means and their associated procedures (such as "variation" among and between groups),
developed by statistician and evolutionary biologist Ronald A. Fisher. In the ANOVA setting, the observed
variance in a particular variable is partitioned into components attributable to different sources of
variation. In its simplest form, ANOVA provides a statistical test of whether or not the means of several
groups are equal, and therefore generalizes the t-test to more than two groups. ANOVA models are
useful for comparing (testing) three or more means (groups or variables) for statistical significance. It
is conceptually similar to multiple two-sample t-tests, but is more conservative (it results in less type I
error) and is therefore suited to a wide range of practical problems. In Six Sigma, ANOVA is used to find
out if there are differences in the performances of different groups, and ultimately to find out if these
differences count, or are important enough that a significant change or adjustment should be made. It
serves as a guide on which aspect(s) of a process improvements can, or should, be made.
ANOVA is the synthesis of several ideas and it is used for multiple purposes. As a consequence, it is
difficult to define concisely or precisely. Classical ANOVA for balanced data does the three following
things at once.
1) As exploratory data analysis, an ANOVA is an organization of an additive data decomposition, and
its sums of squares indicate the variance of each component of the decomposition (or, equivalently,
each set of terms of a linear model).
2) Comparisons of mean squares, along with an F-test allow testing of a nested sequence of models.
3) Closely related to the ANOVA is a linear model fit with coefficient estimates and standard errors.
In short, ANOVA is a statistical tool used in several ways to develop and confirm an explanation for the
observed data. Additionally:
1) it is computationally elegant and relatively robust against violations of its assumptions;
2) it provides industrial strength by (multiple sample comparison) statistical analysis;
3) it has been adapted to the analysis of a variety of experimental designs.
As a result, ANOVA has long enjoyed the status of being the most used (some would say abused)
statistical technique in psychological research. "ANOVA "is probably the most useful technique in the
field of statistical inference. ANOVA is difficult to teach, particularly for complex experiments, with
split-plot designs being notorious.
There are three main assumptions:
1) independence of observations — this is an assumption of the model that simplifies the statistical
analysis;
2) normality — the distributions of the residuals are normal;
3) equality (or "homogeneity") of variances, called homoscedasticity — the variance of data in groups
is expected to be the same.
If the populations from which data to be analysed by a one-way analysis of variance (ANOVA) were
sampled violate one or more of the one-way ANOVA test assumptions, the results of the analysis can be
incorrect or misleading. For example, if the assumption of independence is violated, then the one-way
ANOVA is simply not appropriate, although another test (perhaps a blocked one-way ANOVA) can be
appropriate. If the assumption of normality is violated, or outliers are present, then the one-way ANOVA
is not necessarily the most powerful test available. A nonparametric test or employing a transformation
can result in a more powerful test. A potentially more damaging assumption violation occurs when
the population variances are unequal, especially if the sample sizes are not approximately equal
(unbalanced). Often, the effect of an assumption violation on the one-way ANOVA result depends on the
extent of the violation (such as how unequal the population variances are, or how heavy-tailed one or
another population distribution is). Some small violations can have little practical effect on the analysis,
while other violations can render the one-way ANOVA result uselessly incorrect or uninterpretable. In
particular, small or unbalanced sample sizes can increase vulnerability to assumption violation.
vi © ISO 2020 – All rights reserved

TECHNICAL REPORT ISO/TR 22914:2020(E)
Statistical methods for implementation of Six Sigma —
Selected illustration of analysis of variance
1 Scope
This document describes the necessary steps of the one-way and two-way analyses of variance
(ANOVA) for fixed effect models in balanced design. Unbalanced design, random effects and nested
design patterns are not included in this document.
This document provides examples to analyse the differences among group means by splitting the
overall observed variance into different parts. Several illustrations from different fields with different
emphasis suggest the procedure of the analysis of variance.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO 3534-1:2006, Statistics — Vocabulary and symbols — Part 1: General statistical terms and terms used
in probability
ISO 3534-3:2013, Statistics — Vocabulary and symbols — Part 3: Design of experiments
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO 3534-1, ISO 3534-3 and the
following apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org/
3.1
response variable
variable representing the outcome of an experiment
[SOURCE: ISO 3534-3:2013, 3.1.3, modified — the notes have been removed.]
3.2
predictor variable
variable that can contribute to the explanation of the outcome of an experiment.
[SOURCE: ISO 3534-3:2013, 3.1.4, modified — the notes have been removed.]
3.3
model
formalized representation of outcomes of an experiment
[SOURCE: ISO 3534-3:2013, 3.1.2, modified — the notes and examples have been removed.]
3.4
analysis of variance
ANOVA
technique which subdivides the total variation of a response variable into components associated with
defined sources of variation
[SOURCE: ISO 3534-3:2006, 3.3.8, modified — the notes and examples have been removed.]
3.5
degree of freedom
DF
number of linearly independent effects that can be estimated
[SOURCE: ISO 3534-3:2013, 3.1.32, modified — the symbol ν has been replaced with the abbreviated
term DF, and the notes have been removed.]
3.6
factor
feature under examination as a potential cause of variation
[SOURCE: ISO 3534-3:2013, 3.1.5, modified — the notes have been removed.]
3.7
fixed effects analysis of variance
analysis of variance (3.4) in which the factor levels (3.8) of each factor (3.6) are preselected over the
range of values of the factors
[SOURCE: ISO 3534-3:2013, 3.3.9, modified — the note has been removed.]
3.8
factor level
setting, value or assignment of a factor (3.6)
[SOURCE: ISO 3534-3:2013, 3.1.12, modified — the notes and the example have been removed.]
3.9
factor effect
factor (3.6) that influences the response variable
[SOURCE: ISO 3534-3:2013, 3.1.14, modified — the note has been removed.]
3.10
main effect
factor effect (3.9) applicable in the context of linearly structured models (3.3) with respect to expectation
Note 1 to entry: The main effect can be estimated by averaging the response variable over all other runs provided
the experiment is fully balanced.
[SOURCE: ISO 3534-3:2013, 3.1.15, modified — Notes 1 and 3 have been removed; Note 2 has been
renumbered as Note 1 to entry.]
3.11
one-way analysis of variance
analysis of variance (3.4) in which a single factor (3.6) is investigated
3.12
two-way analysis of variance
analysis of variance (3.4) in which two distinct factors (3.6) are simultaneously investigated for possible
effects on the response variable
2 © ISO 2020 – All rights reserved

3.13
balanced data
set of data in which sample sizes are kept equal for each treatment combination
3.14
F-test
statistical test in which the test statistic has an F-distribution under the null hypothesis
3.15
p-value
probability of observing the observed test statistic value or any other value at least as unfavourable to
the null hypothesis
[SOURCE: ISO 3534-1:2006, 1.49, modified — the example and the notes have been removed.]
3.16
crossed classification
classification according to more than one attribute at the same time
Note 1 to entry: Crossed classification can be illustrated in Figure 1.
Figure 1 — Crossed classification graphic in ANOVA
3.17
interaction
influence of one factor (3.6) on one or more other factors’ impact on the response variable
[SOURCE: ISO 3534-3:2013, 3.1.17, modified — the notes have been removed.]
3.18
replication
multiple occurrences of a given treatment combination or setting of predictor variables (3.2)
[SOURCE: ISO 3534-3:2013, 3.1.36, modified — the notes have been removed.]
4 Symbols and abbreviated terms
H null hypothesis
H alternative hypothesis
DF degree of freedom
F F-statistic
SS sums of squares
MS mean squares
Adj SS adjusted sums of squares
Adj MS adjusted mean squares
5 General description of one-way and two-way classifications
5.1 General
This clause provides general guidelines to conduct the one-way and two-way analysis of variances and
illustrates the necessary steps. The formulae are shown in Annex F.
Five distinct applications illustrating the procedures are given in Annexes A through E. Each of these
examples follows the basic structure in nine steps given in Table 1.
The (common) flowchart for one-way and two-way ANOVA is given in Figure 2.
Table 1 — General ANOVA procedure
1 Stating objectives
2 Data collection plan
3 Variables description
4 Measurement system considerations
5 Performing data collection
6 Verification of ANOVA assumptions
7 Undertaking ANOVA analysis
8 Further analysis
9 Conclusion
4 © ISO 2020 – All rights reserved

Figure 2 — Common flowchart for one-way and two-way ANOVA
5.2 Stating objectives
ANOVA is used to determine if there are differences in the mean in groups of continuous data. Analysis
of variance is often used in Six Sigma projects in the ‘analyse‘ phase of DMAIC (define, measure, analyse,
improve, control) methodology. It is a statistical technique for analysing measurements depending on
several kinds of effects operating simultaneously. Analysis of variance aims for deciding which kinds of
factors are important and estimating the effects of them. It is likely to be one of the most common tests
that will be used by a Six Sigma project.
ANOVA is conducted for a variety of reasons, which include, but are not limited to:
a) assess the need for a model to represent the data;
b) test whether a factor with several levels is effective;
c) test whether two factors have an interaction, which is only applicable for two-way ANOVA;
d) test whether there is any difference between levels of some variables.
Analysis of variance examines the influence of one or two different categorical independent variables
on one continuous dependent variable. One-way ANOVA examines the equality of the means of the
continuous variable for each level of a single categorical explanatory variable. The two-way ANOVA not
only aims at assessing the main effect of each independent variable but also if there is any interaction
between them.
The analysis of variance can be presented in terms of a linear model. The objective of ANOVA is to find
the differences between the data. It provides the basis for optimizing experiment design. Additionally,
in Six Sigma, ANOVA is used to find out if there are differences in the performances of different factors.
It serves as a guide on which aspect(s) of a process improvement can, or should, be made.
5.3 Data collection plan
The data collection plan describes the relationship with the design of the experiment; refer to Factsheet
[1]
26 in ISO 13053-2:2011 for the design of the experiment. It contains the necessary steps for collecting,
characterizing, categorizing, cleaning and contextualizing the data to enable its analysis.
The data collection plan also includes how to manage data quality. Data quality establishes the set of
actions to be taken for ensuring the veracity of the data, such as integrity, completeness, timeliness and
accuracy.
After collecting the data, it is highly recommended to check it for completeness (non-missing), errors or
outliers, since these types of anomalies can distort the data.
For missing data, whether to use methods for dealing with missing data, such as imputation, or not is
decided.
NOTE In statistics, imputation is the process of replacing missing data with substituted values. Once all
missing values have been imputed, the data set can then be analysed using standard techniques for complete
data. For more details about imputation methods, see Reference [2].
5.4 Variables description
Consists of describing the response variable and the independent factors and their relationship with
the process.
5.5 Measurement system considerations
Consists of describing the measurement system analysis in place and the underlying requirements
[3]
in order to minimize the measurement system variation. For details, refer to ISO 22514-6 or
[4]
ISO/TR 12888 .
5.6 Performing data collection
Consists of performing the data collection in accordance to the data collection plan in 5.3.
6 © ISO 2020 – All rights reserved

5.7 Verification of ANOVA assumptions
5.7.1 General
Analysis of variance is used to analyse the effects of factors, which can have an impact on the result of
an experiment. This document focuses on fixed effects analysis of variance for data that satisfy three
conditions: (1) the normality assumption; (2) the assumption of homogeneity of variances; (3) the
independence of the observations.
5.7.2 Test of normality
There are two methods to test the normality of the model error: graphically and numerically,
respectively relying on visual inspection and on statistics. In order to determine normality graphically,
the output of normal probability plots, quantile-quantile plots (Q-Q plots) can be used. If the data are
normally distributed, the Q-Q plot shows a diagonal line. If the Q-Q plot shows a line in an obvious non-
linear fashion, the data are not normally distributed.
Numerically, the well-known tests of normality are the Kolmogorov-Smirnov test, the Shapiro-Wilk test,
the Cramer-von Mises test and the Anderson-Darling test. They can be combined with the graphical
[5]
analysis as performed in 5.8, refer to ISO 5479 .
NOTE When the volume of data increases (which often happens nowadays), the coefficients of Pearson
skewness and kurtosis can be taken into account. This is important because normality tests become powerful
in this case and therefore rejects the hypothesis of normality simply for a small gap. However, the assumption of
normality remains a good working hypothesis.
5.7.3 Test of homogeneity of variance
ANOVA requires that the variances of different populations be equal. This can be determined by the
following approaches: comparison of graphs (box plots); comparison of variances, standard deviations.
The F-test of two sample hypothesis test of variances can be used to determine if the variances of two
populations are equal.
5.7.4 Test of independence
ANOVA requires the independence of the observations. This can be determined, for example, by the
following approaches:
a) it can be checked by investigating the method of data collection; a pattern that is not random
suggests a lack of independence;
b) it can be evaluated by looking at the residuals against any time variables present (e.g., order of
observation), any factors;
c) it can be evaluated by looking at the auto-correlation and the Durbin-Watson statistic.
NOTE Data needs be sorted in correct order for meaningful results. For example, samples collected at the
same time would be ordered by time if it is suspected that results could depend on time.
5.7.5 Outliers identification
[6] [7]
For outliers’ identification and treatment, refer to ISO 16269-4:2010 and ISO 5725-2:1994 .
5.7.6 How to deal with non-standard cases
In many situations, the data do not fulfil all or part of the assumptions as described in 5.7.2 to 5.7.4. In
these cases, the following several options can be adopted:
— transform the data using various algorithms so that the shape of the distribution becomes normally
distributed;
— choose a nonparametric test, such as the Kruskal-Wallis H Test, which does not require the
assumption of normality.
5.8 Undertaking ANOVA analysis
5.8.1 State hypotheses H and H
0 1
State H : the equality hypothesis among subgroups.
State H : the inequality hypothesis among subgroups.
NOTE The hypotheses reflect the commonalities or the lack thereof among subgroups in business terms.
5.8.2 Graphical analysis
One can perform graphical analysis, i.e. histograms, box plots, to gain a better understanding of the
data. Graphical analysis are linked to the business context and the data generating process.
5.8.3 Generate analysis results
A generic table for ANOVA is described, see Table 2.
Table 2 — Analysis of variance table
Sums of Degrees of Variance
Variation Cause Source Type
squares freedom estimate
Factor A 1-way and 2-way
Assignable
Between Factor B 2-way
cause
Interaction 2-way
1-way
Common
Within Error
cause
2-way
Total
NOTE 1 The variance estimate is also known as mean squares.
NOTE 2 For the explicit formula in every case in Table 2 refer to Annex F. All the ANOVA tables can be
interpreted in the same way. They allow to split the aggregate variability inside the data into two parts:
assignable and common. The analysis of variance test determines whether the influence of assignable factors is
statistically significant.
5.8.4 Residual analysis
Check residuals for independence, normality and auto-correlation using graphical visualisation or by
quantitative methods. For graphical visualisation, it can be checked by residual plots. A residual plot is
a graph that shows the residuals on one axis and the independent variable on the other axis.
The best test for auto-correlation is to look at a residual time series plot (residuals vs row number). If
the plot of the residuals versus order does not show any pattern, there is no time dependence in the
residuals.
8 © ISO 2020 – All rights reserved

The test for homogeneity of variance is to look at a plot of residuals versus predicted values. If the
residuals are randomly scattered about zero and have approximately the same scatter for all fitted
values, the constant variance assumption does not appear to be violated.
The best test for normally distributed errors is a normal probability plot or normal quantile plot of the
residuals. If the points on the normal probability plot roughly follow a straight line, one can assume
that the residuals do not deviate substantially from a normal distribution.
5.9 Further analysis
When a statistically significant effect in ANOVA exists, further analysis can be implemented. A
statistically significant effect in ANOVA is often followed up with one or more different follow-up tests.
This can be done in order to assess which groups are different from other groups.
Some tests such as Tukey's range test most commonly compare every group mean with every
other group mean and typically incorporate some methods of controlling for Type I errors. Simple
comparisons compare one group mean with one other group mean. Compound comparisons typically
compare two sets of groups’ means where one set has two or more groups (e.g., compare average group
means of group A, B and C with group D).
For further analysis and model development, see References [8] and [9].
NOTE Tukey's range test, also known as the Tukey's test, Tukey method, Tukey's honest significance test, or
Tukey's HSD (honestly significant difference) test, is a single-step multiple comparison procedure and statistical
test. It can be used on raw data or in conjunction with ANOVA to find means that are significantly different from
each other.
5.10 Conclusion
Based on the results of the above analysis of variance, some conclusions of the effect of the factor on the
response variable can be obtained. With these findings, formulate a conclusion statement that links to
the ANOVA results to the project objectives given in 5.2.
6 Description of Annexes A through E
Five distinct examples of ANOVA are illustrated in Annexes A to E, which have been summarized in
Table 3 with the different aspects indicated.
Table 3 — Example summaries, by Annex
Annex Example ANOVA-details
A Bond strength Two-way ANOVA analysis: detects the factors which

have effects on the bond strength. (Germany, Minitab
Minitab 17, R 3.0, JMP 11 and Q-DAS v12
are examples of suitable products available
commercially. This information is given for the
convenience of users of this document and does
not constitute an endorsement by ISO of these
products.
17)
B Effect of script and training on in- Two-way ANOVA analysis: detects the effect of script
1)
come per sale and training on income per sale (UK, R 3.0 )
C Strength of welded joint Two-way ANOVA analysis in DOE: detects the factors
which have effects on the strength of welded joint.
1)
(India, JMP 11 )
Table 3 (continued)
Annex Example ANOVA-details
D Water consumption in Petroleum Two-way ANOVA analysis: tests whether there is sig-
enterprise nificant difference among different teams and shifts.
1)
(China, Q-DAS V12 )
E The hub total hours used on a task One-way ANOVA analysis: detects whether the hours
used on a task varied significantly by day of the weeks.
1)
(UK, Minitab 17 )
10 © ISO 2020 – All rights reserved

Annex A
(informative)
Bond strength
A.1 Stating objectives
An engineer, acting as a Six Sigma Green Belt, is planning to study the force required to separate two
components held together by an adhesive bonding agent, to find the strongest adhesive formulation.
The average pull force of the strongest adhesive formulation must be at least 50 daN.
The experimenter has assured that there are not any uncontrolled variables in the environment or been
introduced in the experiment sequence. This is a general methodological requirement that needs to be
verified prior to any analysis.
NOTE The Six Sigma Green Belt is looking to a pull test of strength. Twisting is excluded, and the engineer is
not interested in any strength such as thermal shock, and there is no other catalyst in the environment or acid in
the air. The engineer is trying to reproduce the conditions of use and not the condition of production.
A.2 Data collection plan
The engineer is considering three adhesive formulations: A, B and C. Each adhesive's ability to maintain
strength over time is important. Using each formulation, the engineer prepares 72 assemblies so that
24 samples of each formulation can be pull-tested every 3 months to assess strength degradation. The
24 samples are collected in a random fashion.
A.3 Variables description
The considered response variable is the force required to separate two components held together. The
unit of force is daN. For the sake of illustration, only pull force is considered.
There are two factors:
Factor 1: adhesive formulation, which has three levels.
Factor 2: lapsed time under extreme conditions (high temperature and humidity), which also has three
levels: the third month, the sixth month, the ninth month (with a month being defined as a period of
four weeks), because of the length of the period there might be massive change.
A.4 Measurement system considerations
Measurement error is small enough to be ignored.
A.5 Performing data collection
Table A.1 provides the raw data used in the analysis of variance.
Table A.1 — Raw data
Formulation A Formulation B Formulation C
Three Six Nine Three Six Nine Three Six Nine
months months months months months months months months months
40,8 43,3 38,6 49,4 54,2 44,0 68,5 69,6 65,7
49,0 40,7 36,6 55,3 52,2 38,8 71,1 62,5 71,7
43,4 39,6 28,1 53,4 51,4 51,3 69,1 67,5 69,9
46,4 44,0 33,6 53,2 55,4 48,0 71,9 73,1 65,2
42,6 41,3 39,3 49,6 53,5 38,4 70,1 69,4 68,1
43,5 45,5 36,9 58,1 53,7 44,9 60,9 71,5 70,7
45,4 41,4 34,9 47,6 48,4 46,1 71,3 64,0 66,2
48,4 44,8 33,3 61,9 53,6 50,2 62,1 64,4 70,7
40,3 43,2 35,1 53,5 51,5 43,1 64,6 71,5 72,4
44,2 41,3 31,5 51,7 50,1 38,6 67,6 60,1 69,8
46,5 41,7 37,5 50,5 45,5 44,3 71,3 62,5 70,8
45,6 37,2 32,5 43,0 47,0 43,6 62,9 60,2 64,6
38,4 47,3 26,8 46,2 43,3 44,6 73,1 62,3 63,1
46,8 45,9 44,9 49,6 49,0 45,3 71,9 72,2 66,8
46,8 38,4 25,9 57,5 48,6 44,4 65,7 74,6 61,2
40,6 36,4 37,4 58,4 45,2 40,8 63,6 63,3 67,5
46,5 40,6 32,8 50,4 47,1 43,4 66,5 65,9 67,1
43,1 36,9 27,0 46,8 45,4 49,8 64,0 72,5 62,0
44,3 32,6 27,5 53,5 44,5 49,1 69,1 67,6 67,7
41,8 35,1 39,7 52,8 45,6 45,2 63,7 68,1 64,0
47,2 38,4 34,1 59,6 47,9 45,3 73,3 70,9 66,1
40,5 45,1 26,7 62,4 40,7 49,8 65,9 61,4 62,6
42,7 35,2 31,5 48,9 48,9 48,3 69,9 68,6 63,7
41,3 39,3 29,4 59,5 47,1 45,1 56,2 66,2 69,0
A.6 Verification of ANOVA assumptions
A.6.1 General
Verification of ANOVA assumptions mainly includes the aspects of normality, independence,
homogeneity.
A.6.2 Test of independence
Independence can be checked by investigating the method of data collection. The experimenter has
considered the use of randomization for collecting each 24 of the 216 samples.
A.6.3 Test of normality
The normality of data is validated using the normal probability plot of residuals in a residual analysis
procedure (see Figure A.3).
The probability plot shows that normal assumption is satisfied.
12 © ISO 2020 – All rights reserved

A.6.4 Test for homogeneity of variance
1)
Next, the equivalence of variance is tested in 9 samples with 24 observations each. Minitab creates
Figure A.1.
Key
Y1 formulation
Y2 month
The Bartletts' test p-value is 0,111.
Figure A.1 — 95 % Bonferroni confidence intervals
Figure A.1 shows the 95 % Bonferroni confidence intervals for standard deviations and the p-value of
Bartlett’s test. The 95 % Bonferroni confidence intervals arise from a multiple comparison test. If two
intervals do not overlap, the difference between the corresponding standard deviations is statistically
significant. From Figure A.1, it can be seen that all of these intervals overlap. Moreover, the p-value
of Bartlett's test is 0,111, which is greater than 0,05, suggest that there is no evidence to reject the
equality of variances.
NOTE In statistics, Bartlett's test (see Reference [10]) is used to test if k samples are from populations with
equal variances.
A.7 Undertaking ANOVA analysis
The hypotheses Hand H are as follows:
0 1
H : there is no difference in the pull force among different formulations and months.
H : there is a difference in the pull force between at least one formulation and/or one month and
the others.
Then, a graphical analysis is conducted to know more about data quality.
Graphical analysis
1) Minitab is an example of a suitable product available commercially. This information is given for the convenience
of users of this document and does not constitute an endorsement by ISO of this product.
A box plot of the data is displayed for each formulation and month to assess the mean, range, and shape
of the data, see Figure A.2.
Key
Y force 4 three-B 7 three-C
1 three-A 5 six-B 8 six-C
2 six-A 6 nine-B 9 nine-C
3 nine-A
Figure A.2 — Box plot of pull force
Minitab creates each box and whisker (the line extending from each end of the box) based on the spread
of the data. The within-group variability is assessed by looking at the spread in the interquartile
range, whiskers, and outliers. The variability between formulations and bonding time is assessed by
comparing the medians and the relative locations of the interquartile ranges.
The box plots show that:
— at each bonding time, formulation A has the lowest pull force, formulation B has the second lowest
pull force, and formulation C has the highest pull force;
— the pull force from formulation C is constant over the bonding times.
ANOVA procedure
The full model is analysed, which contains the two main effects and their interaction. The factors and
the response for the model are entered. In this example, there are two factors: formulation and month.
Table A.2 shows the factor information. The response is the variable of interest — in this case, the force
required to separate components. Minitab creates the result of Table A.3.
Table A.2 — Factor information
Factor Type Levels Values
Formulation Fixed 3 A,B,C
Month Fixed 3 3,6,9
14 © ISO 2020 – All rights reserved

Table A.3 — Result of ANOVA: pull force versus formulation, month
Source DF Adj SS Adj MS F-value p-value
Month 2 1 435,9 718 42,68 0,000
Formulation 2 28 591,3 14 295,6 849,89 0,000
Formulation × month 4 731,3 182,8 10,87 0,000
Error 207 3 481,9 16,8
Total 215 3 4240,4
R-square: 89,83 %                     R-square (adjusted): 89,44 %
The p-value are used to test the significance of each term. In this model, all of the terms are significant at
the 0,05 α-level. A significant two-factor interaction indicates that the effect of a factor on the response
depends on the level of the other factor. In this example, the effect of bonding time on pull force depends
on the formulation used. Thus, it does not make sense to individually interpret the effects of month and
formulation. The model explains 89,83 % of the variability in the response.
Residual analysis
a) Normal probability plot b) Versus fits
c) Histogram d) Versus order
Key
X1 residual X2 fitted value X3 residual X4 observation order
Y1 percent Y2 residual Y3 frequency Y4 residual
Figure A.3 — Residual plots for pull force
The residual plots are used to check the assumptions about the error distribution again. Figure A.3 is
the four-in-one residual plot.
— Normal probability plot – Because the points on the normal probability plot roughly follow
a straight line, one can assume that the residuals do not deviate substantially from a normal
distribution.
— Histogram –the normal probability plot is used to make a decision about the normality of the
residuals. With a reasonably large sample size, the histogram displays compatible information.
— Versus fits – The constant variance assumption does not appear to be violated because the residuals
are randomly scattered about zero and have approximately the same scatter for all fitted values.
— Versus order – The plot of the residuals versus order does not show any pattern. Therefore, there
is no time dependence in the residuals.
Understanding the effects
Minitab provides two plots to help visualize the effects.
— Main effects plots show the relative strength and the direction of the effects. Figure A.4 is the main
effects plot for pull force.
— Interaction plots show how the level of one factor influences the effect of another factor. Figure A.5
is the interaction plot for pull force. Because the interaction between formulation and month is
significant, only the interaction plot is displayed.
Key
X1 formulation
X2 month
Y mean of pull force
Figure A.4 — Main effects plot for pull force
16 © ISO 2020 – All rights reserved

Key
X1 formulation 1 formulation A 4 month 3
X2 month 2 formulation B 5 month 6
Y force 3 formulation C 6 month 9
Figure A.5 — Interaction plot for pull force
The interaction plot shows how the level of one factor influences the effect of another factor. The lines
on the plot are not parallel, which suggest there is an interaction. The p-value in the ANOVA table is
0,000 which shows that the interaction is significant.
This plot also shows that, on average:
— formulation C has the highest pull force;
— there seems to be no difference between the bonding times for formulation C;
— the pull force decreases over time for both formulations A and B.
A.8 Further analysis
Multiple comparisons
The p-values in the ANOVA table (Table A.3) indicate differences between the factor level means, but
do not identify the means that differ. To determine which mean differences are significant, one can test
them using one of the comparison methods.
Tukey's method compares all possible pairs of level means of the specified factors and interaction
terms. Table A.4 is the group information table.
Table A.4 — Grou
...