ISO 16336:2014
(Main)Applications of statistical and related methods to new technology and product development process — Robust parameter design (RPD)
Applications of statistical and related methods to new technology and product development process — Robust parameter design (RPD)
ISO 16336:2014 gives guidelines for applying the optimization method of robust parameter design, also called as parameter design, an effective methodology for optimization based on Taguchi Methods, to achieve robust products. ISO 16336:2014 prescribes signal-to-noise ratio (hereafter SN ratio) as a measure of robustness, and the procedures of parameter design to design robust products utilizing this measure. The word "robust" in this International Standard means minimized variability of product's function under various noise conditions, that is, insensitivity of the product's function to the changes in the levels of noises. For robust products, their responses are sensitive to signal and insensitive to noises. The approach of ISO 16336:2014 can be applied to any products that are designed and manufactured, including machines, chemical products, electronics, foods, consumer goods, software, new materials, and services. Manufacturing technologies are also regarded as products that are used by manufacturing processes.
Application de méthodologies statistiques et connexes pour le développement de nouvelles technologies et de nouveaux produits — Modèle paramétrique robuste
General Information
Standards Content (Sample)
DRAFT INTERNATIONAL STANDARD ISO/DIS 16336
ISO/TC 69/SC 8 Secretariat: JISC
Voting begins on Voting terminates on
2013-02-16 2013-05-16
INTERNATIONAL ORGANIZATION FOR STANDARDIZATION • МЕЖДУНАРОДНАЯ ОРГАНИЗАЦИЯ ПО СТАНДАРТИЗАЦИИ • ORGANISATION INTERNATIONALE DE NORMALISATION
Robust parameter design (RPD)
Plan paramètre robuste (RPD)
ICS 03.120.30
To expedite distribution, this document is circulated as received from the committee
secretariat. ISO Central Secretariat work of editing and text composition will be undertaken at
publication stage.
Pour accélérer la distribution, le présent document est distribué tel qu'il est parvenu du
secrétariat du comité. Le travail de rédaction et de composition de texte sera effectué au
Secrétariat central de l'ISO au stade de publication.
THIS DOCUMENT IS A DRAFT CIRCULATED FOR COMMENT AND APPROVAL. IT IS THEREFORE SUBJECT TO CHANGE AND MAY NOT BE
REFERRED TO AS AN INTERNATIONAL STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS BEING ACCEPTABLE FOR INDUSTRIAL, TECHNOLOGICAL, COMMERCIAL AND USER PURPOSES, DRAFT
INTERNATIONAL STANDARDS MAY ON OCCASION HAVE TO BE CONSIDERED IN THE LIGHT OF THEIR POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN NATIONAL REGULATIONS.
RECIPIENTS OF THIS DRAFT ARE INVITED TO SUBMIT, WITH THEIR COMMENTS, NOTIFICATION OF ANY RELEVANT PATENT RIGHTS OF WHICH
THEY ARE AWARE AND TO PROVIDE SUPPORTING DOCUMENTATION.
© International Organization for Standardization, 2013
ISO/DIS 16336
Copyright notice
This ISO document is a Draft International Standard and is copyright-protected by ISO. Except as permitted
under the applicable laws of the user’s country, neither this ISO draft nor any extract from it may be
reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic,
photocopying, recording or otherwise, without prior written permission being secured.
Requests for permission to reproduce should be addressed to either ISO at the address below or ISO’s
member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Reproduction may be subject to royalty payments or a licensing agreement.
Violators may be prosecuted.
ii © ISO 2013 – All rights reserved
ISO/DIS 16336
Contents Page
Foreword . iv
Introduction . v
1 Scope . 1
2 Normative references . 2
3 Terms and definitions . 2
4 Parameter Design for Robust Products — Overview . 4
4.1 Parameter Design for Robust Products — Requirements . 4
4.2 Assessing the robustness (SN ratio) of a system . 4
4.3 Utilization of robustness assessment . 5
4.4 An efficient method for assessing technical ideas — parameter design . 6
4.5 Two-step optimization (Strategy for parameter design) . 7
4.6 Determination of optimum design . 9
5 Assessment of robustness by SN ratio . 9
5.1 Concepts of SN ratio . 9
5.2 Types of SN ratio . 10
5.3 Procedure of the assessment and quantification of robustness . 10
5.4 Formulation of SN ratio: Computation using decomposition of total sum of squares . 11
5.4.1 Zero-point proportional equation ideal function (Dynamic characteristic) . 11
5.4.2 Linear equation ideal function (Dynamic characteristic) . 13
5.4.3 Reference point proportional ideal function (Dynamic characteristic) . 15
5.4.4 Nominal-the-best response (Static/non-dynamic characteristic) . 15
5.4.5 Smaller-the-better response (Static/non-dynamic characteristic) . 16
5.4.6 Larger-the-better response (Static/non-dynamic characteristic) . 16
5.4.7 SN ratio for digital characteristics . 17
5.5 Some topics of SN ratio . 17
5.5.1 Using SN ratios to compare systems . 17
5.5.2 Nonlinear signal and output response cases . 18
5.5.3 SN ratios of static/non-dynamic characteristics . 18
6 Procedure of parameter design . 18
7 Case study - Parameter design of a lamp cooling system . 28
Annex A (informative) Comparison of system robustness by SN ratio . 38
Annex B (informative) Examples and SN ratio in various technical fields . 45
Bibliography . 70
ISO/DIS 16336
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies
(ISO member bodies). The work of preparing International Standards is normally carried out through ISO
technical committees. Each member body interested in a subject for which a technical committee has been
established has the right to be represented on that committee. International organizations, governmental and
non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the
International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of technical committees is to prepare International Standards. Draft International Standards
adopted by the technical committees are circulated to the member bodies for voting. Publication as an
International Standard requires approval by at least 75 % of the member bodies casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO shall not be held responsible for identifying any or all such patent rights.
ISO 16336 was prepared by Technical Committee ISO/TC 69, Applications of statistical methods,
Subcommittee SC 8, Application of statistical and related methodology for new technology and product
development.
iv © ISO 2002 – All rights reserved
ISO/DIS 16336
Introduction
Parameter design can be applied in product design stage to identify optimum nominal values of design
parameters based on assessment of robustness of its function. Robustness assessment should be
performed as a consideration of overall loss during the product’s life cycle. The overall loss is composed of
costs and losses at each stage of the product's life. It should include all the costs incurred during not only its
production stage, but also its disposal stages.
When a product is not robust, the product causes many environmental and social economic losses (including
losses to the manufacturer and users) due to its poor quality caused by functional variability throughout its
usable lifetime from shipping to final disposal. Product suppliers should have responsibilities and obligations to
supply robust products to the market to avert losses and damages resulting from defects in the products.
At product development and design stages, the product suppliers should therefore anticipate the defects and
failures of products in the market by applying preventative measures, and also should design robust products
by optimizing their design from the point of view of robustness.
At manufacturing stage, the product suppliers should manufacture their products that meet the product
specifications. One can optimize manufacturing processes to produce the products that meet the
specifications. However, robustness against customer’s environment and products’ aging can only be
addressed by product design.
Parameter design methodology provides effective methods for achieving robustness through its design of
specification determination, and it is a preventive counter measure against various losses in the market.
Parameter Design for Robust Products, this document, is directly targeted at losses incurred in the usage
stage. Where possible, losses at other stages are also investigated so that the results of parameter design
can be used to perform optimum product design for the whole of the product's life cycle.
DRAFT INTERNATIONAL STANDARD ISO/D,6 16336
Parameter Design for Robust Products
1 Scope
This document gives a guidance of applying the optimization method of parameter design, an effective
methodology for optimization based on Taguchi Methods, to achieve robust products.
Aim of applying parameter design in product design is to prevent defects, failures and quality problems that
may occur during the usage of the product, and to minimize the loss in the market. Robust product, output of
parameter design, is a product which is designed such a way to minimize the user’s loss caused by defects,
failures and quality problems. One should note that defects, failures and quality problems are caused by
functional variability of non-robust product. In the parameter design, optimum nominal values of product’s
design parameters can be selected by treating product’s design parameters as control factors and by
assessing robustness under noise factors. The use of parameter design at the development and design
stages makes it possible to determine the optimum product design and specification so that the product is
more robust in the market. Choice of noise factors strongly depends on the market of the product.
This document prescribes signal-to-noise ratio (hereafter SN ratio) as a measure of robustness and the
procedures of parameter design to design robust products utilizing this measure. The word “robust” in this
document means minimized variability of product’s function under various noise conditions, that is,
insensitivity of the product’s function to the changes in the level of noises. For robust product, its response
should be sensitive to signal, and insensitive to noises.
The robustness of a product should essentially be quantified in terms of the economical losses caused by
variability of product’s function at the usage stage. Accordingly, when the robustness of a product is
estimated at the development and design stages, the designer should forecast and calculate the future
economical losses in the market where the product will be used. However, it is often difficult to perform
concrete evaluation of future losses at the development and design stages.
In practice, many product’s defects and failures occur mainly due to the product’s characteristics that deviate
from or vary around the designed target values due to the change in usage environment and deterioration, i.e.
noise conditions. The variability of product’s response should be used as a measure of robustness, because
market losses increase in proportion to the magnitude of variability of product’s characteristics due to noise
effects. The variability due to noises includes the deviation from the designed target value and the variation
around the designed target value in market. SN ratio, corresponding to the inverse of the variation measure,
should be used as a measure of goodness in robustness. In other words, the inverse of SN ratio is
proportional to the market losses.
For the experimental plan of parameter design, direct product of inner array and outer arrays is proposed.
Control factors should be assigned to inner array, and signal and noise factors should be assigned to outer
array. By using a direct product plan, all the first level interactions between control factors and noise factors
can be assessed and can be utilized to select the optimum level of control factors from the point of view of
robustness.
Assessing robustness through SN ratio is a key of parameter design. Outer array is for evaluating SN ratio,
robustness, for each combination of levels of control factors indicated by the inner array. Inner array is for
comparing SN ratios and selecting optimum combination of system’s design parameters. While experimental
layout for inner array may have many variations, orthogonal array L is strongly recommended and then only
the application of orthogonal array L is discussed in this document.
ISO/DIS 16336
The approach of this document can be applied to any products that are designed and manufactured, including
machines, chemical products, electronics, foods, consumer goods, software, new material, services.
Manufacturing technologies are also regarded as products that are used by manufacturing processes.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO 3534-1, Statistics — Vocabulary and symbols — Part 1: Probability and general statistical terms
ISO 3534-3, Statistics — Vocabulary and symbols — Part 3: Design of experiments
3 Terms and definitions
For the purposes of this document, the terms and definitions given in ISO 3534-1, ISO 3534-3 and the
following apply.
3.1
function:
work which a system performs in order to fulfil its objective.
Note Function can be expressed by mathematical form of input-output relation.
3.2
robustness:
degree of smallness in variability of system’s function under various noise conditions.
Note System’s performance can be assessed by robustness. SN ratio is a quantitative measure of robustness.
3.3
signal-to-noise ratio
SN ratio
ratio of useful effect to harmful effect in response variations.
Note 1 SN ratio is usually expressed in db value. The notation of db is used instead of dB for SN ratios of robustness
measure.
Note 2 Anti-logarithm value of SN ratio, real number, is inverse of variation measure such as variance or a coefficient of
variation, and inversely proportional to the monetary loss.
Note 3 The change in response caused by intentional change of signal value is useful effect. In case of the ideal
function being zero point proportional, the linear slope forced through the zero point is the useful term.
Note 4 The change in response caused by noise factors is harmful effect. Effect of noise factor and deviation from the
ideal function are examples.
Note 5 SN ratio should contain the variability under noise factors and the discrepancy from the ideal function under
average usage condition.
3.4
sensitivity
amount of change in response caused by unit change of input.
Note 1 Sensitivity is usually expressed in db value.
2 © ISO 2002 – All rights reserved
ISO/DIS 16336
Note 2 For the dynamic characteristic cases, sensitivity shows the magnitude of linear coefficient due to signal, β ,
where β is proportional constant.
Note 3 For the nominal-the-best response, the sensitivity shows the magnitude of mean, m , where m is the average of
response.
3.5
noise
variable which disturbs the system’s function.
Note 1 User’s condition contains noise and signal.
Note 2 Noise is composed of internal noise, such as aging and wear, and external noise, such as environment during
usage. They are sometime called as capacity and demand respectively. Deterioration and change of internal constant
of the system over time or in parts are examples of internal noises. Usage conditions and environment conditions of the
product are examples of external noises. Three categories of noise factors are: 1) Environment, 2) Aging and changes
over time, 3) Manufacturing Variations.
3.6
signal
input variable to the system, which is intentionally changed by user to get an intended value of response in
input-output relation
Note 1 User’s condition contains signal and noise. Any variable/factor at user’s conditions is either a signal or a noise
factor
Note 2 There are two kinds of signal; active signal and passive signal. Active signal is operated by user to get
intended response, for example, rotating angle of steering wheel to change the vehicle’s direction. Passive signal is used
by user to know the value of input from response reading, for example, temperature in thermal measurement. In both
cases, output will change by changing the value of signal but user wants to get response value in active case, and user
wants to know the value of signal in passive case.
3.7
dynamic characteristics
output response which has the ideal target values depending on the value of signal.
Note The relation between dynamic characteristics and signal can be expressed by input-output functional form.
Output of system’s function is dynamic characteristics in many cases.
3.8
static characteristics
non-dynamic characteristics
output response which has a fixed target value.
Note Static characteristics can be categorized into three groups depending on the target value; nominal-the-best,
smaller-the-better, and lager-the-better characteristics, where target value is finite value, zero, and infinity respectively.
3.9
inner array
experimental plan where design parameters are assigned as control factors and indicative factors. Each
treatment run will be assessed for robustness using SN ratio and sensitivity.
Note 1 Orthogonal arrays are recommendable for inner array in parameter design, because many design parameters
can be taken into consideration in one set of experiment.
Note 2 Experimental factors should be categorized by their roles and assigned separately to inner array and outer array
in parameter design.
ISO/DIS 16336
3.10
outer array
experimental plan where users’ conditions are assigned as noise factors and signal factors for evaluating SN
ratio and sensitivity.
Note 1 Orthogonal arrays are recommended to assign control factors, i.e. inner array in parameter design, because
many design parameters (control factors) can be taken into consideration in one set of experiment.
Note 2 Experimental factors should be categorized by their roles and assigned separately to inner array and outer array
in parameter design.
4 Parameter Design for Robust Products — Overview
4.1 Parameter Design for Robust Products — Requirements
Parameter Design for Robust Products (PDRP) is a rational and efficient procedure for discovering technical
means to improve robustness in designing process. It is therefore necessary to provide the following two
procedures:
a) A procedure for the accurate assessment and quantification of robustness
b) A procedure for the efficient assessment of multiple technical means (ideas)
This clause provides the approach to achieving the goals of PDRP, and more detailed and specific steps of
the PDRP procedure are described in clauses 5 and 6.
4.2 Assessing the robustness (SN ratio) of a system
How robustness should be accurately assessed by SN ratio (item a) )? Robustness is associated with many
factors, so it could not be assessed by a simple measurement. To clarify hidden properties associated with
robustness, the problem should be approached from the following two viewpoints.
a) Use of Ideal Function: Ideal function is the target of the system. Actual function of the system should be
measured and evaluated based on the definition of ideal function. It is important to avoid defects, failure
mode or quality problems for the ideal function of the system.
b) Use of Noise Factors: Noise effects should be intentionally introduced by changing noise levels and the
function of the system should be measured and evaluated under those predetermined noise conditions.
Evaluation of robustness strongly depends on the choice of noise factors and their levels. Apply effective
noise strategies.
The function of a system is the work that the system performs in order to fulfil its objective. For example, the
function of an electric lamp is to transform electrical energy into light energy, and the function of a wind turbine
is to use natural wind energy to rotating energy to perform a work such as pumping water. These functions are
normally expressed in the functional form of an input/output relationship between the input energy and the
output energy as the work obtained. Functional form can be expressed in a number of ways. Zero-point
proportional equation is common in energy transformation of real physical systems. The details will be
discussed in Clause 5.
The input and the output characteristics are fixed based on system’s function. Input characteristic is called
the "signal" in the input/output relationship, this is because the changes in the output are acquired by
intentional varying the input signal in the real usage and in the PDRP experiment. The signal is energy or
information necessary to do a work and the signal factor is a user’s condition for changing signal when the
user of the system tries to control the output. Output is also called as response. Because of its variableness,
the input signal changes 3-4 levels and can usually measure the straightness of the ideal linear relationship
for a dynamic characteristic. There is no signal factor for a static characteristic, because it has only one target
output.
4 © ISO 2002 – All rights reserved
ISO/DIS 16336
It is important to find out the suitable measurement method of output characteristic. In time dependent
phenomena, for example, detection of output characteristic is difficult. New measurement method should be
developed in some cases. The output characteristic is the purpose of the system. In the case of illumination,
output is quantity of light, and in the case of a water pump, it is quantity of water. The measurement method
which can measure the output quantity efficiently is desirable.
Noise conditions are the sources that inhibit the system's function (i.e., its input/output relation). Examples
include environmental conditions such as the temperature or humidity range in which the system is actually
used, the actual supplied voltage or electrical noise conditions, the frequency of operations and/or the stress,
and the length of time for which the system is in operation or left idle after it has been purchased. These noise
conditions always reduce the system's function to a lower level than the level at the time of purchase. Since
the purpose of PDRP is to clarify the differences in robustness by measuring the extent of this reduction, the
variation of system’s function under noise conditions should be clarified in the evaluation in the PDRP. This
is the reason why noise conditions should be taken into the PDRP as noise factors. For the effective noise
strategy, various types of noise should be examined in actual usage and environment conditions and aging
assessing.
Figure 1 shows an overview of the evaluation of robustness using function and noise factors. Here, multiple
data should be obtained for the objective system under the various levels of noise factor and an SN ratio is
calculated from the mean and the variance of these data. SN ratio as a robustness measure should be
calculated using data X to X . Formulations of SN ratio are shown in clause 5. When more than two systems
1 k
are compared, the same levels of the same noise factor should be applied for all objective systems.
Noise levels
N N … N
1 2 k
Evaluated system X X … X
1 2 k
N …Noise level
i
X …Data
i
Figure 1 — Robustness assessment with noise conditions
When multiple different types of noise factors are applied in an experiment, the orthogonal array can be used
for determining the noise levels. In Figure 2, noise levels, N to N , is determined by the combination of levels
1 k
of each noise factor indicated by orthogonal array.
Orthogonal array for noise factors
1 2 3 4 … k
A : Ambient temperature, humidity …
B : New product and degraded product …
C : Usage frequency …
N N N N … N
1 2 3 4 k
Evaluated system X X X X … X
1 2 3 4 k
A, B, C…Noise factor
Figure 2 — Robustness assessment with noise factors assigned to an orthogonal array
4.3 Utilization of robustness assessment
The ideal function of the system and the noise strategy are the key issues for the robustness assessment of
PDRP. , It is essential to measure and evaluate the variability and efficiency of basic function of the system.
The intent of doing this is that the evaluation of function covers wholly the technical issues of the system to
ISO/DIS 16336
prevent the technical problems. Robustness assessment also involves assessing the effects of noise factors
that inhibit the required function. The assessment results are expressed by SN ratio.
SN ratio should discriminate the true difference in robustness between designs. Since the absolute value of
SN ratio is affected by the setting levels of noise factors, only relative comparison of SN ratios is meaningful to
perform. Thus, it is preferable to perform benchmarking in assessing robustness.
A feature of this approach is that the only information needed to obtain the SN ratio is just the knowledge of
the ideal function of the system and the noise conditions that inhibit this function, and that no detailed
technical information about the system is needed. This means that SN ratios should be calculated in the
same way for various systems with different technical constituents as long as the systems have the same
basic function. As the robustness of systems can be accurately assessed by SN ratios, then the robustness
of various systems with different design concepts can be assessed and compared.
The comparison of various systems based on different technologies or different design concepts should be
performed in the same way, such as a conventional system and a competing system that uses different
design concepts. This is the idea to conduct benchmarking on various designs by using SN ratios.
4.4 An efficient method for assessing technical ideas — parameter design
Basic technologies and mechanisms should first be selected to design a system of industrial products. When
there are multiple system concepts to be benchmarked, the robustness assessment introduced in the previous
section can be used to select the best design concept.
After selecting the design concept, the next step is to perform a detailed design by selecting the optimum
nominal values of system design parameters so that the intended function is the most robust and efficient. .
The system design parameter optimization method performed at this detailed design stage is called
Parameter Design for Robust Products (PDRP), which is the method recommended by this document.
Consider what sort of states might be significant. When a system is in an optimum state, it achieves the best
overall performance in all conceivable usage states. More specifically, an industrial product can stably perform
its intended function anytime even when it is kept under a wide range of temperature and humidity conditions,
and when it is used in many different ways and in different environments. The optimum design conditions are
the combination of design parameters that are set to maximize the robustness of the product. Since
optimization by parameter design implies optimizing for robustness, minimized variability and maximized
efficiency, judgments should be made using robustness measure, SN ratio.
The basis of optimization to apply the robustness assessment in the previous section is a criterion for
optimization. This means that robustness assessment should be performed with regard to all the possible
designs in design space, but in practice this is impossible. This is because a vast number of tests would have
to be performed to take all possible combinations of design parameters into consideration.
As a more practical method for use in development and design stage, an experimental method using an
orthogonal array is recommended where the combinations of many design parameters can be tested under
limited number of experimental runs. An orthogonal array is recommended not only because it can reduce the
number of experimental runs comparing with full factorial plan with same number of control factors but also
because it can assign maximum number of control factors in a set under the situation of same number of
experimental runs. Reliability of experimental results should be confirmed in the confirmation experiment for
reproducibility checks. Clause 6 describes a specific method for performing confirmation experiment to check
reproducibility.
Procedure of PDRP should be as follows;
[Step 1] Clarify the system’s ideal function.
[Step 2] Select signal factor and its range,
[Step 3] Select measurement method of output response.
6 © ISO 2002 – All rights reserved
ISO/DIS 16336
[Step 4] Develop noise strategy, and select noise factors and their levels.
[Step 5] Select control factors and their levels from design parameters.
[Step 6] Assign experimental factors to inner or outer array.
[Step 7] Conduct experiment and collect data.
[Step 8] Calculate SN ratio η and sensitivity S.
[Step 9] Generate factorial effect diagrams on SN ratio and sensitivity.
[Step 10] Select the optimum condition.
[Step 11] Estimate the improvement by the gain.
[Step 12] Conduct confirmation experiment and check the gain and “reproducibility”.
4.5 Two-step optimization (Strategy for parameter design)
Figure 3 presents an overview of PDRP as described above. The experiment in this figure includes two
orthogonal arrays, one orthogonal array for control factors (inner array) and the other orthogonal array for
noise factors (outer array). This is called a direct product plan. The number of experimental data
corresponds to the product of the numbers of tests respectively specified in the two orthogonal arrays. For
example, for the combination of L and L , the number of tests comes to 18×12=216. Something like this is
18 12
very doable in case of computer simulation.
Full factorial plan may be used for outer array for noise factors instead of orthogonal array in some cases. Or,
in case of physical test, it should be recommended to compound noise factors into one 2 or 3 level factor.
However, it is always recommended to use an orthogonal array for inner array for design parameters,
because many design parameters can be assigned to one orthogonal array.
Outer orthogonal array for noise factors
1 2 3 4 … k
A: temperature
B: humidity
C: …….
N N N N … N Mean value SN ratio
1 2 3 4 k
…
…
Inner orthogonal array for design
parameters
ISO/DIS 16336
…
…
…
Figure 3 — Direct product plan for parameter design
The experimental data obtained for each combination of design parameters consist of multiple data with the
corresponding number of noise factor levels. To find out the optimum values of design parameters for
robustness, the sensitivity (mean value in case of Nominal-the-Best response) and the SN ratio should be
calculated for each line of inner array, that is, combination of design parameter. Then the factorial effects of
design parameters on sensitivity and SN ratio should also be calculated, and they are summarized in factorial
effect diagrams as shown in Figures 4 and 5. Specific calculation formulas are described in Clause 6. The
optimum values of design parameters are selected using the diagram for sensitivity and SN ratios. Sensitivity
represents the mean value of the data set (in case of static characteristics), and SN ratio represents
robustness. This so-called "two-step" design procedure is a very important concept for the design of
robustness.
The factorial effect diagram shows how the system’s function is affected by each design parameter
incorporated into the experiment. If a factor has a large gradient, it has a large effect on the system’s
function. The two types of factorial effect diagram represent the degree of influence relating to SN ratios and
sensitivity respectively. An important point in two-step design is to pay more attention to SN ratios than to
sensitivity. In the first step, the levels of design parameters should be selected to maximize the SN ratio in the
factorial effect diagram on SN ratios (Figure 4), and then, in the second step, typically just one design
parameter should be used to adjust the mean value — i.e., the sensitivity — to the target value. For this
adjustment, it is desired to use a factor with minimal effect on SN ratio. The first step is to optimize the
design for robustness by SN ratio, and the second step is for adjusting the mean to the target value by
sensitivity. This is the reason why PDRP is called as two-step optimization.
36.0
32.0
28.0
24.0
A1 A2 B1 B2 B3 C1 C2 C3 D1 D2 D3 E1 E2 E3 F1 F2 F3 G1 G2 G3 H1 H2 H3
Figure 4 — Factorial effect diagram on SN ratio
(robustness)
0.0
-2.0
-4.0
-6.0
A1 A2 B1 B2 B3 C1 C2 C3 D1 D2 D3 E1 E2 E3 F1 F2 F3 G1 G2 G3 H1 H2 H3
Figure 5 — Factorial effect diagram on sensitivity (mean value)
Why is two-step optimization important? Which is more difficult — SN ratio design or sensitivity design?
8 © ISO 2002 – All rights reserved
SN ratio(db)
Sensitivity(db)
ISO/DIS 16336
The order of optimizations is important to design the robustness reliably. Consider an example of the case of
voice recording. If audio data is recorded in place where there is a lot of background noise, then adjusting
the volume when it is played back will hardly make the recorded sound any easier to listen to. To extract the
information buried in the noise, techniques such as noise reduction to cancel the effects of the noise or use a
microphone that is less sensitive to background noise must be employed. Improving SN ratio requires
advanced technical ability and counter measure in recording. On the other hand, when the mean recording
level is too low, it can easily be improved by adjusting the volume during playback. The controlling of the
mean value of the playback sound by volume can be often accomplished by a relatively simple method in this
way. Adjusting the mean value can be performed by sensitivity.
Another example is visual imaging function such as photographs and video pictures. The average tone level
can easily be corrected, but pictures taken in dark conditions are often very noisy and results in a low-quality
picture. There are also limits to how much the picture quality can be improved by image processing.
It is relatively easy to achieve adjusting the mean value, because it usually takes just one parameter. It is easy
to adjust the average energy level. On the other hand, it is not easy to improve the robustness. It is desired
to have as many control factors as possible. In designing a system, it is therefore wise to give greater priority
to setting the optimum levels of design parameters to maximize SN ratio. This forms the foundation of the two-
step optimization where robustness optimization by SN ratio should have the first priority.
4.6 Determination of optimum design
Once parameter design experiment clarifies which design parameters affect the SN ratio and which design
parameters affect the sensitivity, then the final optimum design should be determined by considering other
constraints such as cost and delivery requirements.
Thus, since a system's overall optimum design are determined by an overall balance of many constraints, it is
preferable to prepare experimental data in which the individual experimental factors cover a wider range within
the design space. Since improving robustness is particularly difficult, it is recommended to take rather wide
range of control factor levels within the design space.
In parameter design, robustness optimization is performed through maximizing SN ratio. It means minimizing
user’s loss after product is shipped. SN ratio is the quantitative measure of user’s loss due to defects,
failures and quality problems caused variability or lack of robustness. User’s loss includes losses by
malfunction and defects, additional maintenance cost, etc.
According to Taguchi’s quality loss function, SN ratio can be transformed to user’s loss in the monetary unit
and the total loss to society caused by the product can be derived by adding other cost, such as product
development cost, material cost, production cost, shipping cost, normal maintenance cost, disposal cost etc.
Total loss should be the quality measure of the product. In the product design stage, product designer
should consider the total loss from the view point of technology. However it is difficult for designer to forecast
the social loss in design stage, but the designer should, at least, assess and optimize the product design from
the view point of robustness.
5 Assessment of robustness by SN ratio
5.1 Concepts of SN ratio
The variability in function of a system should be assessed and optimized by parameter design for designing a
robust product. When a subsystem is to be assessed for robustness, one should consider noise conditions at
the whole system level in users’ hand. It is critical to assure robustness at the system level.
A system's function should be defined as the functional form of input/output relationship in the usage of the
system. Users manipulate a signal to get an intended output of the system. Signal is an input characteristic
that is intentionally set to vary the output of the system. A functional form that represents the ideal
input/output relationship of a system's function is called as the system's ideal function. However, in actual
usage conditions, the function is deviated from the ideal function due to noise conditions. Deviation from the
ISO/DIS 16336
ideal function should be evaluated and expressed by one numerical measure in the first step of the PDRP
optimization that is the step to assess the robustness using signal-to-noise ratio (SN ratio).
The user’s conditions, under which the system is actually used, contain only signal and noises. As
mentioned above, the signal is input to the system that is set intentionally to change the output of a system.
The signal has large effect on the system's output. On the other hand, the effect of the noises has a negative
impact on the system’s output. The effect of signal should be maximized and the effect of noises should be
minimized. In the experiment to assess the robustness using SN ratio, characteristic for input should be
treated as a signal factor and noise sources should be treated as noise factors. Categorizing the variables in
user’s condition is important to clarify the purpose of experiment.
The SN ratio is an index that quantitatively expresses how closely the actual signal/output relationship is
departed from the ideal functional form under various noise conditions. As SN ratio increases, the actual
signal/output relationship becomes closer to the ideal, and the loss to society will decrease. In the opposite
case, it becomes farther to the ideal, and the loss to society will increase.
5.2 Types of SN ratio
There are three types of SN ratio in Parameter Design for Robust Products, that is, SN ratio for dynamic
characteristic, SN ratio for static or non-dynamic characteristic, and SN ratio for digital system.
SN ratio for dynamic characteristic is to consider the stability/robustness of the relationship between the signal
and the corresponding outputs. The SN ratios for dynamic characteristics can be broadly subdivided into
three types by the functional forms of system’s ideal function — zero-point proportional ideal function,
reference-point proportional ideal function, and linear equation ideal function. The choice of the functional
form of ideal function depends on the physics of the intended system. In the majority of cases, the ideal
function can be expressed by a zero-point proportional equation.
SN ratio for static or non-dynamic, characteristic is to consider the stability of the fixed outputs (the signal is
constant). The SN ratios of static characteristics can be broadly subdivided into three types by the values of
system’s fixed target — nominal-the-best, smaller-the-better, and larger-the-better. The choice of the fixed
target depends on the system’s intent. The value of fixed target is finite value for nominal-the-best, zero for
smaller-the-better, and infinity for larger-the-better.
SN ratio for digital system is used in the case of binary outputs that can only take a value of 0 or 1 (e.g.,
computer control systems),
Procedures to formulate each type of SN ratio will be shown in the following sections.
5.3 Procedure of the assessment and quantification of robustness
The procedure to formulate SN ratio should be as follows;
[Step 1] Clarify the system’s ideal function.
Function is a work that system performs in order to fulfil its objective. Function has input signal to represent
the operator’s intention in dynamic case. Output characteristics or response of the system is varied by the
input signal to fulfil the system’s objective. Function can be expressed by a mathematical form of the relation
between the input signal and the output characteristics/response.
Define ideal function, i.e. ideal relationship between the input signal and the output characteristics/response
based on the system’s functi
...
INTERNATIONAL ISO
STANDARD 16336
First edition
2014-07-01
Applications of statistical and related
methods to new technology and
product development process —
Robust parameter design (RPD)
Application de méthodologies statistiques et connexes pour le
développement de nouvelles technologies et de nouveaux produits —
Modèle paramétrique robuste
Reference number
©
ISO 2014
© ISO 2014
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO’s member body in the country of
the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO 2014 – All rights reserved
Contents Page
Foreword .iv
Introduction .v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions and symbols. 1
3.1 Term and definitions . 1
3.2 Symbols . 3
4 Robust parameter design — Overview . 4
4.1 Requirements . 4
4.2 Assessing the robustness of a system . 4
4.3 Robustness assessment through SN ratio . 6
4.4 An efficient method for assessing technical ideas — Parameter design . 7
4.5 Two-step optimization (Strategy of parameter design) . 8
4.6 Determination of the optimum design .10
5 Assessment of robustness by SN ratio .10
5.1 Concepts of SN ratio .10
5.2 Types of SN ratio .11
5.3 Procedure of the quantification of robustness .11
5.4 Formulation of SN ratio: Calculation using decomposition of total sum of squares .13
5.5 Some topics of SN ratio .19
6 Procedure of a parameter design experiment .20
6.1 General .20
6.2 (Step 1) Clarify the system’s ideal function .20
6.3 (Step 2) Select a signal factor and its range .21
6.4 (Step 3) Select measurement method of output response.21
6.5 (Step 4) Develop noise strategy and select noise factors and their levels .21
6.6 (Step 5) Select control factors and their levels from design parameters .22
6.7 (Step 6) Assign experimental factors to inner or outer array .22
6.8 (Step 7) Conduct experiment and collect data .23
6.9 (Step 8) Calculate SN ratio, η, and sensitivity, S . 23
6.10 (Step 9) Generate factorial effect diagrams on SN ratio and sensitivity .26
6.11 (Step 10) Select the optimum condition .28
6.12 (Step 11) Estimate the improvement in robustness by the gain .28
6.13 (Step 12) Conduct a confirmation experiment and check the gain and “reproducibility” .29
7 Case study — Parameter design of a lamp cooling system .30
Annex A (informative) Comparison of a system’s robustness using SN ratio .40
Annex B (informative) Case studies and SN ratio in various technical fields .47
Bibliography .72
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation on the meaning of ISO specific terms and expressions related to conformity
assessment, as well as information about ISO’s adherence to the WTO principles in the Technical Barriers
to Trade (TBT) see the following URL: Foreword - Supplementary information
The committee responsible for this document is ISO/TC 69, Applications of statistical methods,
Subcommittee SC 8, Application of statistical and related methodology for new technology and product
development.
iv © ISO 2014 – All rights reserved
Introduction
Robust parameter design, also called parameter design, can be applied in product design stage to
identify the optimum nominal values of design parameters based on the assessment of robustness of its
function. Robustness assessment is performed as a consideration of overall loss during the product’s life
cycle. The overall loss is composed of costs and losses at each stage of the product’s life. It includes all the
costs incurred during not only its production stage, but also its disposal stages.
When a product is not robust, the product causes many environmental and social economic losses
(including losses to the manufacturer and the users) due to its poor quality caused by functional
variability throughout its usable lifetime from shipping to final disposal. Product suppliers have
responsibilities and obligations to supply robust products to the market to avert losses and damages
resulting from defects in the products.
The aim of applying parameter design in product design is to prevent defects, failures, and quality
problems that can occur during the usage of the product. A robust product, an output of parameter
design, is a product which is designed in such a way as to minimize user’s quality losses caused by
defects, failures, and quality problems. Note that defects, failures, and quality problems are caused
by functional variability of a non-robust product. In parameter design, optimum nominal values of
a product’s design parameters can be selected by treating a product’s design parameters as control
factors and by assessing robustness under noise factors. The use of parameter design at development
and design stages makes it possible to determine the optimum product design and specification so that
the product is robust in the market.
At manufacturing stage, the product suppliers manufacture their products that meet the product
specifications. One can optimize manufacturing processes to produce the products that meet the
specifications. However, robustness against customer’s environment and products’ aging can be
addressed only by product design.
Robust parameter design methodology provides effective methods for achieving robustness through its
design of specification determination, and it is a preventive countermeasure against various losses in
the market.
In practice, many product’s defects and failures occur due to the product’s response that deviates from
or varies around the designed target values by the change in usage environment and deterioration,
i.e. noise conditions. The variability of product’s response due to noises can be used as a measure of
robustness, because market losses increase in proportion to the magnitude of variability of product’s
response. SN ratio, corresponding to the inverse of the variability measure, is used as a measure of
goodness in robustness. In other words, the higher the SN ratio is, the less the market losses are.
For the experimental plan of parameter design, direct product of inner array and outer arrays is
proposed. Control factors are assigned to the inner array, and signal and noise factors are assigned to
the outer array. By using a direct product plan, all the first level interactions between control factors
and noise factors can be assessed and can be utilized to select the optimum level of control factors from
the point of view of robustness.
Assessing robustness through SN ratio is a key of parameter design. The outer array is for evaluating SN
ratio, robustness, for each combination of levels of control factors indicated by the inner array. The inner
array is for comparing SN ratios and selecting the optimum combination of system’s design parameters.
As for the inner array, an orthogonal array L , is recommended as an efficient plan, and then only
the applications of an orthogonal array L are discussed in this International Standard. Applications
of experimental layout other than orthogonal array L can be found in the examples in references in
the Bibliography. More detailed discussions on inner array and orthogonal arrays can be found in the
references.
Robust parameter design (RPD), and thus this International Standard, is directly targeted at the losses
incurred at the usage stage. Where possible, losses at other stages are also investigated so that the
results of parameter design can be applied to perform the optimum product design for the whole stages
of the product’s life cycle.
INTERNATIONAL STANDARD ISO 16336:2014(E)
Applications of statistical and related methods to new
technology and product development process — Robust
parameter design (RPD)
1 Scope
This International Standard gives guidelines for applying the optimization method of robust parameter
design, also called as parameter design, an effective methodology for optimization based on Taguchi
Methods, to achieve robust products.
This International Standard prescribes signal-to-noise ratio (hereafter SN ratio) as a measure of
robustness, and the procedures of parameter design to design robust products utilizing this measure.
The word “robust” in this International Standard means minimized variability of product’s function
under various noise conditions, that is, insensitivity of the product’s function to the changes in the levels
of noises. For robust products, their responses are sensitive to signal and insensitive to noises.
The approach of this International Standard can be applied to any products that are designed and
manufactured, including machines, chemical products, electronics, foods, consumer goods, software,
new materials, and services. Manufacturing technologies are also regarded as products that are used by
manufacturing processes.
2 Normative references
The following documents, in whole or in part, are normatively referenced in this document and are
indispensable for its application. For dated references, only the edition cited applies. For undated
references, the latest edition of the referenced document (including any amendments) applies.
ISO 3534-1, Statistics — Vocabulary and symbols — Part 1: General statistical terms and terms used in
probability
ISO 3534-3, Statistics — Vocabulary and symbols — Part 3: Design of experiments
3 Terms and definitions and symbols
3.1 Term and definitions
For the purposes of this document, the terms and definitions given in ISO 3534-1 and ISO 3534-3, and
the following apply.
3.1.1
function
work which a system performs in order to fulfil its objective
Note 1 to entry: A function can be expressed by the mathematical form of input-output relation.
3.1.2
robustness
degree of smallness in variability of a system’s function under various noise conditions
Note 1 to entry: System’s performance can be assessed by robustness. SN ratio is a quantitative measure of
robustness.
3.1.3
signal-to-noise ratio
SN ratio
ratio of useful effects to harmful effects in response variations
Note 1 to entry: SN ratio is usually expressed in db value. The notation of db is used instead of dB for SN ratios of
robustness measurements.
Note 2 to entry: The anti-logarithm value of an SN ratio, real number, is the inverse of a variation measure such as
a variance or a coefficient of variation, and inversely proportional to monetary loss.
Note 3 to entry: The change in response caused by intentional change of input signal value is a useful effect. In
case of the ideal function being zero point proportional, the linear slope forced through the zero point is a useful
term.
Note 4 to entry: The change in response caused by noise factors is a harmful effect. Effects of noise factors and
deviation from the ideal function are examples.
Note 5 to entry: SN ratio should contain the variability under noise factors and the discrepancy from the ideal
function under average usage condition.
3.1.4
sensitivity
amount of change in response caused by unit change of input
Note 1 to entry: Sensitivity is usually expressed in db value.
Note 2 to entry: For dynamic characteristic cases, the sensitivity shows the magnitude of linear coefficient due to
input signal, β , where β is a proportional constant.
Note 3 to entry: For the nominal-the-best response, the sensitivity shows the magnitude of mean, m , where m is
an average of responses.
3.1.5
noise
variable which disturbs a system’s function
Note 1 to entry: Any variable in the user’s conditions for operating is either a signal or a noise.
Note 2 to entry: Noise is composed of internal noise and external noise. They are sometimes called as capacity
and demand, respectively. Changes of internal constant of the system or its parts over time, such as deterioration,
aging. and wear, and manufacturing variations are examples of internal noises. Usage conditions and environment
conditions of the product are examples of external noises.
3.1.6
signal
input variable to the system, which is intentionally changed by the user to get an intended value of
response in input-output relation
Note 1 to entry: Any variable in the user’s conditions for operating is either a signal or a noise
Note 2 to entry: There are two kinds of signal: active signal and passive signal. Active signal is operated by user
to get intended response, for example, rotating angle of a steering wheel to change the vehicle’s direction. Passive
signal is used by user to know the value of input from response reading, for example, temperature in thermal
measurement. In both cases, output will change by changing the value of the signal but the user wants to get
response value in the active case, and the user wants to know the value of signal in the passive case.
3.1.7
dynamic characteristics
output response which has multiple ideal target values depending on the value of a signal
Note 1 to entry: The relation between dynamic characteristics and a signal can be expressed by input-output
functional form. The output of a system’s function is dynamic characteristics in many cases.
2 © ISO 2014 – All rights reserved
3.1.8
static characteristics
non-dynamic characteristics
output response which has a fixed target value
Note 1 to entry: Static characteristics can be categorized into three groups depending on the target value; nominal-
the-best, smaller-the-better, and larger-the-better characteristics, where the target value is a finite value, zero,
and infinity, respectively.
3.1.9
inner array
experimental plan where design parameters are assigned as control factors or indicative factors
Note 1 to entry: Each treatment run will be assessed for robustness using SN ratio and sensitivity.
Note 2 to entry: Orthogonal arrays are recommended for the inner array because many design parameters can be
taken into consideration in one set of experiments as control factors.
Note 3 to entry: Experimental factors should be categorized by their roles and assigned separately to inner array
or outer array based on their roles in parameter design. Control factors and indicative factors should be assigned
to the inner array.
3.1.10
outer array
experimental plan where variables in users’ conditions are assigned as noise factors or signal factors for
evaluating SN ratio and sensitivity
Note 1 to entry: Any variable in user’s conditions for operating is either a signal or a noise.
Note 2 to entry: Experimental factors should be categorized by their roles and assigned separately to inner array
or outer array based on their roles in parameter design. Noise factors and signal factors should be assigned to the
outer array.
3.2 Symbols
f degree of freedom
k number of levels of signal factor
L linear form
L linear form for level of i
i
M signal factor/input signal
Mi signal level of i
M value of signal level of i
i
N noise factor
n number of levels of noise factor
Ni noise level of i
p standardized error rate
r sum of squares of input signal levels/effective divisor
S sensitivity
S total sum of squares
T
S sum of squares due to mean
m
S sum of squares due to linear slope β
β
S sum of squares due to the variation of linear slope β between noise levels
N ×β
S sum of squares due to error
e
S estimated value of sensitivity for optimum condition
opt
S estimated value of sensitivity for baseline condition
base
S estimated value of sensitivity for current condition
cur
V variance due to error/error variance
e
V variance due to pooled error/variance due to error and noise
N
y output response
β sensitivity coefficient/linear slope
∆S gain in sensitivity
∆η gain in SN ratio
η SN ratio
η estimated value of SN ratio for optimum condition
opt
η estimated value of SN ratio for baseline condition
base
η estimated value of SN ratio for current condition
cur
ρ standardized contribution ratio
4 Robust parameter design — Overview
4.1 Requirements
Robust parameter design is a rational and efficient assessment for discovering technical means to
improve robustness in the designing process. It is, therefore, necessary to provide the following two
procedures:
a) a procedure for accurate and simple evaluation of robustness;
b) a procedure for efficient assessment of multiple technical means.
This clause provides the approach to the goal of parameter design, and more detailed and specific steps
of a robustness evaluation and a parameter design experiment are described in Clauses 5 and 6.
4.2 Assessing the robustness of a system
How can the robustness of a system be accurately assessed by the SN ratio? The robustness of a system is
associated with many usage conditions of the system, so it cannot be assessed by a simple measurement.
To clarify hidden factors associated with the robustness, the assessment should be approached from the
following two viewpoints.
a) Use of an ideal function: The ideal function is a target function of the system. Actual function of the
system should be measured and compared with the ideal function of the system in the robustness
4 © ISO 2014 – All rights reserved
evaluation. It is important to avoid defects, failure modes, or quality problems for achieving the
ideal function of the system.
b) Use of noise factors: Actual system in usage is working under various noise conditions. Noise
effects should be intentionally introduced in the experiment by changing noise levels and the
actual function of the system should be measured and evaluated under those predetermined noise
conditions. Evaluation of the robustness strongly depends on the choice of noise factors and their
levels. It is essential to apply effective noise strategies.
The function of a system is a work that it performs in order to fulfil its objective. For example, the function
of an electric lamp is to transform electric energy into light energy, and the function of a wind turbine
is to transform natural wind energy into rotating energy to perform a work such as water pumping.
The function is normally expressed in a mathematical functional form of a relationship between input
and output energies. The mathematical functional form can be expressed in many ways. Zero-point
proportional formula is common in energy transformation of real physical systems. The details will be
discussed in Clause 5.
Input and output characteristics are fixed based on the system’s ideal function. Input characteristic is
called as a signal in input-output relationship; this is because the changes in output are acquired by the
user’s intentional changing of the input in real usage and also in the experiment of parameter design. The
signal is associated with energy or information necessary to perform its function. The signal factor is
one of the user’s conditions for changing the input when the users of the system try to control the output
of the system. The signal factor has three or more levels in the experiment for dynamic characteristic
so that the straightness of the actual input-output relationship could be evaluated. There is no signal
factor for static characteristic because it has only one target output. Output characteristic is called as an
output response or simply a response.
It is important to identify a suitable measurement method of output response. In time-dependent
phenomena, for example, detection of output response is difficult in some cases. New measurement
methods should be developed in those cases. The output response is associated with the purpose of a
system. In the case of illumination, for example, the output response is magnitude of light, and in the
case of a water pump, it is quantity of water.
Noise condition is a source that makes the system’s actual function deviate from the ideal function.
Examples include environmental conditions in actual working, such as temperature and humidity, an
actual supplied voltage, electrical noise conditions, frequency of operations, and stress. They are called
as external noises. On the other hand, there are noise conditions which are called internal noises, such
as aging and wear. Examples include operating and/or idling time length after started, deterioration
of system’s parts after long operation, and manufacturing variability of a system and/or its parts.
These noise conditions always reduce the system’s functional performance to lower level than the level
expected at the time of design. Since the purpose of robustness assessment is to clarify the performance
by measuring the extent of this reduction, the variation of the system’s function under noise conditions
should be estimated in robustness evaluation. This is the reason why noise conditions should be
taken into the parameter design experiment as noise factors. Three categories of noise factors are a)
environment, b) aging and changes over time, and c) manufacturing variations. For the effective noise
strategy, various types of noise should be examined in actual usage and environmental conditions.
Figure 1 shows an overview of the evaluation of robustness using a noise factor. Here, multiple data from
X to X should be obtained for the objective system under noise levels, N1 to Nn, and SN ratio η should
1 n
be calculated using the data from X to X as a robustness measure. Formulations of SN ratio are shown
1 n
in Clause 5. When more than two systems are compared, the same levels of the same noise factor should
be applied for all objective systems
Noise factor
Objective system
Data
Noise levels
N1 N2 …NnSN ratio
Evaluated system X X … X η
1 2 n
Ni…Noise level
X …Data
i
Figure 1 — Robustness assessment with a noise factor
When multiple different types of noise factors are applied in an experiment, an orthogonal array can
be applied for determining the noise levels. In Figure 2, noise levels, N1 to Nn, are determined by the
combination of levels of noise factors, such as A, B, and C, indicated by the orthogonal array. Experimental
layout other than orthogonal arrays can be applicable for determining the noise levels.
Orthogonal array for
noise factors
Objective system Data
Noise levels by orthogonal array
123 4… n
A : Ambient temperature, humidity …
B : New product and degraded product…
C : Usage frequency…
Noise level N1 N2 N3 N4 …Nn SN ratio
Evaluated system …
X X X X X η
1 2 3 4 n
A, B, C…Noise factors
Figure 2 — Robustness assessment with noise levels assigned by an orthogonal array
4.3 Robustness assessment through SN ratio
The ideal function and the noise strategy are the key issues for the robustness assessment of robust
parameter design. It is essential to measure and evaluate the variability and efficiency of the actual
function of the system. The intent of doing this is that the evaluation covers the whole technical issues
of the system’s operations to prevent technical problems. Robustness evaluation also involves assessing
the effects of noise factors that inhibit the required function. The results are expressed by SN ratio and
sensitivity.
The SN ratio can discriminate the true difference in robustness between designs. Only relative
comparison of SN ratios is meaningful to perform, because absolute values of SN ratios are affected by
setting levels of noise factors. Thus, it is preferable to perform benchmarking in assessing robustness.
A feature of this approach is that the only information needed to evaluate SN ratios is just that on the
knowledge of the function of the system and the noise conditions. No detailed technical information about
the objective system is needed. SN ratios can be calculated in the same way for the objective systems
as long as they have the same function, that is, the same input-output relationship, even if they have
different technical constituents. Since the robustness of systems can be accurately evaluated through
SN ratios, then the robustness of various systems with different design concepts can be assessed and
compared.
6 © ISO 2014 – All rights reserved
The comparison of various systems based on different technologies or different design concepts can be
performed in the same way through SN ratio. Systems, such as conventional systems and newly developed
systems, one’s own systems and one’s competitor’s systems, can be evaluated and assessed in the same
way through SN ratio, when they have the same function. This is the idea to conduct benchmarking on
various designs in the robustness assessment through SN ratio.
4.4 An efficient method for assessing technical ideas — Parameter design
Basic technologies and mechanisms should be selected as a design concept first to start designing a
system of industrial products. When there are multiple system design concepts to be benchmarked, the
robustness assessment introduced in the previous subclause can be applied to select the best design
concept.
After selecting the best design concept, the next step is to perform a detailed design by selecting values of
system’s design parameters. In this detailed design step, designers can optimize the system by selecting
the optimum nominal values so that the function of the designed system becomes the most robust and
efficient. The system design optimization method performed at this step is called parameter design,
because design optimization is performed by setting design parameters to the optimum nominal values.
Consider what sort of states might be significant. When a system is in the optimum state, it achieves the
best overall performance in all conceivable usage conditions. More specifically, an industrial system can
stably perform its intended function anytime, even when, for example, it is working under a wide range
of temperature and humidity, and when it is used in many different ways and in different environments.
The optimum design conditions are taken as a combination of design parameters’ values that maximize
the robustness of the product. Since optimization by parameter design implies optimizing for maximized
robustness, that is, minimized variability and maximized efficiency, judgments should be made by using
robustness measure, SN ratio, and sensitivity.
The basis of the system design optimization by robustness assessment through SN ratio is a criterion for
optimization in parameter design. The robustness assessment should be performed with regard to all the
possible designs in design space, but in practice this is impossible. This is because a vast number of tests
would have to be performed to take all possible combinations of design parameters into consideration.
As a more practical method for applying in development and design stages, an experiment using an
orthogonal array is recommended where the combinations of many design parameters can be tested
under a limited number of experimental runs. An orthogonal array plan is recommended not only
because it can reduce the number of experimental runs comparing with a full factorial plan with the
same number of control factors, but also because it can assign maximum number of control factors in
a plan under the situation of same number of experimental runs. Reliability of experimental results
should be confirmed in the confirmation experimental run for reproducibility check. Clause 6 describes
a specific method for performing the confirmation experiment to check the reproducibility.
Procedure of a parameter design experiment should be as follows:
— (Step 1) Clarify the system’s ideal function;
— (Step 2) Select signal factor and its range;
— (Step 3) Select measurement method of output response;
— (Step 4) Develop noise strategy, and select noise factors and their levels;
— (Step 5) Select control factors and their levels from design parameters
— (Step 6) Assign experimental factors to inner or outer array;
— (Step 7) Conduct experiment and collect data;
— (Step 8) Calculate SN ratio, η, and sensitivity, S;
— (Step 9) Generate factorial effect diagrams on SN ratio and sensitivity;
— (Step 10) Select the optimum condition;
— (Step 11) Estimate the improvement in robustness by the gain;
— (Step 12) Conduct a confirmation experiment and check the gain and “reproducibility”.
4.5 Two-step optimization (Strategy of parameter design)
Figure 3 presents an overview of parameter design as described above. The experiment in this figure
includes two orthogonal arrays; one orthogonal array for control factors, that is, for design parameters
(inner array) and the other orthogonal array for noise factors (outer array). This layout is called a direct
product plan. The number of experimental data corresponds to the product of the numbers of runs
respectively specified in two orthogonal arrays. For example, in the case of the combination of inner
array, L , and outer array, L , each array has runs of m = 18 and n = 12, and the number of total runs
18 12
comes to 18 × 12 = 216.
Outer
orthogonal array for
noise factors
Inner
orthogonal
Data
array for
control factors
Noise levels by orthogonal array
Noise factors 123 4… n
A : temperature
B : humidity
C : …….
N1 N2 N3 N4 …NnSensitivity SN ratio
1 S η
1 1
2 S η
2 2
3 S η
3 3
4 S η
4 4
… … …
m S η
m m
Figure 3 — Direct product plan for parameter design
Full factorial plan can be used as an outer array plan for noise and signal factors instead of an orthogonal
array plan in some cases. In the case of physical tests, it is recommended to compound many noise
factors into one compounded noise factor. However, it is always recommended to use an orthogonal
array plan as an inner array for design parameters, because many design parameters can be assigned
in one orthogonal array.
The experimental data obtained for each combination of levels of control factors consist of multiple data
with the corresponding number of noise levels. To find out the optimum values of design parameters for
robustness, the sensitivity (mean value in case of nominal-the-best response), and the SN ratio should be
calculated for each row of inner array, that is, for the combination of values of design parameters. Then
8 © ISO 2014 – All rights reserved
Evaluated systems
with assigned
design parameters
the factorial effects of control factors on sensitivity and SN ratio should also be calculated, and they
are summarized in factorial effect diagrams as shown in Figures 4 and 5. Specific calculation formulae
are described in Clause 6. The optimum values of design parameters are selected using the diagrams
on sensitivity and SN ratio. The sensitivity represents the mean value of the data set (in case of static
characteristics), and the SN ratio represents robustness.
The factorial effect diagram shows how the system’s function is affected by each design parameter
incorporated into the experiment. If a factor has a large gradient, it has a large effect on the system’s
function. Two types of factorial effect diagram represent the degree of influence relating to SN ratio and
sensitivity. An important point in two-step design is to pay more attention to SN ratio than to sensitivity.
In the first step, the optimum level of control factor should be selected to maximize SN ratio in the
factorial effect diagram on SN ratio (see Figure 4), and then, in the second step, typically just one design
parameter should be used to adjust a mean value or linear slope, i.e. sensitivity (see Figure 5), to the
target value. For this adjustment, it is desired to select one factor with maximal effect on sensitivity and
minimal effect on SN ratio. The first step is to optimize the design for robustness by SN ratio, and the
second step is to adjust the magnitude to the target value by sensitivity. This two-step design procedure
is a very important concept for the design of robustness. This is the reason why parameter design for
robustness is also called as a two-step optimization.
A1 A2 B1 B2 B3 C1 C2 C3 D1 D2 D3 E1 E2 E3 F1 F2 F3 G1 G2 G3 H1 H2 H3
Figure 4 — Factorial effect diagram on SN ratio (robustness)
-2
-4
-6
A1 A2 B1 B2 B3 C1 C2 C3 D1 D2 D3 E1 E2 E3 F1 F2 F3 G1 G2 G3 H1 H2 H3
Figure 5 — Factorial effect diagram on sensitivity (mean value or linear slope)
Why is the two-step optimization important? Which is more difficult, the robustness optimization by SN
ratio or the magnitude adjustment by sensitivity?
The order of the optimizations is important to design a robust system efficiently. Consider an example
of the case of voice recording. If audio data are recorded with high background noise, then adjusting the
volume will hardly make the recorded sound any easier to listen to when it is played back. To extract
SN ratio (db)
Sensitivity (db)
information buried in the noise, techniques such as noise reduction to cancel the effects of noises or
using a microphone that is less sensitive to the background noise shall be employed. Improving SN ratio
requires advanced technical ability and counter-measures in recording. On the other hand, when the
average recording level is too low, it can easily be improved by adjusting the volume during playback.
The controlling of the average value of the playback sound by a volume can often be accomplished by a
relatively simple method in this way. Adjusting the magnitude can be performed by sensitivity.
Another example is visual imaging function such as photographs and video pictures. The average tone
level can easily be corrected, but a picture taken in dark conditions is often very noisy and results in
a low-quality picture. There are also limits to how much the picture quality can be improved by image
processing.
It is relatively easy to achieve adjusting the magnitude, because it needs just one adjusting parameter.
It is easy to adjust the average energy level. On the other hand, it is not easy to improve robustness. It
is desired to have as many control factors as possible. In designing a system, it is therefore wise to give
greater priority to setting the optimum levels of design parameters to maximize SN ratio. This forms
the foundation of the two-step optimization, where robustness optimization by the SN ratio should have
the first priority.
4.6 Determination of the optimum design
Once parameter design experiment clarifies which design parameters affect SN ratio and which design
parameters affect sensitivity as in factorial effect diagrams, one can select a set of optimum values
of design parameters based on robustness. Then, the final optimum design can be determined by
considering other constraints such as cost and delivery requirements.
Since a system’s final optimum design should be determined by the overall balance of many constraints,
it is preferable to select an experimental plan where each experimental factor covers a wider range in a
design space. There is a possibility that the optimum value might exist in the range far from experiences;
then it is recommended that the levels of control factors should cover a range as widely as possible.
In the parameter design, robustness optimization is performed through maximizing SN ratio. The SN
ratio is a quantitative measure of the user’s quality loss due to defects, failures, and quality problems
caused by lack of robustness. The user’s quality loss can include losses by malfunctions, by defects, by
additional maintenance costs, etc.
According to Taguchi’s quality loss function, the SN ratio can be transformed to user’s quality loss in
monetary unit. Total loss to the society caused by the product can be derived from the user’s quality
loss by adding other cost, such as product development cost, material cost, production cost, shipping
cost, normal maintenance cost, disposal cost, etc. The total social loss should be a quality measure
of the product. In product design stage, a product designer should consider the total social loss from
the viewpoint of technology. However, it is difficult for the designer to forecast the total social loss in
the design stage, but he should, at least, assess and optimize the product design from the viewpoint of
robustness. Robust parameter design focuses on the user’s quality loss from the viewpoint of robust
engineering, that is, the variability in function of the product.
5 Assessment of robustness by SN ratio
5.1 Concepts of SN ratio
The variability in function of a system should be evaluated and optimized through parameter design
for designing a robust product. When a subsystem is to be assessed for robustness, one should consider
noise conditions at the whole system level in the users’ hand. It is critical to ensure the robustness at
whole system level.
A system’s function can be defined as a functional form of input-output relationship at usage stage
of the system. Users manipulate a signal to get an intended output response of the system. Signal is
an input characteristic that is intentionally set to change the output of the system. A functional form
10 © ISO 2014 – All rights reserved
that represents the ideal input-output relationship of the system’s function is called as a system’s ideal
fun
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.