ASTM E1355-97
(Guide)Standard Guide for Evaluating the Predictive Capability of Deterministic Fire Models
Standard Guide for Evaluating the Predictive Capability of Deterministic Fire Models
SCOPE
1.1 This guide provides a methodology for evaluating the predictive capabilities of a fire model for a specific use.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
An American National Standard
Designation: E 1355 – 97
Standard Guide for
Evaluating the Predictive Capability of Deterministic Fire
Models
This standard is issued under the fixed designation E 1355; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (e) indicates an editorial change since the last revision or reapproval.
1. Scope 3.1.1 model evaluation—the process of quantifying the
accuracy of chosen results from a model when applied for a
1.1 This guide provides a methodology for evaluating the
specific use.
predictive capabilities of a fire model for a specific use.
3.1.2 model validation—the process of determining the
1.2 The methodology is presented in terms of four areas of
correctness of the assumptions and governing equations imple-
evaluation:
mented in a model when applied to the entire class of problems
1.2.1 Defining the model and scenarios for which the
addressed by the model.
evaluation is to be conducted,
3.1.3 model verification—the process of determining the
1.2.2 Verifying the appropriateness of the theoretical basis
correctness of the solution of a system of governing equations
and assumptions used in the model,
in a model. With this definition, verification does not imply the
1.2.3 Verifying the mathematical and numerical robustness
solution of the correct set of governing equations, only that the
of the model, and
given set of equations is solved correctly.
1.2.4 Quantifying the uncertainty and accuracy of the model
3.2 For additional definitions of terms used in this guide
results in predicting of the course of events in similar fire
refer to Terminology E 176.
scenarios.
1.3 This standard does not purport to address all of the
4. Summary of Guide
safety concerns, if any, associated with its use. It is the
4.1 A recommended process for evaluating the predictive
responsibility of the user of this standard to establish appro-
capability of fire models is described. This process includes a
priate safety and health practices and determine the applica-
brief description of the model and the scenarios for which
bility of regulatory limitations prior to use.
evaluation is sought. Then, methodologies for conducting an
1.4 The output from this document should not be used for
analysis to quantify the sensitivity of model predictions to
regulatory purposes or the basis for regulations.
various uncertain factors are presented, and several alternatives
2. Referenced Documents for evaluating the accuracy of the predictions of the model are
provided. Finally, guidance is given concerning the relevant
2.1 ASTM Standards:
documentation required to summarize the evaluation process.
E 176 Terminology of Fire Standards
E 603 Guide for Room Fire Experiments
5. Significance and Use
E 1472 Guide for Documenting Computer Software for Fire
2 5.1 The process of model evaluation is critical to establish-
Models
2 ing both the acceptable uses and limitations of fire models. It is
E 1591 Guide for Data for Fire Models
not possible to evaluate a model in total; instead, this guide is
2.2 International Standards Organization Standards:
3 intended to provide a methodology for evaluating the predic-
Guide to the Expression of Uncertainty in Measurement
tive capabilities for a specific use. Validation for one applica-
3. Terminology tion or scenario does not imply validation for different sce-
narios. Several alternatives are provided for performing the
3.1 Definitions Specific to This Guide:
evaluation process including: comparison of predictions
against standard fire tests, full-scale fire experiments, field
experience, published literature, or previously evaluated mod-
This guide is under the jurisdiction of ASTM Committee E-5 on Fire Standards
els.
and is the direct responsibility of Subcommittee E05.39 on Fire Modeling.
Current edition approved July 10, 1997. Published June 1998. Originally
5.2 The use of fire models currently extends beyond the fire
published as E 1355 – 90. Last previous edition E 1355 – 92.
research laboratory and into the engineering, fire service and
Annual Book of ASTM Standards, Vol 04.07.
legal communities. Sufficient evaluation of fire models is
Available from American National Standards Institute, 11 West 42nd Street,
13th Floor, New York, NY 10036. necessary to ensure that those using the models can judge the
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.
E1355–97
adequacy of the scientific and technical basis for the models, judging the results of the evaluation. Details applicable to
select models appropriate for a desired use, and understand the evaluation of the predictive capability of fire models are
level of confidence which can be placed on the results provided in 7.2.
predicted by the models. Adequate evaluation will help prevent 6.3 Theoretical Basis and Assumptions in the Model—An
the unintentional misuse of fire models.
independent review of the underlying physics and chemistry
5.3 This guide is intended to be used in conjunction with inherent in a model ensures appropriate application of submod-
other guides under development by Committee E-5. It is
els which have been combined to produce the overall model.
intended for use by: Details applicable to evaluation of the predictive capability of
5.3.1 Model Developers/Marketers—To document the use- fire models are provided in Section 8.
fulness of a particular calculation method perhaps for specific 6.4 Mathematical and Numerical Robustness—The com-
applications. Part of model development includes identification puter implementation of the model should be checked to ensure
of precision and limits of applicability, and independent such implementation matches the stated documentation. De-
testing. tails applicable to evaluation of the predictive capability of fire
models are provided in Section 9.
5.3.2 Model Users—To assure themselves that they are
using an appropriate model for an application and that it 6.5 Quantifying the Uncertainty and Accuracy of the Model:
provides adequate accuracy. 6.5.1 Model Uncertainty—Even deterministic models rely
5.3.3 Developers of Model Performance Codes—To be sure
on inputs often based on experimental measurements, empiri-
that they are incorporating a valid calculation procedures into cal correlations, or estimates made by engineering judgement.
codes. Uncertainties in the model inputs can lead to corresponding
5.3.4 Approving Offıcials—To ensure that the results of uncertainties in the model outputs. Sensitivity analysis is used
to quantify these uncertainties in the model outputs based upon
calculations using mathematical models stating conformance to
this guide, cited in a submission, show clearly that the model known or estimated uncertainties in model inputs. Guidance
for obtaining input data for fire models is provided by Guide
is used within its applicable limits and has an acceptable level
of accuracy. E 1591. Details of sensitivity analysis applicable to evaluation
of the predictive capability of fire models are provided in
5.3.5 Educators—To demonstrate the application and ac-
ceptability of calculation methods being taught. Section 10.
5.4 This guide is not meant to describe an acceptance testing 6.5.2 Experimental Uncertainty—In general, the result of
measurement is only the result of an approximation or estimate
procedure.
of the specific quantity subject to measurement, and thus the
5.5 The primary emphasis of this guide is on zone models of
result is complete only when accompanied by a quantitative
compartment fires. However, other types of mathematical
statement of uncertainty. Guidance for conducting full-scale
models need similar evaluations of their predictive capabilities.
compartment tests is provided by Guide E 603. Guidance for
determining the uncertainty in measurements is provided in the
6. General Methodology
ISO Guide to the Expression of Uncertainty in Measurement.
6.1 The methodology is presented in terms of four areas of
6.5.3 Model Evaluation—Obtaining accurate estimates of
evaluation:
fire behavior using predictive fire models involves insuring
6.1.1 Defining the model and scenarios for which the
correct model inputs appropriate to the scenarios to be mod-
evaluation is to be conducted,
eled, correct selection of a model appropriate to the scenarios
6.1.2 Assessing the appropriateness of the theoretical basis
to be modeled, correct calculations by the model chosen, and
and assumptions used in the model,
correct interpretation of the results of the model calculation.
6.1.3 Assessing the mathematical and numerical robustness
Evaluation of a specific scenario with different levels of
of the model, and
knowledge of the expected results of the calculation addresses
6.1.4 Quantifying the uncertainty and accuracy of the model
these multiple sources of potential error. Details applicable to
results in predicting the course of events in similar fire
evaluation of the predictive capability of fire models are
scenarios.
provided in Section 11.
6.2 Model and Scenario Definition:
6.2.1 Model Documentation—Sufficient documentation of
7. Model and Scenario Definition
calculation models, including computer software, is absolutely
7.1 Model Documentation—Provide the following informa-
necessary to assess the adequacy of the scientific and technical
tion:
basis of the models, and the accuracy of computational
7.1.1 The name and version of the model,
procedures. Also, adequate documentation will help prevent
7.1.2 The name of the model developer(s),
the unintentional misuse of fire models. Guidance on the
7.1.3 A list of relevant publications,
documentation of computer-based fire models is provided in
7.1.4 A statement of the stated uses, limitations, and results
Guide E 1472. Details applicable to evaluation of the predic-
of the model,
tive capability of fire models are provided in 7.1.
7.1.5 The type of model (zone, field, etc.),
6.2.2 Scenario Documentation—Provide a complete de-
scription of the scenarios or phenomena of interest in the 7.1.6 A statement of the modeling rigor, including:
evaluation to facilitate appropriate application of the model, to 7.1.6.1 The assumptions inherent in the model and the
aid in developing realistic inputs for the model, and criteria for governing equations included in the model formulation, and
E1355–97
7.1.6.2 The numerics employed to solve the equations and and the mechanical response. Time scales associated with the
the method by which individual solutions are coupled. processes may be substantially different, which easily causes
7.1.7 Additional assumptions of the model as they relate to numerical difficulties. Such problems are called stiff. Some
the stated uses or other potential uses, numerical methods have difficulty with stiff problems since
7.1.8 The input data required to run the model, and they slavishly follow the rapid changes even when they are less
7.1.9 Property data that are defined with the computer important than the general trend in the solution. Special
program or were assumed in the model development. algorithms have been devised for solving stiff problems.
7.2 Scenarios for Which Evaluation is Sought—Provide the 9.1.5 Numerical accuracy of predictive fire models has been
following information: considered in the literature.
7.2.1 A description of the scenarios or phenomena of
interest, 10. Model Sensitivity
7.2.2 A list of quantities predicted by the model for which
10.1 Fire growth models are typically based on a system of
evaluation is sought, and
ordinary differential equations of the form
7.2.3 The degree of accuracy required for each quantity.
dz
5 f~z, p, t! z t5 0 5 z (1)
~ !
dt
8. Theoretical Basis for the Model
8.1 The theoretical basis of the model should be reviewed
where:
by one or more recognized experts fully conversant with the z(z ,z ,.,z ) = the solution vector for the system of
1 2 m
chemistry and physics of fire phenomena but not involved with
equations (for example, mass, tem-
the production of the model. This review should include: perature, or volume)
8.1.1 An assessment of the completeness of the documen- p(p ,p ,.,p ) = a vector of input parameters (for
1 2 n
tation particularly with regard to the assumptions and approxi- example, room area, room height,
mations. heat release rate), and
8.1.2 An assessment of whether there is sufficient scientific t = time.
evidence in the open scientific literature to justify the ap- The solutions to these equations are, in general, not known
proaches and assumptions being used. explicitly and must be determined numerically. To study the
sensitivity of such a set of equations, the partial derivatives of
8.1.3 Empirical or reference data used for constants and
default values in the code should also be assessed for accuracy an output z with respect to an input p (forj=1,.,mand I
j i
=1,.,n) should be examined.
and applicability in the context of the model.
10.2 A sensitivity analysis of a model is a study of how
9. Mathematical and Numerical Robustness
changes in model parameters affect the results generated by the
9.1 Analyses which can be performed include:
model. Model predictions may be sensitive to uncertainties in
9.1.1 Analytical Tests—If the program is to be applied to a input data, to the level of rigor employed in modeling the
situation for which there is a known mathematical solution,
relevant physics and chemistry, and to the accuracy of numeri-
analytical testing is a powerful way of testing the correct cal treatments. The purpose of conducting a sensitivity analysis
functioning of a model. However, there are relatively few
is to assess the extent to which uncertainty in model inputs is
situations (especially for complex scenarios) for which analyti- manifested to become uncertainty in the results of interest from
cal solutions are known.
the model. This information can be used to:
9.1.2 Code Checking—The code can be verified on a 10.2.1 Determine the dominant variables in the models,
structural basis preferably by a third party either totally
10.2.2 Define the acceptable range of values for each input
manually or by using code checking programs to detect variable,
irregularities and inconsistencies within the computer code. A
10.2.3 Quantify the sensitivity of output variables to varia-
process of code checking can increase the level of confidence tions in input data, and
in the program’s ability to process the data to the program
10.2.4 Inform and caution any potential users about the
correctly, but it cannot give any indication of the likely degree and level
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.