ASTM D6122-23

Standard Practice for Validation of the Performance of Multivariate Online, At-Line, Field and Laboratory Infrared Spectrophotometer, and Raman Spectrometer Based Analyzer Systems

SIGNIFICANCE AND USE
5.1 The primary purpose of this practice is to permit the user to validate numerical values produced by a multivariate, infrared or near-infrared laboratory or process (online or at-line) analyzer calibrated to measure a specific chemical concentration, chemical property, or physical property. If the analyzer results agree with the primary test method to within limits based on the multivariate model for the user-prespecified statistical confidence level, these results can be considered ’validated’ to the user pre-specified confidence limit for a specific application, and hence can be considered useful for that specific application.  
5.2 Procedures are described for verifying that the instrument, the model, and the analyzer system are stable and properly operating.  
5.3 A multivariate analyzer system inherently utilizes a multivariate calibration model. In practice, the model both implicitly and explicitly spans some subset of the population of all possible samples that could be in the complete multivariate sample space. The model is applicable only to samples that fall within the subset population used in the model construction. A sample measurement cannot be validated unless applicability is established. Applicability cannot be assumed.  
5.3.1 Outlier detection methods are used to demonstrate applicability of the calibration model for the analysis of the process sample spectrum. The outlier detection limits are based on historical as well as theoretical criteria. The outlier detection methods are used to establish whether the results obtained by an analyzer are potentially valid. The validation procedures are based on mathematical test criteria that indicate whether the process sample spectrum is within the range spanned by the analyzer system calibration model. If the sample spectrum is an outlier, the analyzer result is invalid. If the sample spectrum is not an outlier, then the analyzer result is valid providing that all other requirements for validity are...
SCOPE
1.1 This practice covers requirements for the validation of measurements made by laboratory, field, or process (online or at-line) infrared (near- or mid-infrared analyzers, or both), and Raman analyzers, used in the calculation of physical, chemical, or quality parameters (that is, properties) of liquid petroleum products and fuels. The properties are calculated from spectroscopic data using multivariate modeling methods. The requirements include verification of adequate instrument performance, verification of the applicability of the calibration model to the spectrum of the sample under test, and verification that the uncertainties associated with the degree of agreement between the results calculated from the infrared or Raman measurements and the results produced by the PTM used for the development of the calibration model meets user-specified requirements. Initially, a limited number of validation samples representative of current production are used to do a local validation. When there is an adequate number of validation samples with sufficient variation in both property level and sample composition to span the model calibration space, the statistical methodology of Practice D6708 can be used to provide general validation of this equivalence over the complete operating range of the analyzer. For cases where adequate property and composition variation is not achieved, local validation shall continue to be used.  
1.1.1 For some applications, the analyzer and PTM are applied to the same material. The application of the multivariate model to the analyzer output (spectrum) directly produces a PPTMR for the same material for which the spectrum was measured. The PPTMRs are compared to the PTMRs measured on the same materials to determine the degree of agreement.  
1.1.2 For other applications, the material measured by the analyzer system is subjected to a consistent additive treatment prior to being analyzed by the PTM...

General Information

• Status: Published
• Publication Date: 30-Jun-2023
• ICS: 17.180.30 - Optical measuring instruments
• Technical Committee: D02 - Petroleum Products, Liquid Fuels, and Lubricants
• Drafting Committee: D02.25 - Performance Assessment and Validation of Process Stream Analyzer Systems

Relations

• Replaces: ASTM D6122-22 - Standard Practice for Validation of the Performance of Multivariate Online, At-Line, Field and Laboratory Infrared Spectrophotometer, and Raman Spectrometer Based Analyzer Systems
• Effective Date: 01-Jul-2023
• Refers: ASTM D6708-24 - Standard Practice for Statistical Assessment and Improvement of Expected Agreement Between Two Test Methods that Purport to Measure the Same Property of a Material
• Effective Date: 01-Mar-2024
• Refers: ASTM D2699-24 - Standard Test Method for Research Octane Number of Spark-Ignition Engine Fuel
• Effective Date: 01-Mar-2024
• Refers: ASTM D86-23ae1 - Standard Test Method for Distillation of Petroleum Products and Liquid Fuels at Atmospheric Pressure
• Effective Date: 01-Dec-2023
• Refers: ASTM D1265-23a - Standard Practice for Sampling Liquefied Petroleum (LP) Gases, Manual Method
• Effective Date: 01-Dec-2023
• Refers: ASTM D86-23a - Standard Test Method for Distillation of Petroleum Products and Liquid Fuels at Atmospheric Pressure
• Effective Date: 01-Dec-2023
• Refers: ASTM D2699-23b - Standard Test Method for Research Octane Number of Spark-Ignition Engine Fuel
• Effective Date: 01-Nov-2023
• Refers: ASTM D5842-23 - Standard Practice for Sampling and Handling of Fuels for Volatility Measurement
• Effective Date: 01-Oct-2023
• Refers: ASTM D2699-23a - Standard Test Method for Research Octane Number of Spark-Ignition Engine Fuel
• Effective Date: 01-Oct-2023
• Refers: ASTM D86-23 - Standard Test Method for Distillation of Petroleum Products and Liquid Fuels at Atmospheric Pressure
• Effective Date: 01-Mar-2023
• Refers: ASTM D1265-23 - Standard Practice for Sampling Liquefied Petroleum (LP) Gases, Manual Method
• Effective Date: 01-Mar-2023
• Refers: ASTM E456-13a(2022)e1 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 01-Apr-2022
• Refers: ASTM E456-13a(2022) - Standard Terminology Relating to Quality and Statistics
• Effective Date: 01-Apr-2022
• Refers: ASTM D6708-21 - Standard Practice for Statistical Assessment and Improvement of Expected Agreement Between Two Test Methods that Purport to Measure the Same Property of a Material
• Effective Date: 01-May-2021
• Refers: ASTM D5842-19 - Standard Practice for Sampling and Handling of Fuels for Volatility Measurement
• Effective Date: 01-Nov-2019

Overview

ASTM D6122-23 provides a comprehensive standard practice for validating the performance of multivariate online, at-line, field, and laboratory infrared spectrophotometer and Raman spectrometer based analyzer systems. Developed by ASTM International, this practice is critical for ensuring the reliability of infrared and Raman analyzers used to measure the physical, chemical, or quality properties of liquid petroleum products and fuels. The validation process ensures that results from these advanced analyzer systems are statistically equivalent to those obtained by traditional primary test methods (PTM), enhancing quality control, process efficiency, and regulatory compliance in the petroleum and fuels industries.

Key Topics

• Scope of Application

  • Covers laboratory, field, at-line, and online infrared (near- and mid-infrared) and Raman analyzers.
  • Applies to systems that calculate properties of liquid petroleum products and fuels from spectroscopic data using multivariate modeling.

• Validation Procedures

  • Local Validation: Uses a limited set of real-world samples representative of current production, to show agreement between analyzer and PTM results within user-specified confidence limits.
  • General Validation: Employs a broader, more statistically varied sample set, allowing for validation across the full calibration space using established methodologies such as ASTM D6708.
  • Ongoing (Continual) Validation: Involves routine quality assurance and control chart monitoring to maintain validated status of analyzer systems after initial validation.

• Model Applicability

  • Emphasizes that validation is only valid for samples that fall within the calibration model’s defined population.
  • Outlier detection methods are applied to ensure that the sample spectra are within the calibration range and suitable for reliable measurement.

• Statistical Confidence and Uncertainty

  • Results are only considered validated if they meet user-defined statistical confidence levels.
  • The difference between analyzer results and PTM must fall within calculated uncertainty bounds, ensuring high confidence in reported values.

• Instrument and Model Performance Verification

  • Details procedures for verifying both hardware stability and correct model operation.
  • Includes performance checks with instrument verification samples to detect any drift or malfunction.

Applications

ASTM D6122-23 is widely applied in:

• Process Analytical Technology: Supporting compliance and optimization in refinery and petrochemical operations by enabling rapid, real-time quality analysis.
• Product Release and Certification: Validating analyzer systems for product property certification in alignment with regulatory and contractual requirements.
• Research and Development: Providing framework for the evaluation of new analyzers, calibration transfers, and multivariate modeling techniques.
• Quality Assurance: Ensuring ongoing analyzer reliability through regular validation, supporting manufacturing consistency and customer confidence.

Industries that benefit include petroleum refining, fuel distribution, quality control laboratories, and chemical processing, where accurate measurement of multiple product qualities is essential.

Related Standards

To effectively implement ASTM D6122-23, familiarity with related ASTM standards is recommended, including:

• ASTM D6708: Statistical assessment of agreement between test methods for the same property.
• ASTM D8321: Development and validation of multivariate analyses for spectroscopic measurements.
• ASTM D3764: Performance validation of process stream analyzer systems.
• ASTM D8340: Performance-based qualification of spectroscopic analyzer systems.
• ASTM D6299: Statistical quality assurance and control charting techniques.
• ASTM E131/E456/E932/E1421: Terminology and performance measurement guidelines for molecular spectroscopy.

These standards provide critical support for sample handling, calibration, measurement, statistical validation, and ongoing quality control.

---

Adhering to ASTM D6122-23 ensures robust validation of multivariate infrared and Raman analyzer systems, supporting regulatory compliance, process optimization, and industry best practices in liquid fuel and petroleum product analysis.

Standards Content (Sample)

This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: D6122 − 23
Standard Practice for
Validation of the Performance of Multivariate Online, At-
Line, Field and Laboratory Infrared Spectrophotometer, and
Raman Spectrometer Based Analyzer Systems
This standard is issued under the fixed designation D6122; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
INTRODUCTION
Operation of a laboratory or process stream analyzer system typically involves five sequential
activities. (1) Correlation—Prior to the initiation of the procedures described in this practice, a
multivariate model is derived which relates the spectrum produced by the analyzer to the Primary Test
Method Result (PTMR). (1a) If the analyzer and Primary Test Method (PTM) measure the same
material, then the multivariate model directly relates the spectra to PTMR collected on the same
samples. Alternatively (1b) if the analyzer measures the spectra of a material that is subjected to
treatment prior to being measured by the PTM, then the multivariate model relates the spectra of
the untreated sample to the PTMR for the same sample after treatment. (2) Analyzer Qualification—
When an analyzer is initially installed, or after major maintenance has been performed, diagnostic
testing is performed to demonstrate that the analyzer meets the manufacturer’s specifications and
historical performance standards. These diagnostic tests may require that the analyzer be adjusted so
as to provide predetermined output levels for certain reference materials (3) Local Validation—A
local validation is performed using an independent but limited set of materials that were not part of
the correlation activity. This local validation is intended to demonstrate that the agreement between the
Predicted Primary Method Test Results (PPTMRs) and the PTMRs are consistent with expectations
based on the multivariate model. (4) General Validation—After an adequate number of PPTMRs and
PTMRs have been accrued on materials that were not part of the correlation activity and which
adequately span the multivariate model compositional space, a comprehensive statistical assessment
can be performed to demonstrate that the PPTMRs agree with the PTMRs to within user-specified
requirements. (5) Continual Validation—Subsequent to a successful local or general validation,
quality assurance control chart monitoring of the differences between PPTMR and PTMR is conducted
during normal operation of the process analyzer system to demonstrate that the agreement between the
PPTMRs and the PTMRs established during the General Validation is maintained. This practice deals
with the third, fourth, and fifth of these activities.
“Correlation where analyzer measures a material which is subjected to treatment before being
measured by the PTM” as outlined in this practice can be applied to biofuels where the biofuel
material is added at a terminal or other facility and not included in the process stream material sampled
by the analyzer at the basestock manufacturing facility. The “treatment” shall be a constant percentage
addition of the biofuels material to the basestock material. The correlation is deemed valid only for
the specific percentage addition and type of biofuel material used in its development.
1. Scope* products and fuels. The properties are calculated from spectro-
scopic data using multivariate modeling methods. The require-
1.1 This practice covers requirements for the validation of
ments include verification of adequate instrument performance,
measurements made by laboratory, field, or process (online or
verification of the applicability of the calibration model to the
at-line) infrared (near- or mid-infrared analyzers, or both), and
spectrum of the sample under test, and verification that the
Raman analyzers, used in the calculation of physical, chemical,
uncertainties associated with the degree of agreement between
or quality parameters (that is, properties) of liquid petroleum
*A Summary of Changes section appears at the end of this standard
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
D6122 − 23
the results calculated from the infrared or Raman measure- prediction is evaluated independently. The user will typically
ments and the results produced by the PTM used for the have multiple validation procedures running simultaneously in
development of the calibration model meets user-specified parallel.
requirements. Initially, a limited number of validation samples
1.3 Results used in analyzer validation are for samples that
representative of current production are used to do a local
were not used in the development of the multivariate model,
validation. When there is an adequate number of validation
and for spectra which are not outliers or nearest neighbor
samples with sufficient variation in both property level and
inliers relative to the multivariate model.
sample composition to span the model calibration space, the
statistical methodology of Practice D6708 can be used to 1.4 When the number, composition range or property range
provide general validation of this equivalence over the com- of available validation samples do not span the model calibra-
plete operating range of the analyzer. For cases where adequate tion range, a local validation is done using available samples
property and composition variation is not achieved, local representative of current production. When the number, com-
validation shall continue to be used. position range and property range of available validation
1.1.1 For some applications, the analyzer and PTM are samples becomes comparable to those of the model calibration
applied to the same material. The application of the multivari- set, a general validation can be done.
ate model to the analyzer output (spectrum) directly produces
1.4.1 Local Validation:
a PPTMR for the same material for which the spectrum was
1.4.1.1 The calibration samples used in developing the
measured. The PPTMRs are compared to the PTMRs measured
multivariate model must show adequate compositional and
on the same materials to determine the degree of agreement.
property variation to enable the development of a meaningful
1.1.2 For other applications, the material measured by the
correlation, and must span the compositional range of samples
analyzer system is subjected to a consistent additive treatment
to be analyzed using the model to ensure that such analyses are
prior to being analyzed by the PTM. The application of the
done via interpolation rather than extrapolation. The Standard
multivariate model to the analyzer output (spectrum) produces
Error of Calibration (SEC) is a measure of how well the
a PPTMR for the treated material. The PPTMRs based on the
PTMRs and PPTMRs agree for this set of calibration samples.
analyzer outputs are compared to the PTMRs measured on the
SEC includes contributions from spectrum measurement error,
treated materials to determine the degree of agreement.
PTM measurement error, and model error. Sample (type)
1.1.3 In some cases, a two-step procedure is employed. In
specific biases are a part of the model error. Typically,
the first step, the analyzer and PTM are applied to the
spectroscopic analyzers are very precise, so that spectral
measurement of a blendstock material. In a second step, the
measurement error is small relative to the other types of error.
PPTMRs produced in Step 1 are used as inputs to a second
1.4.1.2 During initial analyzer validation, the compositional
model that predicts the results obtained when the PTM is
range of available samples may be small relative to the range
applied to the analysis of the finished blended product pro-
of the calibration set. Because of the high precision of the
duced by additivation to the blendstock. If the analyzer used in
spectroscopic measurement, the average difference between
the first step is a multivariate spectroscopic based analyzer,
the PTMRs and PPTMRs may reflect a sample (type) specific
then this practice is used to access the degree of agreement
bias which is statistically observable, but which are less than
between PPTMRs and PTMRs. Otherwise, Practice D3764 is
the uncertainty of PPTMR, U(PPTMR). Therefore, the bias and
used to compare the PPTMRs to the PTMRs for this blendstock
precision of the PTMR/PPTMR differences are not used as the
to determine the degree of agreement. Since this second step
basis for local validation.
does not use spectroscopic data, the validation of the second
1.4.1.3 Based on SEC, and the leverage statistic, the uncer-
step is done using Practice D3764. If the first step uses a
tainty of each PPTMR, U(PPTMR) is calculated. During
multivariate spectrophotometric analyzer, then only samples
validation, for each non-outlier sample, a determination is
for which the spectra are not outliers relative to the multivariate
made as to whether the absolute difference between PPTMR
model are used in the second step. Note that the second model
and PTMR, |δ|, is less than or equal to U(PPTMR). Counts are
might accommodate variable levels of additive material addi-
maintained as to the total number of non-outlier validation
tion to the blend stock.
samples, and the number of samples for which |δ| is less than
1.2 Multiple physical, chemical, or quality properties of the
or equal to U(PPTMR). Given the total number of non-outlier
sample under test are typically predicted from a single spectral
validation samples, an inverse binomial distribution is used to
measurement. In applying this practice, each property predic-
calculate the minimum number of results for which |δ| must be
tion is validated separately. The separate validation procedures
less than U(PPTMR). If the number of results for which |δ| is
for each property may share common features, and be affected
less than U(PPTMR) is greater than or equal to this minimum,
by common effects, but the performance of each property
then the results are consistent with the expectations of the
multivariate model, and the analyzer passes local validation.
The calculations involved are described in detail in Section 11
This practice is under the jurisdiction of ASTM Committee D02 on Petroleum
and Annex A4.
Products, Liquid Fuels, and Lubricants and is the direct responsibility of Subcom-
mittee D02.25 on Performance Assessment and Validation of Process Stream
1.4.1.4 The user must establish that results that are consis-
Analyzer Systems.
tent with the expectations based on the multivariate model will
Current edition approved July 1, 2023. Published January 2024. Originally
be adequate for the intended application. A 95 % probability is
approved in 1997. Last previous edition approved in 2022 as D6122 – 22. DOI:
10.1520/D6122-23. recommended for the inverse binomial distribution calculation.
D6122 − 23
The user may adjust this based on the criticality of the 1.9 This practice is not intended as a quantitative perfor-
application. See Annex A4 for details. mance standard for the comparison of analyzers of different
design.
1.4.2 General Validation:
1.4.2.1 When the validation samples are of sufficient
1.10 Although this practice deals primarily with validation
number, and their compositional and property ranges are
of infrared and Raman analyzers, the procedures and statistical
comparable to that of the model calibration set, then a General
tests described herein are also applicable to other types of
Validation can be done.
analyzers which employ multivariate models.
1.4.2.2 General Validation is conducted by doing a D6708
1.11 This standard does not purport to address all of the
based assessment between results from the analyzer system (or
safety concerns, if any, associated with its use. It is the
subsystem) produced by application of the multivariate model,
responsibility of the user of this standard to establish appro-
(such results are herein referred to as PPTMRs), versus the
priate safety, health, and environmental practices and deter-
PTMRs for the same sample set. The system (or subsystem) is
mine the applicability of regulatory limitations prior to use.
considered to be validated if the D6708 meets the following
1.12 This international standard was developed in accor-
condition:
dance with internationally recognized principles on standard-
(1) No bias correction can statistically improve the agree-
ization established in the Decision on Principles for the
ment between the PPTMRs versus the PTMRs, and
Development of International Standards, Guides and Recom-
(2) R computed as per D6708 meets user-specified re-
xy
mendations issued by the World Trade Organization Technical
quirements.
Barriers to Trade (TBT) Committee.
1.4.2.3 For analyzers used in product release or product
quality certification applications, the precision and bias re- 2. Referenced Documents
quirement for the degree of agreement are typically based on 2
2.1 ASTM Standards:
the site or published precision of the PTM.
D86 Test Method for Distillation of Petroleum Products and
Liquid Fuels at Atmospheric Pressure
NOTE 1—In most applications of this type, the PTM is the specification-
cited test method. D1265 Practice for Sampling Liquefied Petroleum (LP)
Gases, Manual Method
1.4.2.4 This practice does not describe procedures for es-
D1319 Test Method for Hydrocarbon Types in Liquid Petro-
tablishing precision and bias requirements for analyzer system
leum Products by Fluorescent Indicator Adsorption
applications. Such requirements must be based on the critical-
D2699 Test Method for Research Octane Number of Spark-
ity of the results to the intended business application and on
Ignition Engine Fuel
contractual and regulatory requirements. The user must estab-
D3700 Practice for Obtaining LPG Samples Using a Float-
lish precision and bias requirements prior to initiating the
ing Piston Cylinder
validation procedures described herein.
D3764 Practice for Validation of the Performance of Process
1.5 This practice does not cover procedures for establishing
Stream Analyzer Systems
the calibration model (correlation) used by the analyzer.
D4057 Practice for Manual Sampling of Petroleum and
Calibration procedures are covered in Practice D8321 and
Petroleum Products
references therein.
D4177 Practice for Automatic Sampling of Petroleum and
Petroleum Products
1.6 This practice is intended as a review for experienced
D5599 Test Method for Determination of Oxygenates in
persons. For novices, this practice will serve as an overview of
Gasoline by Gas Chromatography and Oxygen Selective
techniques used to verify instrument performance, to verify
Flame Ionization Detection
model applicability to the spectrum of the sample under test,
D5769 Test Method for Determination of Benzene, Toluene,
and to verify that the degree of agreement between PPTMRs
and Total Aromatics in Finished Gasolines by Gas
and PTMRs meet user requirements.
Chromatography/Mass Spectrometry
1.7 This practice specifies appropriate statistical tools, out-
D5842 Practice for Sampling and Handling of Fuels for
lier detection methods, for determining whether the spectrum
Volatility Measurement
of the sample under test is a member of the population of
D6299 Practice for Applying Statistical Quality Assurance
spectra used for the analyzer calibration. The statistical tools
and Control Charting Techniques to Evaluate Analytical
are used to determine if the infrared measurement results in a
Measurement System Performance
valid property or parameter estimate.
D6708 Practice for Statistical Assessment and Improvement
of Expected Agreement Between Two Test Methods that
1.8 The outlier detection methods do not define criteria to
Purport to Measure the Same Property of a Material
determine whether the sample or the instrument is the cause of
D7278 Guide for Prediction of Analyzer Sample System Lag
an outlier measurement. Thus, the operator who is measuring
Times
samples on a routine basis will find criteria to determine that a
spectral measurement lies outside the calibration, but will not
have specific information on the cause of the outlier. This
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
practice does suggest methods by which instrument perfor-
contact ASTM Customer Service at [e-mail]. For Annual Book of ASTM
mance tests can be used to indicate if the outlier methods are
Standards volume information, refer to the standard’s Document Summary page on
responding to changes in the instrument response. the ASTM website.
D6122 − 23
D7453 Practice for Sampling of Petroleum Products for sampling include sample loop, sample conditioning system and
Analysis by Process Stream Analyzers and for Process excess sample return system (see Fig. 1 in D3764 for example).
Stream Analyzer System Validation Online analyzers that utilize insertion probes include fiber
D7808 Practice for Determining the Site Precision of a optics and sample probes.
Process Stream Analyzer on Process Stream Material
3.1.6.2 Discussion—At-line, field and laboratory analyzers
D7717 Practice for Preparing Volumetric Blends of Dena- include the instrument and all associated sample introduction
tured Fuel Ethanol and Gasoline Blendstocks for Labora-
apparatuses.
tory Analysis
3.1.7 between-method reproducibility (R ), n—a quantita-
XY
D7915 Practice for Application of Generalized Extreme
tive expression of the random error associated with the
Studentized Deviate (GESD) Technique to Simultane-
difference between two results obtained by different operators
ously Identify Multiple Outliers in a Data Set
using different apparatus and applying the two methods X and
D8009 Practice for Manual Piston Cylinder Sampling for
Y, respectively, each obtaining a single result on an identical
Volatile Crude Oils, Condensates, and Liquid Petroleum
test sample, when the methods have been assessed and an
Products
appropriate bias-correction has been applied in accordance
D8321 Practice for Development and Validation of Multi-
with this practice; it is defined as the 95 % confidence limit for
variate Analyses for Use in Predicting Properties of
the difference between two such single and independent
Petroleum Products, Liquid Fuels, and Lubricants based
results. D6708
on Spectroscopic Measurements
3.1.8 calibration samples, n—in multivariate spectroscopic
D8340 Practice for Performance-Based Qualification of
measurement, the set of samples with known (measured by the
Spectroscopic Analyzer Systems
PTM) component concentrations or property values that are
E131 Terminology Relating to Molecular Spectroscopy
used for creating a multivariate model. D8321
E275 Practice for Describing and Measuring Performance of
3.1.9 calibration transfer, n—a method of applying a mul-
Ultraviolet and Visible Spectrophotometers
E456 Terminology Relating to Quality and Statistics tivariate calibration developed using spectra from one analyzer
for analysis of spectra collected on a second analyzer by
E932 Practice for Describing and Measuring Performance of
Dispersive Infrared Spectrometers mathematically modifying the multivariate model or by instru-
ment standardization.
E1421 Practice for Describing and Measuring Performance
of Fourier Transform Mid-Infrared (FT-MIR) Spectrom-
3.1.10 control limits, n—limits on a control chart that are
eters: Level Zero and Level One Tests
used as criteria for signaling the need for action, or for judging
E1655 Practices for Infrared Multivariate Quantitative
whether a set of data does or does not indicate a state of
Analysis
statistical control. D6299
E1866 Guide for Establishing Spectrophotometer Perfor-
3.1.11 general validation, n—a comprehensive evaluation
mance Tests
of the agreement between the PPTMR and the PTMR done on
E1944 Practice for Describing and Measuring Performance
a set of samples that adequately span the multivariate model
of Laboratory Fourier Transform Near-Infrared (FT-NIR)
composition space using the statistical methodology of Practice
Spectrometers: Level Zero and Level One Tests
D6708 to demonstrate all required criteria in D6708 are met,
and R meets user requirements.
xy
3. Terminology
3.1.12 inlier, n—see nearest neighbor distance inlier.
3.1 Definitions:
3.1.13 inlier detection methods, n—statistical tests which
3.1.1 For definitions of terms and symbols relating to IR and
are conducted to determine if a spectrum resides within a
Raman spectroscopies, refer to Terminology E131.
region of the multivariate calibration space which is sparsely
3.1.2 For definitions of terms and symbols relating to
populated.
multivariate calibration, refer to Practices D8321.
3.1.3 For definitions of terms relating to statistical quality
3.1.14 instrument, n—for multivariate spectroscopic ana-
control, refer to Practice D6299 and Terminology E456.
lyzers used in the analysis of liquid petroleum products and
3.1.4 action limit, n—for multivariate spectroscopic analyz-
fuels, the spectrometer or spectrophotometer, associated elec-
ers used in the analysis of liquid petroleum products and fuels,
tronics and computer, spectrometer or spectrophotometer cell,
the limiting value from an instrument performance test, beyond
and if utilized, transfer optics.
which the analyzer is expected to produce potentially invalid
3.1.15 instrument performance verification sample (IPV
results.
sample), n—for multivariate spectroscopic analyzers used in
3.1.5 analyzer, n—see analyzer system.
the analysis of liquid petroleum products and fuels, a material
3.1.6 analyzer system, n—for equipment used in the analysis representative of the product being analyzed which is ad-
equately stored in sufficient quantity to be used as a check on
of liquid petroleum products and fuels, all piping, hardware,
computer, software, instrument, linear correlation or multivari- instrument performance; instrument performance verification
samples are used in instrument performance tests and as checks
ate model required to analyze a process or product sample; the
on calibration transfer, but the samples and their spectra are
analyzer system may also be referred to as the analyzer, or the
generally not reproducible long term.
total analyzer system. D3764
3.1.6.1 Discussion—Online analyzers that utilize extractive 3.1.15.1 Discussion—In E1866 and previous versions of
D6122 − 23
this practice, an instrument performance verification samples 3.1.23 multivariate calibration, n—an analyzer calibration
were referred to as test samples. that relates the spectrum at multiple wavelengths or frequen-
cies to the physical, chemical, or quality parameters.
3.1.16 instrument qualification sample (IQ sample), n—for
multivariate spectroscopic analyzers used in the analysis of
3.1.24 multivariate model, n—the mathematical expression
liquid petroleum products and fuels, a single pure compound,
or the set of mathematical operations that relates component
or a known, reproducible mixture of compounds whose spectra
concentrations or properties to spectra for a set of calibration
is constant over time such that it can be used in an instrument
samples.
performance test.
3.1.24.1 Discussion—The multivariate model includes any
3.1.16.1 Discussion—In E1866 and previous versions of
preprocessing done to the spectra or concentration or properties
this practice, an instrument qualification sample was referred to
prior to the development of the correlation between spectra and
as a check sample.
properties, and any post-processing done to the initially pre-
dicted results. D8321
3.1.17 instrument standardization, n—a procedure for stan-
dardizing the response of multiple instruments such that a
3.1.25 nearest neighbor distance inlier, n—the spectrum of
common multivariate model is applicable for measurements
a sample not used in the calibration which, when analyzed,
conducted by these instruments, the standardization being
resides within a gap in the multivariate calibration space, and
accomplished by way of adjustment of the spectrophotometer
for which the result is subject to possible interpolation error.
hardware or by way of mathematical treatment of the collected
3.1.26 outlier detection limits, n—the limiting value for
spectra.
application of an outlier detection method to a spectrum,
3.1.18 line sample, n—process material that can be safely
beyond which the spectrum represents an extrapolation of the
withdrawn from a sample port or associated facilities without
calibration model.
significantly altering the property of interest so that the
3.1.27 outlier detection methods, n—statistical tests which
material can be used to perform analyzer system validation; the
are conducted to determine if the analysis of a spectrum using
material is withdrawn in accordance with Practices D1265,
a multivariate model represents an interpolation of the model.
D3700, D4057, D4177, D5842, D7453, or D8009, whichever
3.1.28 outlier spectrum, n—a spectrum whose analysis by a
is applicable, during a period when the material flowing
multivariate model represents an extrapolation of the model.
through the analyzer is of uniform quality and the analyzer
results are practically constant. D3764
3.1.29 post-processing, v—performing a mathematical op-
3.1.19 liquid petroleum products and fuels, n—in relation to eration on an intermediate analyzer result to produce the final
process analyzers, any single-phase liquid material that is result, including correcting for temperature effects, adding a
produced at a facility in the petroleum and petrochemical mean property value of the analyzer calibration, and converting
industries and will be in whole or in part of a petroleum into appropriate units for reporting purposes.
product; it is inclusive of biofuels, renewable fuels,
3.1.30 Predicted Primary Test Method Result(s) (PPTMR),
blendstocks, alternative blendstocks, and additives. D8340
n—result(s) from the analyzer system, after application of any
3.1.20 local validation, n—an evaluation of the agreement
necessary correlation, that is interpreted as predictions of what
between the PPTMR and PTMR done on a set of samples that
the primary test method results would have been, if it was
do not necessarily span the compositional space of the multi-
conducted on the same material. D3764
variate model so as to demonstrate that the agreement is
3.1.31 prediction deviation(s) (δ), n—calculated differ-
consistent with expectations based on the multivariate model.
ence(s) (including algebraic sign) between predicted primary
3.1.21 model degrees of freedom (dof), n—the dimension of
test method result and primary test result, defined as (PPTMR
the multivariate space defined by the number of calibration
– PTMR).
sample spectra, the number of model variables, and the number
3.1.31.1 Discussion—This is also referred to as prediction
of variables used in defining the property level dependence of
residuals in Practice D6708.
the Standard Error of Calibration (SEC).
3.1.31.2 Discussion—Local validation uses the absolute
3.1.21.1 Discussion—For a multivariate model that is not
value of the prediction deviations, |δ|. D3764
mean-centered, dof = n-k-c, where n is the number of calibra-
3.1.32 preprocessing, v—performing mathematical opera-
tion samples, k is the number of model variables, and c is 0, 1
tions on raw spectral data prior to multivariate analysis or
or 2 depending on whether SEC is level independent, has a
model development, such as selecting spectral regions, correct-
linear dependence on property level, or has a power depen-
ing for baseline, smoothing, mean centering, and assigning
dence. For a mean-centered model, dof = n-k-c-1.
weights to certain spectral positions. D8321
3.1.22 model variables, n—the independent variables de-
3.1.33 primary test method (PTM), n—the analytical pro-
rived from the calibration spectra which are regressed against
cedure used to generate the reference values against which the
the calibration sample properties to produce the multivariate
analyzer is both calibrated and validated. D3764
model.
3.1.22.1 Discussion—For MLR, the model variables would 3.1.34 primary test method result(s) (PTMR), n—test re-
be the absorbance at the selected wavelengths or frequencies; sult(s) produced from an ASTM or other established standard
for PCR or PLS, the model variables are the Principal test method that is accepted as the reference measure of a
Components or latent variables. property. D3764
D6122 − 23
3.1.35 Standard Error of Calibration (SEC), n—a measure set of validation samples will not exceed the uncertainty of
of the agreement between PPTMR and PTMR for the samples PPTMR more than one time in 20 in the long term.
used in developing a multivariate model.
3.2.9 exponentially weighted moving average control chart,
3.1.35.1 Discussion—If the model error is level
n—a control chart based on the exponentially weighted average
n
of individual observations from a system; the observations may
independent, thenSEC5Œ ~PPTMR 2 PTMR ! ⁄ dof, where
( i i
i51
be the differences between the analyzer result, and the result
dof is the model degrees of freedom and n is the number of
from the primary test method.
calibration samples.
3.2.10 individual observation control chart, n—a control
3.1.35.2 Discussion—If the model error is level dependent,
chart of individual observations from a system; the observa-
then SEC is expressed as a function of m which is the average tions may be the differences between the analyzer result and
of PPTMR and PTMR, and SEC(m) is calculated using a
the result from the primary test method.
procedure described in Annex A4 and in Practice D8321 Annex
3.2.11 in-line probe, n—a spectrophotometer cell installed
A2.
in a process pipe or slip stream loop and connected to the
3.1.36 uncertainty of Predicted Primary Test Method Result analyzer by optical fibers.
(U(PPTMR)), n—the interval about PPTMR in which PTMR is
3.2.12 moving range of two control chart, n—a control chart
expected to occur 95 % of the time in the long run.
that monitors the change in the absolute value of the difference
3.1.36.1 Discussion—PTMR is expected to fall in the range
between two successive differences of the analyzer result
between PPTMR – U(PPTMR) and PPTMR + U(PPTMR)
minus the result from the primary test method.
95 % of the time over the long run.
3.2.13 optical background, n—the spectrum of radiation
3.1.36.2 Discussion—U(PPTMR) = t(p,dof) · SEC(m)
incident on a sample under test, typically obtained by measur-
=11h where t is the Student’s T value for probability level p
ing the radiation transmitted through the spectrophotometer
and model degrees of freedom dof, SEC(m) is the model
cell when no sample is present, or when an optically thin or
Standard Error of Calibration at property level m, where m is
nonabsorbing liquid is present.
the average of PPTMR and PTMR, and h is the leverage
3.2.14 performance test, n—a test that verifies that the
calculated for the spectrum being analyzed to produce PPTMR.
performance of the instrument is consistent with historical data
For more details, see Annex A4.
and adequate to produce valid results.
3.2 Definitions of Terms Specific to This Standard:
3.2.15 physical correction, n—a type of post-processing
3.2.1 analyzer calibration, n—see multivariate calibration.
where the correction made to the numerical value produced by
3.2.2 analyzer model, n—see multivariate model.
the multivariate model is based on a separate physical mea-
surement of, for example, sample density, sample path length,
3.2.3 analyzer repeatability, n—a statistical measure of the
or particulate scattering.
expected short-term variability of results produced by the
analyzer for samples whose spectra are neither outliers nor
3.2.16 process analyzer system, n—see analyzer system.
nearest neighbor inliers.
3.2.17 process analyzer validation samples, n—see valida-
3.2.4 analyzer result, n—the numerical estimate of a
tion samples.
physical, chemical, or quality parameter produced by applying
3.2.18 spectrometer cell, Raman, n—an apparatus which
the calibration model to the spectral data collected by the
allows a liquid hydrocarbon to flow past an optical surface or
analyzer.
surfaces that allow(s) transmission of the laser light into the
3.2.5 analyzer site precision, n—a statistical measure of the
sample and the generated Raman scattering light out of the
expected long-term variability of analyzer results for samples
sample.
whose spectra are neither outliers, nor nearest neighbor inliers.
3.2.19 spectrophotometer cell infrared, n—an apparatus
3.2.6 analyzer validation status, n—an indicator as to the
which allows a liquid hydrocarbon to flow between two optical
validity of analyzer results produced by application of the
surfaces which are separated by a fixed distance, the sample
multivariate model to spectra of the process sample.
path length, while simultaneously allowing light to pass
3.2.6.1 Discussion—Prior to the analyzer passing probation-
through the liquid.
ary local validation, the analyzer validation status and the
3.2.20 transfer optics, n—a device which allows movement
validity of the results is unknown; once the analyzer passes
of light from the spectrophotometer to a remote spectropho-
probationary local validation, the analyzer validation status is
tometer cell and back to the spectrophotometer; transfer optics
pass, and results are validated as long as the spectrum is not an
include optical fibers or other optical light pipes.
outlier or nearest neighbor inlier; if the analyzer fails proba-
3.2.21 validated result, n—a result produced by the analyzer
tionary or continual validation, the analyzer status is fail, and
for a sample whose spectrum is neither an outlier nor a nearest
analyzer results are not validated.
neighbor inlier that is equivalent, within control limits to the
3.2.7 analyzer validation test, n—see validation test.
result expected from the primary test method, so that the result
3.2.8 expectations based on the multivariate model, n—the can be used instead of the direct measurement of the sample by
absolute difference between the PPTMR and the PTMR for a the primary test method.
D6122 − 23
3.2.22 validation reference material (VRM), n—a line 3.3.21 SSE—the (unweighted) Sum of Squared Errors; a
sample retain, composite sample, or tank sample which is subscript C, L, or P indicates constant, linear, or power
representative of current production, has a measured PTMR, regression.
and is used in place of a line sample during the validation
3.3.22 t(p, dof)—Student’s T-value at probability p for dof
process.
degrees of freedom
3.2.23 validation samples, n—samples that are used to
3.3.23 U(PPTMR)—Uncertainty of PPTMR
compare the analyzer results to the primary test method results
3.3.24 WSSE—the Weighted Sum of Squared Errors; a
through the use of control charts and statistical tests; validation
subscript C, L, or P indicates constant, linear, or power
samples used in the initial validation may be line and instru-
regression.
ment performance verifications, whereas validation samples
3.3.25 WSSM—the Weighted Model Sum of Squares; a
used in the periodic validation are line samples.
subscript C, L, or P indicates constant, linear, or power
3.2.24 validation test, n—a test performed on a validation
regression.
sample that demonstrates that the result produced by the
3.3.26 WSST—the Total Sum of Squares; a subscript C, L, or
analyzer and the result produced by the primary test method are
P indicates constant, linear, or power regression.
equivalent to within control limits.
3.3.27 z —the transform of m ; z = max(m ) – m .
i i i i i
3.3 Symbols:
3.3.28 z¯ —the transform of m¯ ; z¯ = max(m¯ ) – m.
j j j j
3.3.1 c—the number of coefficients needed to describe the
property level dependence of SEC minus 1.
4. Summary of Practice
3.3.2 δ—difference between PPTMR and PTMR; |δ| indi-
4.1 This section describes, in summary form, the steps
cates the absolute value of the difference; δ indicates the
i
th involved in the validation of an infrared analyzer for prediction
difference for the i sample.
of a single physical, chemical, or quality property over the long
ˆ
3.3.3 δ —the estimate of δ produced by the SEC level
i i
term. If multiple properties are predicted from a single spectral
dependence model.
measurement, the validation of each property prediction is
3.3.4 π —the Root Mean Square Difference between considered a separate application of this practice. These
j
th
PPTMR and PTMR for the samples in the j subset. separate applications of the practice may share certain features,
but the analyzer performance for the prediction of each
ˆ
3.3.5 π —the estimate of π produced by a weighted regres-
j j
property is evaluated separately.
sion model.
4.2 Before this practice may be undertaken, certain precon-
¯
3.3.6 π—the weighted average of the π over the s subsets.
ditions shall be satisfied. The preconditions are described in
3.3.7 dof—model degrees of freedom
Section 7.
3.3.8 F—an F-ratio; subscript indicates whether the ratio is
4.3 This practice consists of five major procedures.
between WSSE values for (L-C) linear and constant, (P-C)
4.3.1 Each time a spectrum of a sample is collected using a
power and constant, or (P-L) power and linear SEC level
laboratory or process analyzer, statistical tests are performed to
dependence models.
verify that the multivariate model is applicable to the spectrum.
3.3.9 h—leverage statistic Only spectra whose analysis represents interpolation of the
multivariate model and which are sufficiently close to spectra
3.3.10 k—number of variables in multivariate model
in the calibration may be used in the analyzer validation.
th
3.3.11 m —the average of the PPTMR and PTMR for the i
i
4.3.2 When the analyzer is initially installed, or after major
sample.
maintenance is concluded, an analyzer qualification is per-
formed. Performance tests are conducted to verify that the
3.3.12 m¯ —the average of the m values for the samples in
i i
th
instrument is functioning properly. The intent of these tests is
the j subset.
to provide a rapid indication of the state of the instrument.
3.3.13 n—number of model calibration samples
These tests are necessary but not sufficient to demonstrate valid
3.3.14 PTM—Primary Test Method
analyzer results.
3.3.15 PTMR—Primary Test Method Result
NOTE 2—Major maintenance is any change to the analyzer system
hardware or software that is shown by historical data or simulations to
3.3.16 PPTMR—Predicted Primary Test Method Result
cause a statistically observable change in the analyzer performance
3.3.17 RMSEC—Root Mean Square Error of Calibration relative to before the maintenance. What constitutes major maintenance is
specific to the analyzer hardware and software employed. Users should
3.3.18 R —between method reproducibility
XY
consult the analyzer manufacturer as to what types of maintenance should
be considered major. Any maintenance which requires calibration transfer
3.3.19 s—the number of subsets used in the weighted
to be performed should be considered major maintenance. Any mainte-
regression.
nance for which performance changes are routinely compensated for in
analyzer software or in the multivariate model are not considered major
3.3.20 SEC—Standard Error of Calibration; SEC(m) indi-
maintenance.
cates SEC at property level m; a subscript C, L, or P after the
SEC indicates whether SEC has a constant, linear, or power 4.3.3 After the analyzer qualification is successfully
dependence on property level. completed, a probationary local validation test is conducted on
D6122 − 23
at least 20 samples that were not used in developing the 4.4 During routine operation of the analyzer, validation tests
multivariate model. The purpose of this probationary valida- are conducted on a regular, periodic basis to demonstrate that
tion is to verify that the results produced by the analyzer (the the analyzer results remain in statistical agreement with results
for the primary test method. Prediction deviations (δ) are
PPTMRs) agree with the results from the primary test method
(the PTMRs) to within expectations based on the multivariate monitored using statistical quality control charts at a frequency
model. As the spectra of these initial validation samples are that is commensurate with the criticality of the application.
Between validation tests, performance tests are conducted to
collected, they are analyzed with the multivariate model to
verify that the instrument is performing in a consistent fashion.
produce the PPTMRs. The absolute differences between
PPTMR and PTMRs, |δ|, are compared to the 95 % uncertain-
ties of the PPTMRs, U(PPTMR). If |δ| does not exceed 5. Significance and Use
U(PPTMR) for more than three samples, probationary local
5.1 The primary purpose of this practice is to permit the user
validation continues until 20 validation samples have been
to validate numerical values produced by a multivariate,
processed. If |δ| is less than U(PPTMR) for at least 17 of the 20
infrared or near-infrared laboratory or process (online or
initial validation samples, then the predictions are consistent
at-line) analyzer calibrated to measure a specific chemical
with the expectations based on the multivariate model, and the
concentration, chemical property, or physical property. If the
system or subsystem performance is considered to be proba-
analyzer results agree with the primary test method to within
tionary validated for materials and property ranges representa-
limits based on the multivariate model for the user-prespecified
tive of those used in the validation. providing that the spectra
statistical confidence level, these results can be considered
used in generating the results are neither outliers or nearest
’validated’ to the user pre-specified confidence limit for a
neighbor inliers. If |δ| is greater than U(PPTMR) for more than
specific application, and hence can be considered useful for
3 of the initial validation samples, the analyzer system fails
that specific application.
probationary validation. An investigation of the cause of the
5.2 Procedures are described for verifying that the
failure should be conducted, and corrective action taken. The
instrument, the model, and the analyzer system are stable and
validation process then restarts with initial performance test-
properly operating.
ing.
5.3 A multivariate analyzer system inherently utilizes a
NOTE 3—Probationary validation is done using the local validation
multivariate calibration model. In practice, the model both
procedure since 20 samples is too few to conduct a general validation. 20
samples would seldom if ever span the property and composition range of
implicitly and explicitly spans some subset of the population of
a multivariate model based on more than 3 variables.
all possible samples that could be in the complete multivariate
sample space. The model is applicable only to samples that fall
4.3.4 After probationary local validation is achieved, con-
within the subset population used in the model construction. A
tinued statistical quality control chart monitoring and analyses
sample measurement cannot be validated unless applicability is
of |δ| are carried out with new production samples to ensure
established. Applicability cannot be assumed.
ongoing prediction performance of the PPTMR meets the
levels established from the probationary validation. The |δ| are 5.3.1 Outlier detection methods are used to demonstrate
compared to U(PPTMR), and a count is maintained of the total
applicability of the calibration model for the analysis of the
number of non-outlier validation samples, and the number for process sample spectrum. The outlier detection limits are based
which |δ| is less than or equal to U(PPTMR). An inverse on historical as well as theoretical criteria. The outlier detection
binomial distribution is used to calculate the minimum number
methods are used to establish whether the results obtained by
of samples for which |δ| must be less than U(PPTMR). As long an analyzer are potentially valid. The validation procedures are
as this minimum is met, the analyzer system passes
...

This document is not an ASTM standard and is intended only to provide the user of an ASTM standard an indication of what changes have been made to the previous version. Because
it may not be technically possible to adequately depict all changes accurately, ASTM recommends that users consult prior editions as appropriate. In all cases only the current version
of the standard as published by ASTM is to be considered the official document.
Designation: D6122 − 22 D6122 − 23
Standard Practice for
Validation of the Performance of Multivariate Online, At-
Line, Field and Laboratory Infrared Spectrophotometer, and
Raman Spectrometer Based Analyzer Systems
This standard is issued under the fixed designation D6122; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
INTRODUCTION
Operation of a laboratory or process stream analyzer system typically involves five sequential
activities. (1) Correlation—Prior to the initiation of the procedures described in this practice, a
multivariate model is derived which relates the spectrum produced by the analyzer to the Primary Test
Method Result (PTMR). (1a) If the analyzer and Primary Test Method (PTM) measure the same
material, then the multivariate model directly relates the spectra to PTMR collected on the same
samples. Alternatively (1b) if the analyzer measures the spectra of a material that is subjected to
treatment prior to being measured by the PTM, then the multivariate model relates the spectra of
the untreated sample to the PTMR for the same sample after treatment. (2) Analyzer Qualification—
When an analyzer is initially installed, or after major maintenance has been performed, diagnostic
testing is performed to demonstrate that the analyzer meets the manufacturer’s specifications and
historical performance standards. These diagnostic tests may require that the analyzer be adjusted so
as to provide predetermined output levels for certain reference materials (3) Local Validation—A
local validation is performed using an independent but limited set of materials that were not part of
the correlation activity. This local validation is intended to demonstrate that the agreement between the
Predicted Primary Method Test Results (PPTMRs) and the PTMRs are consistent with expectations
based on the multivariate model. (4) General Validation—After an adequate number of PPTMRs and
PTMRs have been accrued on materials that were not part of the correlation activity and which
adequately span the multivariate model compositional space, a comprehensive statistical assessment
can be performed to demonstrate that the PPTMRs agree with the PTMRs to within user-specified
requirements. (5) Continual Validation—Subsequent to a successful local or general validation,
quality assurance control chart monitoring of the differences between PPTMR and PTMR is conducted
during normal operation of the process analyzer system to demonstrate that the agreement between the
PPTMRs and the PTMRs established during the General Validation is maintained. This practice deals
with the third, fourth, and fifth of these activities.
“Correlation where analyzer measures a material which is subjected to treatment before being
measured by the PTM” as outlined in this practice can be applied to biofuels where the biofuel
material is added at a terminal or other facility and not included in the process stream material sampled
by the analyzer at the basestock manufacturing facility. The “treatment” shall be a constant percentage
addition of the biofuels material to the basestock material. The correlation is deemed valid only for
the specific percentage addition and type of biofuel material used in its development.
This practice is under the jurisdiction of ASTM Committee D02 on Petroleum Products, Liquid Fuels, and Lubricants and is the direct responsibility of Subcommittee
D02.25 on Performance Assessment and Validation of Process Stream Analyzer Systems.
Current edition approved April 1, 2022July 1, 2023. Published June 2022January 2024. Originally approved in 1997. Last previous edition approved in 20212022 as
D6122 – 21.D6122 – 22. DOI: 10.1520/D6122-22.10.1520/D6122-23.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
D6122 − 23
1. Scope*
1.1 This practice covers requirements for the validation of measurements made by laboratory, field, or process (online or at-line)
infrared (near- or mid-infrared analyzers, or both), and Raman analyzers, used in the calculation of physical, chemical, or quality
parameters (that is, properties) of liquid petroleum products and fuels. The properties are calculated from spectroscopic data using
multivariate modeling methods. The requirements include verification of adequate instrument performance, verification of the
applicability of the calibration model to the spectrum of the sample under test, and verification that the uncertainties associated
with the degree of agreement between the results calculated from the infrared or Raman measurements and the results produced
by the PTM used for the development of the calibration model meets user-specified requirements. Initially, a limited number of
validation samples representative of current production are used to do a local validation. When there is an adequate number of
validation samples with sufficient variation in both property level and sample composition to span the model calibration space, the
statistical methodology of Practice D6708 can be used to provide general validation of this equivalence over the complete
operating range of the analyzer. For cases where adequate property and composition variation is not achieved, local validation shall
continue to be used.
1.1.1 For some applications, the analyzer and PTM are applied to the same material. The application of the multivariate model
to the analyzer output (spectrum) directly produces a PPTMR for the same material for which the spectrum was measured. The
PPTMRs are compared to the PTMRs measured on the same materials to determine the degree of agreement.
1.1.2 For other applications, the material measured by the analyzer system is subjected to a consistent additive treatment prior to
being analyzed by the PTM. The application of the multivariate model to the analyzer output (spectrum) produces a PPTMR for
the treated material. The PPTMRs based on the analyzer outputs are compared to the PTMRs measured on the treated materials
to determine the degree of agreement.
1.1.3 In some cases, a two-step procedure is employed. In the first step, the analyzer and PTM are applied to the measurement
of a blendstock material. In a second step, the PPTMRs produced in Step 1 are used as inputs to a second model that predicts the
results obtained when the PTM is applied to the analysis of the finished blended product produced by additivation to the
blendstock. If the analyzer used in the first step is a multivariate spectroscopic based analyzer, then this practice is used to access
the degree of agreement between PPTMRs and PTMRs. Otherwise, Practice D3764 is used to compare the PPTMRs to the PTMRs
for this blendstock to determine the degree of agreement. Since this second step does not use spectroscopic data, the validation
of the second step is done using Practice D3764. If the first step uses a multivariate spectrophotometric analyzer, then only samples
for which the spectra are not outliers relative to the multivariate model are used in the second step. Note that the second model
might accommodate variable levels of additive material addition to the blend stock.
1.2 Multiple physical, chemical, or quality properties of the sample under test are typically predicted from a single spectral
measurement. In applying this practice, each property prediction is validated separately. The separate validation procedures for
each property may share common features, and be affected by common effects, but the performance of each property prediction
is evaluated independently. The user will typically have multiple validation procedures running simultaneously in parallel.
1.3 Results used in analyzer validation are for samples that were not used in the development of the multivariate model, and for
spectra which are not outliers or nearest neighbor inliers relative to the multivariate model.
1.4 When the number, composition range or property range of available validation samples do not span the model calibration
range, a local validation is done using available samples representative of current production. When the number, composition range
and property range of available validation samples becomes comparable to those of the model calibration set, a general validation
can be done.
1.4.1 Local Validation:
1.4.1.1 The calibration samples used in developing the multivariate model must show adequate compositional and property
variation to enable the development of a meaningful correlation, and must span the compositional range of samples to be analyzed
using the model to ensure that such analyses are done via interpolation rather than extrapolation. The Standard Error of Calibration
(SEC) is a measure of how well the PTMRs and PPTMRs agree for this set of calibration samples. SEC includes contributions
from spectrum measurement error, PTM measurement error, and model error. Sample (type) specific biases are a part of the model
error. Typically, spectroscopic analyzers are very precise, so that spectral measurement error is small relative to the other types of
error.
D6122 − 23
1.4.1.2 During initial analyzer validation, the compositional range of available samples may be small relative to the range of the
calibration set. Because of the high precision of the spectroscopic measurement, the average difference between the PTMRs and
PPTMRs may reflect a sample (type) specific bias which is statistically observable, but which are less than the 95 % uncertainty
of PPTMR, U(PPTMR). Therefore, the bias and precision of the PTMR/PPTMR differences are not used as the basis for local
validation.
1.4.1.3 Based on SEC, and the leverage statistic, a 95 % uncertainty forthe uncertainty of each PPTMR, U(PPTMR) is calculated.
During validation, for each non-outlier sample, a determination is made as to whether the absolute difference between PPTMR and
PTMR, |δ|, is less than or equal to U(PPTMR). Counts are maintained as to the total number of non-outlier validation samples, and
the number of samples for which |δ| is less than or equal to U(PPTMR). Given the total number of non-outlier validation samples,
an inverse binomial distribution is used to calculate the minimum number of results for which |δ| must be less than U(PPTMR).
If the number of results for which |∆||δ| is less than U(PPTMR) is greater than or equal to this minimum, then the results are
consistent with the expectations of the multivariate model, and the analyzer passes local validation. The calculations involved are
described in detail in Section 11 and Annex A4.
1.4.1.4 The user must establish that results that are consistent with the expectations based on the multivariate model will be
adequate for the intended application. A 95 % probability is recommended for the inverse binomial distribution calculation. The
user may adjust this based on the criticality of the application. See Annex A4 for details.
1.4.2 General Validation:
1.4.2.1 When the validation samples are of sufficient number, and their compositional and property ranges are comparable to that
of the model calibration set, then a General Validation can be done.
1.4.2.2 General Validation is conducted by doing a D6708 based assessment between results from the analyzer system (or
subsystem) produced by application of the multivariate model, (such results are herein referred to as PPTMRs), versus the PTMRs
for the same sample set. The system (or subsystem) is considered to be validated if the D6708 meets the following condition:
(1) No bias correction can statistically improve the agreement between the PPTMRs versus the PTMRs, and
(2) R computed as per D6708 meets user-specified requirements.
xy
1.4.2.3 For analyzers used in product release or product quality certification applications, the precision and bias requirement for
the degree of agreement are typically based on the site or published precision of the PTM.
NOTE 1—In most applications of this type, the PTM is the specification-cited test method.
1.4.2.4 This practice does not describe procedures for establishing precision and bias requirements for analyzer system
applications. Such requirements must be based on the criticality of the results to the intended business application and on
contractual and regulatory requirements. The user must establish precision and bias requirements prior to initiating the validation
procedures described herein.
1.5 This practice does not cover procedures for establishing the calibration model (correlation) used by the analyzer. Calibration
procedures are covered in Practice D8321 and references therein.
1.6 This practice is intended as a review for experienced persons. For novices, this practice will serve as an overview of techniques
used to verify instrument performance, to verify model applicability to the spectrum of the sample under test, and to verify that
the degree of agreement between PPTMRs and PTMRs meet user requirements.
1.7 This practice specifies appropriate statistical tools, outlier detection methods, for determining whether the spectrum of the
sample under test is a member of the population of spectra used for the analyzer calibration. The statistical tools are used to
determine if the infrared measurement results in a valid property or parameter estimate.
1.8 The outlier detection methods do not define criteria to determine whether the sample or the instrument is the cause of an outlier
measurement. Thus, the operator who is measuring samples on a routine basis will find criteria to determine that a spectral
D6122 − 23
measurement lies outside the calibration, but will not have specific information on the cause of the outlier. This practice does
suggest methods by which instrument performance tests can be used to indicate if the outlier methods are responding to changes
in the instrument response.
1.9 This practice is not intended as a quantitative performance standard for the comparison of analyzers of different design.
1.10 Although this practice deals primarily with validation of infrared and Raman analyzers, the procedures and statistical tests
described herein are also applicable to other types of analyzers which employ multivariate models.
1.11 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility
of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of
regulatory limitations prior to use.
1.12 This international standard was developed in accordance with internationally recognized principles on standardization
established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued
by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
2. Referenced Documents
2.1 ASTM Standards:
D86 Test Method for Distillation of Petroleum Products and Liquid Fuels at Atmospheric Pressure
D1265 Practice for Sampling Liquefied Petroleum (LP) Gases, Manual Method
D1319 Test Method for Hydrocarbon Types in Liquid Petroleum Products by Fluorescent Indicator Adsorption
D2699 Test Method for Research Octane Number of Spark-Ignition Engine Fuel
D3700 Practice for Obtaining LPG Samples Using a Floating Piston Cylinder
D3764 Practice for Validation of the Performance of Process Stream Analyzer Systems
D4057 Practice for Manual Sampling of Petroleum and Petroleum Products
D4177 Practice for Automatic Sampling of Petroleum and Petroleum Products
D5599 Test Method for Determination of Oxygenates in Gasoline by Gas Chromatography and Oxygen Selective Flame
Ionization Detection
D5769 Test Method for Determination of Benzene, Toluene, and Total Aromatics in Finished Gasolines by Gas
Chromatography/Mass Spectrometry
D5842 Practice for Sampling and Handling of Fuels for Volatility Measurement
D6299 Practice for Applying Statistical Quality Assurance and Control Charting Techniques to Evaluate Analytical Measure-
ment System Performance
D6708 Practice for Statistical Assessment and Improvement of Expected Agreement Between Two Test Methods that Purport
to Measure the Same Property of a Material
D7278 Guide for Prediction of Analyzer Sample System Lag Times
D7453 Practice for Sampling of Petroleum Products for Analysis by Process Stream Analyzers and for Process Stream Analyzer
System Validation
D7808 Practice for Determining the Site Precision of a Process Stream Analyzer on Process Stream Material
D7717 Practice for Preparing Volumetric Blends of Denatured Fuel Ethanol and Gasoline Blendstocks for Laboratory Analysis
D7915 Practice for Application of Generalized Extreme Studentized Deviate (GESD) Technique to Simultaneously Identify
Multiple Outliers in a Data Set
D8009 Practice for Manual Piston Cylinder Sampling for Volatile Crude Oils, Condensates, and Liquid Petroleum Products
D8321 Practice for Development and Validation of Multivariate Analyses for Use in Predicting Properties of Petroleum
Products, Liquid Fuels, and Lubricants based on Spectroscopic Measurements
D8340 Practice for Performance-Based Qualification of Spectroscopic Analyzer Systems
E131 Terminology Relating to Molecular Spectroscopy
E275 Practice for Describing and Measuring Performance of Ultraviolet and Visible Spectrophotometers
E456 Terminology Relating to Quality and Statistics
E932 Practice for Describing and Measuring Performance of Dispersive Infrared Spectrometers
E1421 Practice for Describing and Measuring Performance of Fourier Transform Mid-Infrared (FT-MIR) Spectrometers: Level
Zero and Level One Tests
E1655 Practices for Infrared Multivariate Quantitative Analysis
E1866 Guide for Establishing Spectrophotometer Performance Tests
For referenced ASTM standards, visit the ASTM website, www.astm.org, or contact ASTM Customer Service at [e-mail]. For Annual Book of ASTM Standards
volume information, refer to the standard’s Document Summary page on the ASTM website.
D6122 − 23
E1944 Practice for Describing and Measuring Performance of Laboratory Fourier Transform Near-Infrared (FT-NIR)
Spectrometers: Level Zero and Level One Tests
3. Terminology
3.1 Definitions:
3.1.1 For definitions of terms and symbols relating to IR and Raman spectroscopies, refer to Terminology E131.
3.1.2 For definitions of terms and symbols relating to multivariate calibration, refer to Practices D8321.
3.1.3 For definitions of terms relating to statistical quality control, refer to Practice D6299 and Terminology E456.
3.1.4 action limit, n—for multivariate spectroscopic analyzers used in the analysis of liquid petroleum products and fuels, the
limiting value from an instrument performance test, beyond which the analyzer is expected to produce potentially invalid results.
3.1.5 analyzer, n—see analyzer system.
3.1.6 analyzer system, n—for equipment used in the analysis of liquid petroleum products and fuels, all piping, hardware,
computer, software, instrument, linear correlation or multivariate model required to analyze a process or product sample; the
analyzer system may also be referred to as the analyzer, or the total analyzer system. D3764
3.1.6.1 Discussion—
Online analyzers that utilize extractive sampling include sample loop, sample conditioning system and excess sample return system
(see Fig. 1 in D3764 for example). Online analyzers that utilize insertion probes include fiber optics and sample probes.
3.1.6.2 Discussion—
At-line, field and laboratory analyzers include the instrument and all associated sample introduction apparatuses.
3.1.7 between-method reproducibility (R ), n—a quantitative expression of the random error associated with the difference
XY
between two results obtained by different operators using different apparatus and applying the two methods X and Y, respectively,
each obtaining a single result on an identical test sample, when the methods have been assessed and an appropriate bias-correction
has been applied in accordance with this practice; it is defined as the 95 % confidence limit for the difference between two such
single and independent results. D6708
3.1.8 calibration samples, n—in multivariate spectroscopic measurement, the set of samples with known (measured by the PTM)
component concentrations or property values that are used for creating a multivariate model. D8321
3.1.9 calibration transfer, n—a method of applying a multivariate calibration developed using spectra from one analyzer for
analysis of spectra collected on a second analyzer by mathematically modifying the multivariate model or by instrument
standardization.
3.1.10 control limits, n—limits on a control chart that are used as criteria for signaling the need for action, or for judging whether
a set of data does or does not indicate a state of statistical control. D6299
3.1.11 general validation, n—a comprehensive evaluation of the agreement between the PPTMR and the PTMR done on a set of
samples that adequately span the multivariate model composition space using the statistical methodology of Practice D6708 to
demonstrate all required criteria in D6708 are met, and R meets user requirements.
xy
3.1.12 inlier, n—see nearest neighbor distance inlier.
3.1.13 inlier detection methods, n—statistical tests which are conducted to determine if a spectrum resides within a region of the
multivariate calibration space which is sparsely populated.
3.1.14 instrument, n—for multivariate spectroscopic analyzers used in the analysis of liquid petroleum products and fuels, the
spectrometer or spectrophotometer, associated electronics and computer, spectrometer or spectrophotometer cell, and if utilized,
transfer optics.
3.1.15 instrument performance verification sample, sample (IPV sample), n—for multivariate spectroscopic analyzers used in the
D6122 − 23
analysis of liquid petroleum products and fuels, a material representative of the product being analyzed which is adequately stored
in sufficient quantity to be used as a check on instrument performance; instrument performance verification samples are used in
instrument performance tests and as checks on calibration transfer, but the samples and their spectra are generally not reproducible
long term.
3.1.15.1 Discussion—
In E1866 and previous versions of this practice, an instrument performance verification samples were referred to as test samples.
3.1.16 instrument qualification sample, sample (IQ sample), n—for multivariate spectroscopic analyzers used in the analysis of
liquid petroleum products and fuels, a single pure compound, or a known, reproducible mixture of compounds whose spectra is
constant over time such that it can be used in an instrument performance test.
3.1.16.1 Discussion—
In E1866 and previous versions of this practice, an instrument qualification sample was referred to as a check sample.
3.1.17 instrument standardization, n—a procedure for standardizing the response of multiple instruments such that a common
multivariate model is applicable for measurements conducted by these instruments, the standardization being accomplished by way
of adjustment of the spectrophotometer hardware or by way of mathematical treatment of the collected spectra.
3.1.18 line sample, n—process material that can be safely withdrawn from a sample port or associated facilities without
significantly altering the property of interest so that the material can be used to perform analyzer system validation; the material
is withdrawn in accordance with Practices D1265, D3700, D4057, D4177, D5842, D7453, or D8009, whichever is applicable,
during a period when the material flowing through the analyzer is of uniform quality and the analyzer results are practically
constant. D3764
3.1.19 liquid petroleum products and fuels, n—in relation to process analyzers, any single-phase liquid material that is produced
at a facility in the petroleum and petrochemical industries and will be in whole or in part of a petroleum product; it is inclusive
of biofuels, renewable fuels, blendstocks, alternative blendstocks, and additives. D8340
3.1.20 local validation, n—an evaluation of the agreement between the PPTMR and PTMR done on a set of samples that do not
necessarily span the compositional space of the multivariate model so as to demonstrate that the agreement is consistent with
expectations based on the multivariate model.
3.1.21 model degrees of freedom (dof), n—the dimension of the multivariate space defined by the number of calibration sample
spectra, the number of model variables, and the number of variables used in defining the property level dependence of the Standard
Error of Calibration (SEC).
3.1.21.1 Discussion—
For a multivariate model that is not mean-centered, dof = n-k-c, where n is the number of calibration samples, k is the number of
model variables, and c is 0, 1 or 2 depending on whether SEC is level independent, has a linear dependence on property level, or
has a power dependence. For a mean-centered model, dof = n-k-c-1.
3.1.22 model variables, n—the independent variables derived from the calibration spectra which are regressed against the
calibration sample properties to produce the multivariate model.
3.1.22.1 Discussion—
For MLR, the model variables would be the absorbance at the selected wavelengths or frequencies; for PCR or PLS, the model
variables are the Principal Components or latent variables.
3.1.23 multivariate calibration, n—an analyzer calibration that relates the spectrum at multiple wavelengths or frequencies to the
physical, chemical, or quality parameters.
3.1.24 multivariate model, n—the mathematical expression or the set of mathematical operations that relates component
concentrations or properties to spectra for a set of calibration samples.
3.1.24.1 Discussion—
The multivariate model includes any preprocessing done to the spectra or concentration or properties prior to the development of
the correlation between spectra and properties, and any post-processing done to the initially predicted results. D8321
3.1.25 nearest neighbor distance inlier, n—the spectrum of a sample not used in the calibration which, when analyzed, resides
within a gap in the multivariate calibration space, and for which the result is subject to possible interpolation error.
D6122 − 23
3.1.26 outlier detection limits, n—the limiting value for application of an outlier detection method to a spectrum, beyond which
the spectrum represents an extrapolation of the calibration model.
3.1.27 outlier detection methods, n—statistical tests which are conducted to determine if the analysis of a spectrum using a
multivariate model represents an interpolation of the model.
3.1.28 outlier spectrum, n—a spectrum whose analysis by a multivariate model represents an extrapolation of the model.
3.1.29 post-processing, v—performing a mathematical operation on an intermediate analyzer result to produce the final result,
including correcting for temperature effects, adding a mean property value of the analyzer calibration, and converting into
appropriate units for reporting purposes.
3.1.30 Predicted Primary Test Method Result(s) (PPTMR), n—result(s) from the analyzer system, after application of any
necessary correlation, that is interpreted as predictions of what the primary test method results would have been, if it was conducted
on the same material. D3764
3.1.31 prediction deviation(s) (δ),n—calculated difference(s) (including algebraic sign) between predicted primary test method
result and primary test result, defined as (PPTMR – PTMR).
3.1.31.1 Discussion—
This is also referred to as prediction residuals in Practice D6708.
3.1.31.2 Discussion—
Local validation uses the absolute value of the prediction deviations, |δ|. D3764
3.1.32 preprocessing, v—performing mathematical operations on raw spectral data prior to multivariate analysis or model
development, such as selecting spectral regions, correcting for baseline, smoothing, mean centering, and assigning weights to
certain spectral positions. D8321
3.1.33 primary test method (PTM), n—the analytical procedure used to generate the reference values against which the analyzer
is both calibrated and validated. D3764
3.1.34 primary test method result(s) (PTMR), n—test result(s) produced from an ASTM or other established standard test method
that is accepted as the reference measure of a property. D3764
3.1.35 Standard Error of Calibration (SEC), n—a measure of the agreement between PPTMR and PTMR for the samples used
in developing a multivariate model.
3.1.35.1 Discussion—
n
If the model error is level independent, thenSEC5Œ ~PPTMR 2 PTMR ! ⁄dof, where dof is the model degrees of freedom and n
( i i
i51
is the number of calibration samples.
3.1.35.2 Discussion—
If the model error is level dependent, then SEC is expressed as a function of m which is the average of PPTMR and PTMR, and
SEC(m) is calculated using a procedure described in Annex A4 and in Practice D8321 Annex A2.
3.1.36 uncertainty of Predicted Primary Test Method Result (U(PPTMR)), n—the interval about PPTMR in which PTMR is
expected to occur 95 % of the time in the long run.
3.1.36.1 Discussion—
PTMR is expected to fall in the range between PPTMR – U(PPTMR) and PPTMR + U(PPTMR) 95 % of the time over the long
run.
3.1.36.2 Discussion—
U(PPTMR) = t(p,dof) · SEC(m) =11h where t is the Student’s T value for probability level p and model degrees of freedom
dof,SEC(m) is the model Standard Error of Calibration at property level m, where m is the average of PPTMR and PTMR, and
h is the leverage calculated for the spectrum being analyzed to produce PPTMR. For more details, see Annex A4.
3.2 Definitions of Terms Specific to This Standard:
D6122 − 23
3.2.1 analyzer calibration, n—see multivariate calibration.
3.2.2 analyzer model, n—see multivariate model.
3.2.3 analyzer repeatability, n—a statistical measure of the expected short-term variability of results produced by the analyzer for
samples whose spectra are neither outliers nor nearest neighbor inliers.
3.2.4 analyzer result, n—the numerical estimate of a physical, chemical, or quality parameter produced by applying the calibration
model to the spectral data collected by the analyzer.
3.2.5 analyzer site precision, n—a statistical measure of the expected long-term variability of analyzer results for samples whose
spectra are neither outliers, nor nearest neighbor inliers.
3.2.6 analyzer validation status, n—an indicator as to the validity of analyzer results produced by application of the multivariate
model to spectra of the process sample.
3.2.6.1 Discussion—
Prior to the analyzer passing probationary local validation, the analyzer validation status and the validity of the results is unknown;
once the analyzer passes probationary local validation, the analyzer validation status is pass, and results are validated as long as
the spectrum is not an outlier or nearest neighbor inlier; if the analyzer fails probationary or continual validation, the analyzer status
is fail, and analyzer results are not validated.
3.2.7 analyzer validation test, n—see validation test.
3.2.8 expectations based on the multivariate model, n—the absolute difference between the PPTMR and the PTMR for a set of
validation samples will not exceed the uncertainty on the of PPTMR more than one time in 20 in the long term.
3.2.9 exponentially weighted moving average control chart, n—a control chart based on the exponentially weighted average of
individual observations from a system; the observations may be the differences between the analyzer result, and the result from
the primary test method.
3.2.10 individual observation control chart, n—a control chart of individual observations from a system; the observations may be
the differences between the analyzer result and the result from the primary test method.
3.2.11 in-line probe, n—a spectrophotometer cell installed in a process pipe or slip stream loop and connected to the analyzer by
optical fibers.
3.2.12 moving range of two control chart, n—a control chart that monitors the change in the absolute value of the difference
between two successive differences of the analyzer result minus the result from the primary test method.
3.2.13 optical background, n—the spectrum of radiation incident on a sample under test, typically obtained by measuring the
radiation transmitted through the spectrophotometer cell when no sample is present, or when an optically thin or nonabsorbing
liquid is present.
3.2.14 performance test, n—a test that verifies that the performance of the instrument is consistent with historical data and
adequate to produce valid results.
3.2.15 physical correction, n—a type of post-processing where the correction made to the numerical value produced by the
multivariate model is based on a separate physical measurement of, for example, sample density, sample path length, or particulate
scattering.
3.2.16 process analyzer system, n—see analyzer system.
3.2.17 process analyzer validation samples, n—see validation samples.
D6122 − 23
3.2.18 spectrometer cell, Raman, n—an apparatus which allows a liquid hydrocarbon to flow past an optical surface or surfaces
that allow(s) transmission of the laser light into the sample and the generated Raman scattering light out of the sample.
3.2.19 spectrophotometer cell infrared, n—an apparatus which allows a liquid hydrocarbon to flow between two optical surfaces
which are separated by a fixed distance, the sample path length, while simultaneously allowing light to pass through the liquid.
3.2.20 transfer optics, n—a device which allows movement of light from the spectrophotometer to a remote spectrophotometer
cell and back to the spectrophotometer; transfer optics include optical fibers or other optical light pipes.
3.2.21 uncertainty of Predicted Primary Test Method Result (U(PPTMR)), n—the interval about PPTMR in which PTMR is
expected to occur 95 % of the time in the long run.
3.2.21.1 Discussion—
PTMR is expected to fall in the range between PPTMR – U(PPTMR) and PPTMR + U(PPTMR) 95 % of the time over the long
run.
3.2.21.2 Discussion—
=
U(PPTMR) = t(p,dof) · SEC(m) · 11h where t is the Student’s T value for probability level p and model degrees of freedom
dof,SEC(m) is the model Standard Error of Calibration at property level m, where m is the average of PPTMR and PTMR, and
h is the leverage calculated for the spectrum being analyzed to produce PPTMR. For more details, see Annex A4.
3.2.21 validated result, n—a result produced by the analyzer for a sample whose spectrum is neither an outlier nor a nearest
neighbor inlier that is equivalent, within control limits to the result expected from the primary test method, so that the result can
be used instead of the direct measurement of the sample by the primary test method.
3.2.22 validation reference material (VRM), n—a line sample retain, composite sample, or tank sample which is representative of
current production, has a measured PTMR, and is used in place of a line sample during the validation process.
3.2.23 validation samples, n—samples that are used to compare the analyzer results to the primary test method results through the
use of control charts and statistical tests; validation samples used in the initial validation may be line and instrument performance
verifications, whereas validation samples used in the periodic validation are line samples.
3.2.24 validation test, n—a test performed on a validation sample that demonstrates that the result produced by the analyzer and
the result produced by the primary test method are equivalent to within control limits.
3.3 Symbols:
3.3.1 c—the number of coefficients needed to describe the property level dependence of SEC minus 1.
3.3.2 δ—difference between PPTMR and PTMR; |δ| indicates the absolute value of the difference; δ indicates the difference for
i
th
the i sample.
ˆ
3.3.3 δ —the estimate of δ produced by the SEC level dependence model.
i i
th
3.3.4 π —the Root Mean Square Difference between PPTMR and PTMR for the samples in the j subset.
j
ˆ
3.3.5 π —the estimate of π produced by a weighted regression model.
j j
¯
3.3.6 π—the weighted average of the π over the s subsets.
3.3.7 dof—model degrees of freedom
3.3.8 F—an F-ratio; subscript indicates whether the ratio is between WSSE values for (L-C) linear and constant, (P-C) power and
constant, or (P-L) power and linear SEC level dependence models.
3.3.9 h—leverage statistic
D6122 − 23
3.3.10 k—number of variables in multivariate model
th
3.3.11 m —the average of the PPTMR and PTMR for the i sample.
i
th
3.3.12 m¯ —the average of the m values for the samples in the j subset.
i i
3.3.13 n—number of model calibration samples
3.3.14 PTM—Primary Test Method
3.3.15 PTMR—Primary Test Method Result
3.3.16 PPTMR—Predicted Primary Test Method Result
3.3.17 RMSEC—Root Mean Square Error of Calibration
3.3.18 R —between method reproducibility
XY
3.3.19 s—the number of subsets used in the weighted regression.
3.3.20 SEC—Standard Error of Calibration; SEC(m) indicates SEC at property level m; a subscript C,L, or P after the SEC
indicates whether SEC has a constant, linear, or power dependence on property level.
3.3.21 SSE—the (unweighted) Sum of Squared Errors; a subscript C, L, or P indicates constant, linear, or power regression.
3.3.22 t(p, dof)—Student’s T-value at probability p for dof degrees of freedom
3.3.23 U(PPTMR)—Uncertainty onof PPTMR
3.3.24 WSSE—the Weighted Sum of Squared Errors; a subscript C,L, or P indicates constant, linear, or power regression.
3.3.25 WSSM—the Weighted Model Sum of Squares; a subscript C,L, or P indicates constant, linear, or power regression.
3.3.26 WSST—the Total Sum of Squares; a subscript C,L, or P indicates constant, linear, or power regression.
3.3.27 z —the transform of m ; z = max(m ) – m .
i i i i i
3.3.28 z¯ —the transform of m¯ ; z¯ = max(m¯ ) – m.
j j j j
4. Summary of Practice
4.1 This section describes, in summary form, the steps involved in the validation of an infrared analyzer for prediction of a single
physical, chemical, or quality property over the long term. If multiple properties are predicted from a single spectral measurement,
the validation of each property prediction is considered a separate application of this practice. These separate applications of the
practice may share certain features, but the analyzer performance for the prediction of each property is evaluated separately.
4.2 Before this practice may be undertaken, certain preconditions shall be satisfied. The preconditions are described in Section 7.
4.3 This practice consists of five major procedures.
4.3.1 Each time a spectrum of a sample is collected using a laboratory or process analyzer, statistical tests are performed to verify
D6122 − 23
that the multivariate model is applicable to the spectrum. Only spectra whose analysis represents interpolation of the multivariate
model and which are sufficiently close to spectra in the calibration may be used in the analyzer validation.
4.3.2 When the analyzer is initially installed, or after major maintenance is concluded, an analyzer qualification is performed.
Performance tests are conducted to verify that the instrument is functioning properly. The intent of these tests is to provide a rapid
indication of the state of the instrument. These tests are necessary but not sufficient to demonstrate valid analyzer results.
NOTE 2—Major maintenance is any change to the analyzer system hardware or software that is shown by historical data or simulations to cause a
statistically observable change in the analyzer performance relative to before the maintenance. What constitutes major maintenance is specific to the
analyzer hardware and software employed. Users should consult the analyzer manufacturer as to what types of maintenance should be considered major.
Any maintenance which requires calibration transfer to be performed should be considered major maintenance. Any maintenance for which performance
changes are routinely compensated for in analyzer software or in the multivariate model are not considered major maintenance.
4.3.3 After the analyzer qualification is successfully completed, a probationary local validation test is conducted on at least 1520
samples that were not used in developing the multivariate model. The purpose of this probationary validation is to verify that the
results produced by the analyzer (the PPTMRs) agree with the results from the primary test method (the PTMRs) to within
expectations based on the multivariate model. As the spectra of these initial validation samples are collected, they are analyzed with
the multivariate model to produce the PPTMRs. The absolute differences between PPTMR and PTMRs, |∆|,|δ|, are compared to
the 95 % uncertainties of the PPTMRs, U(PPTMR). If |δ| does not exceed U(PPTMR) for more than twothree samples,
probationary local validation continues until 1520 validation samples have been processed. If |δ| is less than U(PPTMR) for at least
1317 of the 1520 initial validation samples, then the predictions are consistent with the expectations based on the multivariate
model, and the system or subsystem performance is considered to be probationary validated for materials and property ranges
representative of those used in the validation. providing that the spectra used in generating the results are neither outliers or nearest
neighbor inliers. If |δ| is greater than U(PPTMR) for more than 23 of the initial validation samples, the analyzer system fails
probationary validation. An investigation of the cause of the failure should be conducted, and corrective action taken. The
validation process then restarts with initial performance testing.
NOTE 3—Probationary validation is done using the local validation procedure since 1520 samples is too few to conduct a general validation. 1520 samples
would seldom if ever span the property and composition range of a multivariate model based on more than 3 variables.
4.3.4 After probationary local validation is achieved, continued statistical quality control chart monitoring and analyses of |δ| are
carried out with new production samples to ensure ongoing prediction performance of the PPTMR meets the levels established
from the probationary validation. The |δ| are compared to U(PPTMR), and a count is maintained of the total number of non-outlier
validation samples, and the number for which |δ| is less than or equal to U(PPTMR). An inverse binomial distribution is used to
calculate the minimum number of samples for which |δ| must be less than U(PPTMR). As long as this minimum is met, the analyzer
system passes continual local validation. If the minimum is not met, the analyzer fails local validation. An investigation of the
cause of the failure should be conducted, and corrective action taken. The validation process then restarts with analyzer
qualification.
4.3.5 Once the total number of (PPTMR/PTMR/δ) data sets for samples from probationary and continual validation reaches 4
times the number of variables in the multivariate model (4k), a general validation can be conducted using the statistical
methodology of Practice D6708 providing that the available validation samples adequately span the full composition and property
range of the multivariate model. The samples used in this general validation should only include those whose spectra are not
outliers or nearest neighbor inliers relative to the multivariate model. The objective of the general validation is to demonstrate that
the PPTMRs agree with the PTMRs to within user-defined limits for bias and precision on at least 4k samples covering a wider
operating envelope.
4.4 During routine operation of the analyzer, validation tests are conducted on a regular, periodic basis to demonstrate that the
analyzer results remain in statistical agreement
...