ASTM E1655-17

Standard Practices for Infrared Multivariate Quantitative Analysis

SIGNIFICANCE AND USE
5.1 These practices can be used to establish the validity of the results obtained by an infrared (IR) spectrometer at the time the calibration is developed. The ongoing validation of estimates produced by analysis of unknown samples using the calibration model should be covered separately (see for example, Practice D6122).  
5.2 These practices are intended for all users of infrared spectroscopy. Near-infrared spectroscopy is widely used for quantitative analysis. Many of the general principles described in these practices relate to the common modern practices of near-infrared spectroscopic analysis. While sampling methods and instrumentation may differ, the general calibration methodologies are equally applicable to mid-infrared spectroscopy. New techniques are under study that may enhance those discussed within these practices. Users will find these practices to be applicable to basic aspects of the technique, to include sample selection and preparation, instrument operation, and data interpretation.  
5.3 The calibration procedures define the range over which measurements are valid and demonstrate whether or not the sensitivity and linearity of the analysis outputs are adequate for providing meaningful estimates of the specific physical or chemical characteristics of the types of materials for which the calibration is developed.
SCOPE
1.1 These practices cover a guide for the multivariate calibration of infrared spectrometers used in determining the physical or chemical characteristics of materials. These practices are applicable to analyses conducted in the near infrared (NIR) spectral region (roughly 780 to 2500 nm) through the mid infrared (MIR) spectral region (roughly 4000 to 400 cm−1).
Note 1: While the practices described herein deal specifically with mid- and near-infrared analysis, much of the mathematical and procedural detail contained herein is also applicable for multivariate quantitative analysis done using other forms of spectroscopy. The user is cautioned that typical and best practices for multivariate quantitative analysis using other forms of spectroscopy may differ from practices described herein for mid- and near-infrared spectroscopies.  
1.2 Procedures for collecting and treating data for developing IR calibrations are outlined. Definitions, terms, and calibration techniques are described. Criteria for validating the performance of the calibration model are described.  
1.3 The implementation of these practices require that the IR spectrometer has been installed in compliance with the manufacturer's specifications. In addition, it assumes that, at the times of calibration and of validation, the analyzer is operating at the conditions specified by the manufacturer.  
1.4 These practices cover techniques that are routinely applied in the near and mid infrared spectral regions for quantitative analysis. The practices outlined cover the general cases for coarse solids, fine ground solids, and liquids. All techniques covered require the use of a computer for data collection and analysis.  
1.5 These practices provide a questionnaire against which multivariate calibrations can be examined to determine if they conform to the requirements defined herein.  
1.6 For some multivariate spectroscopic analyses, interferences and matrix effects are sufficiently small that it is possible to calibrate using mixtures that contain substantially fewer chemical components than the samples that will ultimately be analyzed. While these surrogate methods generally make use of the multivariate mathematics described herein, they do not conform to procedures described herein, specifically with respect to the handling of outliers. Surrogate methods may indicate that they make use of the mathematics described herein, but they should not claim to follow the procedures described herein.  
1.7 The values stated in SI units are to be regarded as standard. No other units of measurement are...

General Information

• Status: Published
• Publication Date: 30-Nov-2017
• ICS: 01.040.17 - Metrology and measurement. Physical phenomena (Vocabularies)
• ICS: 71.040.50 - Physicochemical methods of analysis
• Technical Committee: E13 - Molecular Spectroscopy and Separation Science
• Drafting Committee: E13.11 - Multivariate Analysis

Relations

• Refers: ASTM D6300-24 - Standard Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products, Liquid Fuels, and Lubricants
• Effective Date: 01-Mar-2024
• Refers: ASTM D6299-23a - Standard Practice for Applying Statistical Quality Assurance and Control Charting Techniques to Evaluate Analytical Measurement System Performance
• Effective Date: 01-Dec-2023
• Refers: ASTM D1265-23a - Standard Practice for Sampling Liquefied Petroleum (LP) Gases, Manual Method
• Effective Date: 01-Dec-2023
• Refers: ASTM D6300-23a - Standard Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products, Liquid Fuels, and Lubricants
• Effective Date: 01-Dec-2023
• Refers: ASTM D6122-23 - Standard Practice for Validation of the Performance of Multivariate Online, At-Line, Field and Laboratory Infrared Spectrophotometer, and Raman Spectrometer Based Analyzer Systems
• Effective Date: 01-Jul-2023
• Refers: ASTM E456-13a(2022)e1 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 01-Apr-2022
• Refers: ASTM D6300-19a - Standard Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products and Lubricants
• Effective Date: 01-Dec-2019
• Refers: ASTM D6122-19b - Standard Practice for Validation of the Performance of Multivariate Online, At-Line, and Laboratory Infrared Spectrophotometer Based Analyzer Systems
• Effective Date: 01-Jun-2019
• Refers: ASTM D6122-19a - Standard Practice for Validation of the Performance of Multivariate Online, At-Line, and Laboratory Infrared Spectrophotometer Based Analyzer Systems
• Effective Date: 01-May-2019
• Refers: ASTM D6122-19 - Standard Practice for Validation of the Performance of Multivariate Online, At-Line, and Laboratory Infrared Spectrophotometer Based Analyzer Systems
• Effective Date: 01-Jan-2019
• Refers: ASTM D6122-18 - Standard Practice for Validation of the Performance of Multivariate Online, At-Line, and Laboratory Infrared Spectrophotometer Based Analyzer Systems
• Effective Date: 01-Jul-2018
• Refers: ASTM D6299-17b - Standard Practice for Applying Statistical Quality Assurance and Control Charting Techniques to Evaluate Analytical Measurement System Performance
• Effective Date: 15-Dec-2017
• Refers: ASTM D6299-17a - Standard Practice for Applying Statistical Quality Assurance and Control Charting Techniques to Evaluate Analytical Measurement System Performance
• Effective Date: 15-Nov-2017
• Refers: ASTM E456-13A(2017)e1 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 01-Oct-2017
• Refers: ASTM E456-13A(2017)e3 - Standard Terminology Relating to Quality and Statistics
• Effective Date: 01-Oct-2017

Overview

ASTM E1655-17: Standard Practices for Infrared Multivariate Quantitative Analysis provides comprehensive guidelines for the calibration and use of infrared (IR) spectrometers to determine physical or chemical characteristics of materials. Developed by ASTM International, this standard covers practices applicable in the near-infrared (NIR) and mid-infrared (MIR) spectral regions, addressing both fundamental and advanced aspects of IR quantitative analysis. ASTM E1655-17 is essential for laboratories and industry professionals seeking reliable, validated results for quantitative spectroscopic measurements.

Key Topics

• Multivariate Calibration Procedures
  The standard outlines steps for developing calibration models that relate spectral data to known reference values. It includes:

  • Sample selection and preparation
  • Data collection protocols for calibration, validation, and unknown samples
  • Mathematical techniques like multilinear regression, principal components regression, and partial least squares

• Model Validation
  Validation ensures the calibration model accurately predicts concentrations or properties for new samples, including:

  • Use of separate validation samples
  • Statistical comparison of predicted and reference values
  • Detection of outliers and extrapolation beyond the model’s calibration range

• Instrument Qualification and Routine Monitoring
  Recommendations are made for:

  • Ensuring instruments are installed and operated according to manufacturer specifications
  • Regular performance monitoring with appropriate quality control procedures

• Data Handling and Reference Methods
  Emphasis is placed on:

  • Consistent and precise data acquisition
  • Use of validated reference methods for assigning sample values
  • Ensuring sample composition remains stable during analysis

• Applicability Across Diverse Matrices
  The practices are relevant for:

  • Coarse solids, fine ground solids, and liquids
  • Adoption in both laboratory and process environments

Applications

• Quality Control in Manufacturing
  Employed in industries such as pharmaceuticals, chemicals, food, and petrochemicals for real-time monitoring of product composition and properties.

• Process Analytical Technology (PAT)
  Enables proactive process adjustments by providing rapid quantitative data on raw materials and intermediates.

• Material Identification and Component Analysis
  Applied in research and development for accurate identification and quantification of components in complex mixtures.

• Validation of Analytical Methods
  Offers guidelines for developing robust, validated IR methods suitable for regulatory or internal qualification.

• Calibration Transferability
  Supports the transfer of calibration models between different instruments, ensuring consistent results across multiple sites or devices.

Related Standards

• ASTM D6122: Validation of the Performance of Multivariate Infrared Analyzer Systems
  Specifies procedures for ongoing validation of calibration models and routine performance checks.

• ASTM E168: General Techniques of Infrared Quantitative Analysis
  Offers foundational principles complementing ASTM E1655-17.

• ASTM E932 & E1421: Performance of Dispersive and FT-IR Spectrometers
  Focuses on instrument-specific performance verification.

• ASTM D1265, D4057 & D4177: Sampling Practices for Liquids and Gases
  Provide guidance on representative sample collection crucial for accurate IR analysis.

• ASTM D6299 & D6300: Statistical Quality Assurance and Bias Determination
  Ensure analytical method precision and accuracy meet industry standards.

Conclusion

Adhering to ASTM E1655-17 ensures reliable, accurate, and validated quantitative analysis using infrared spectroscopy. The standard delivers value by harmonizing calibration, validation, and performance verification practices, making it indispensable for laboratories and industries that depend on precise spectroscopic data for decision-making, compliance, and quality assurance. Integrating ASTM E1655-17 with related ASTM and international standards empowers organizations to maintain high-quality analytical results and operational excellence.

Standards Content (Sample)

This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: E1655 − 17
Standard Practices for
Infrared Multivariate Quantitative Analysis
This standard is issued under the fixed designation E1655; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope analyzed. While these surrogate methods generally make use
of the multivariate mathematics described herein, they do not
1.1 These practices cover a guide for the multivariate
conform to procedures described herein, specifically with
calibration of infrared spectrometers used in determining the
respect to the handling of outliers. Surrogate methods may
physical or chemical characteristics of materials. These prac-
indicate that they make use of the mathematics described
tices are applicable to analyses conducted in the near infrared
herein, but they should not claim to follow the procedures
(NIR) spectral region (roughly 780 to 2500 nm) through the
described herein.
mid infrared (MIR) spectral region (roughly 4000 to 400
−1
cm ). 1.7 The values stated in SI units are to be regarded as
NOTE 1—While the practices described herein deal specifically with
standard. No other units of measurement are included in this
mid-andnear-infraredanalysis,muchofthemathematicalandprocedural
standard.
detail contained herein is also applicable for multivariate quantitative
1.8 This standard does not purport to address all of the
analysisdoneusingotherformsofspectroscopy.Theuseriscautionedthat
typicalandbestpracticesformultivariatequantitativeanalysisusingother safety concerns, if any, associated with its use. It is the
formsofspectroscopymaydifferfrompracticesdescribedhereinformid-
responsibility of the user of this standard to establish appro-
and near-infrared spectroscopies.
priate safety, health, and environmental practices and deter-
1.2 Procedures for collecting and treating data for develop-
mine the applicability of regulatory limitations prior to use.
ing IR calibrations are outlined. Definitions, terms, and cali-
1.9 This international standard was developed in accor-
bration techniques are described. Criteria for validating the
dance with internationally recognized principles on standard-
performance of the calibration model are described.
ization established in the Decision on Principles for the
Development of International Standards, Guides and Recom-
1.3 The implementation of these practices require that the
mendations issued by the World Trade Organization Technical
IR spectrometer has been installed in compliance with the
Barriers to Trade (TBT) Committee.
manufacturer’s specifications. In addition, it assumes that, at
the times of calibration and of validation, the analyzer is
2. Referenced Documents
operating at the conditions specified by the manufacturer.
2.1 ASTM Standards:
1.4 These practices cover techniques that are routinely
D1265Practice for Sampling Liquefied Petroleum (LP)
applied in the near and mid infrared spectral regions for
Gases, Manual Method
quantitative analysis. The practices outlined cover the general
D4057Practice for Manual Sampling of Petroleum and
cases for coarse solids, fine ground solids, and liquids. All
Petroleum Products
techniques covered require the use of a computer for data
D4177Practice for Automatic Sampling of Petroleum and
collection and analysis.
Petroleum Products
1.5 These practices provide a questionnaire against which
D4855Practice for Comparing Test Methods (Withdrawn
multivariate calibrations can be examined to determine if they
2008)
conform to the requirements defined herein.
D6122Practice for Validation of the Performance of Multi-
variate Online,At-Line, and Laboratory Infrared Spectro-
1.6 For some multivariate spectroscopic analyses, interfer-
photometer Based Analyzer Systems
encesandmatrixeffectsaresufficientlysmallthatitispossible
D6299Practice for Applying Statistical Quality Assurance
to calibrate using mixtures that contain substantially fewer
and Control Charting Techniques to Evaluate Analytical
chemical components than the samples that will ultimately be
1 2
These practices are under the jurisdiction of ASTM Committee E13 on For referenced ASTM standards, visit the ASTM website, www.astm.org, or
Molecular Spectroscopy and Separation Science and are the direct responsibility of contact ASTM Customer Service at [e-mail]. For Annual Book of ASTM
Subcommittee E13.11 on Multivariate Analysis. Standards volume information, refer to the standard’s Document Summary page on
Current edition approved Dec. 1, 2017. Published January 2018. Originally the ASTM website.
approved in 1997. Last previous edition approved in 2012 as E1655–05(2012). The last approved version of this historical standard is referenced on
DOI: 10.1520/E1655-17. www.astm.org.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E1655 − 17
Measurement System Performance testtheagreementbetweenestimatesmadewiththemodeland
D6300Practice for Determination of Precision and Bias the reference method.
Data for Use in Test Methods for Petroleum Products and
3.2.7 multivariate calibration, n—a process for creating a
Lubricants
model that relates component concentrations or properties to
E131Terminology Relating to Molecular Spectroscopy
the absorbances of a set of known reference samples at more
E168Practices for General Techniques of Infrared Quanti-
than one wavelength or frequency.
tative Analysis
3.2.8 reference method, n—the analytical method that is
E275PracticeforDescribingandMeasuringPerformanceof
used to estimate the reference component concentration or
Ultraviolet and Visible Spectrophotometers
property value which is used in the calibration and validation
E334Practice for General Techniques of Infrared Micro-
procedures.
analysis
E456Terminology Relating to Quality and Statistics
3.2.9 reference values, n—the component concentrations or
E691Practice for Conducting an Interlaboratory Study to
property values for the calibration or validation samples which
Determine the Precision of a Test Method
are measured by the reference analytical method.
E932PracticeforDescribingandMeasuringPerformanceof
3.2.10 spectrometer/spectrophotometer qualification,
Dispersive Infrared Spectrometers
n—the procedures by which a user demonstrates that the
E1421Practice for Describing and Measuring Performance
performance of a specific spectrometer/spectrophotometer is
of Fourier Transform Mid-Infrared (FT-MIR) Spectrom-
adequate to conduct a multivariate analysis so as to obtain
eters: Level Zero and Level One Tests
precision consistent with that specified in the method.
E1866Guide for Establishing Spectrophotometer Perfor-
3.2.11 surrogate calibration, n—a multivariate calibration
mance Tests
that is developed using a calibration set which consists of
E1944Practice for Describing and Measuring Performance
mixtures which contain substantially fewer chemical compo-
of Laboratory Fourier Transform Near-Infrared (FT-NIR)
nents than the samples which will ultimately be analyzed.
Spectrometers: Level Zero and Level One Tests
3.2.12 surrogate method, n—a standard test method that is
3. Terminology
based on a surrogate calibration.
3.1 Definitions—Forterminologyrelatedtomolecularspec-
3.2.13 validation samples—asetofsamplesusedinvalidat-
troscopic methods, refer to Terminology E131. For terminol-
ing the model. Validation samples are not part of the set of
ogy relating to quality and statistics, refer to Terminology
calibration samples. Reference component concentration or
E456.
property values are known (measured by reference method),
3.2 Definitions of Terms Specific to This Standard:
and are compared to those estimated using the model.
3.2.1 analysis,n—inthecontextofthispractice,theprocess
of applying the calibration model to a spectrum, preprocessed
4. Summary of Practices
as required, so as to estimate a component concentration value
4.1 Multivariate mathematics is applied to correlate the
or property.
spectra measured for a set of calibration samples to reference
3.2.2 calibration, n—a process used to create a model
component concentrations or property values for the set of
relating two types of measured data. In the context of this
samples. The resultant multivariate calibration model is ap-
practice, a process for creating a model that relates component
plied to the analysis of spectra of unknown samples to provide
concentrations or properties to spectra for a set of known
an estimate of the component concentration or property values
reference samples.
for the unknown sample.
3.2.3 calibration model, n—the mathematical expression or
4.2 Multilinear regression (MLR), principal components
the set of mathematical operations that relates component
regression(PCR),andpartialleastsquares(PLS)areexamples
concentrations or properties to spectra for a set of reference
of multivariate mathematical techniques that are commonly
samples.
used for the development of the calibration model. Other
mathematical techniques are also used, but may not detect
3.2.4 calibration samples, n—the set of reference samples
outliers, and may not be validated by the procedure described
used for creating a calibration model. Reference component
in these practices.
concentration or property values are known (measured by
reference method) for the calibration samples and a calibration
4.3 Statistical tests are applied to detect outliers during the
modelisfoundwhichrelatesthesevaluestothespectraduring
development of the calibration model. Outliers include high
the calibration.
leverage samples (samples whose spectra contribute a statisti-
cally significant fraction of one or more of the spectral
3.2.5 estimate, n—the value for a component concentration
variables used in the model), and samples whose reference
or property obtained by applying the calibration model for the
values are inconsistent with the model.
analysis of an absorption spectrum.
3.2.6 model validation, n—the process of testing a calibra- 4.4 Validation of the calibration model is performed by
tion model with validation samples to determine bias between using the model to analyze a set of validation samples and
the estimates from the model and the reference method, and to statisticallycomparingtheestimatesforthevalidationsamples
E1655 − 17
toreferencevaluesmeasuredforthesesamples,soastotestfor sample presentation technique among calibration, validation,
bias in the model and for agreement of the model with the andpredictionsampleswillintroducevariationanderrorwhich
reference method. has not been modeled within the calibration. Infrared instru-
mentation is discussed in Section 7 and infrared spectral
4.5 Statistical tests are applied to detect when values esti-
measurements in Section 8.
mated using the model represent extrapolation of the calibra-
6.1.4 Calculating the Mathematical Model—The calcula-
tion.
tion of mathematical (calibration) models may involve a
4.6 Statistical expressions for calculating the repeatability
varietyofdatatreatmentsandcalibrationalgorithms.Themore
of the infrared analysis and the expected agreement between
common linear techniques are discussed in Section 12.A
the infrared analysis and the reference method are given.
variety of statistical techniques are used to evaluate and
optimize the model. These techniques are described in Section
5. Significance and Use
15. Statistics used to detect outliers in the calibration set are
5.1 These practices can be used to establish the validity of
covered in Section 16.
theresultsobtainedbyaninfrared(IR)spectrometeratthetime
6.1.5 ValidationoftheCalibrationModel—Validationofthe
the calibration is developed. The ongoing validation of esti-
efficacy of a specific calibration model (equation) requires that
mates produced by analysis of unknown samples using the
the model be applied for the analysis of a separate set of test
calibration model should be covered separately (see for
(validation)samples,andthatthevaluespredictedforthesetest
example, Practice D6122).
samples be statistically compared to values obtained by the
reference method. The statistical tests to be applied for
5.2 These practices are intended for all users of infrared
validation of the model are discussed in Section 18.
spectroscopy. Near-infrared spectroscopy is widely used for
6.1.6 Application of the Model for the Analysis of
quantitative analysis. Many of the general principles described
in these practices relate to the common modern practices of Unknowns—The mathematical model is applied to the spectra
of unknown samples to estimate component concentrations or
near-infrared spectroscopic analysis. While sampling methods
and instrumentation may differ, the general calibration meth- property values, or both, (see Section 13). Outlier statistics are
used to detect when the analysis involves extrapolation of the
odologies are equally applicable to mid-infrared spectroscopy.
New techniques are under study that may enhance those model (see Section 16).
discussedwithinthesepractices.Userswillfindthesepractices 6.1.7 Routine Analysis and Monitoring—Once the efficacy
to be applicable to basic aspects of the technique, to include
of one or more calibration equations is established, the equa-
sample selection and preparation, instrument operation, and tions must be monitored for continued accuracy and precision.
data interpretation.
Simultaneously, the instrument performance must be moni-
tored so as to trace any deterioration in performance to either
5.3 The calibration procedures define the range over which
thecalibrationmodelitselfortoafailureintheinstrumentation
measurements are valid and demonstrate whether or not the
performance. Procedures for verifying the performance of the
sensitivityandlinearityoftheanalysisoutputsareadequatefor
analysis are only outlined in Section 22. For petrochemicals,
providing meaningful estimates of the specific physical or
these procedures are covered in detail in Practice D6122. The
chemical characteristics of the types of materials for which the
useofPracticeD6122requiresthataqualitycontrolprocedure
calibration is developed.
be established at the time the model is developed. The QC
check sample is discussed in Section 22. For practices to
6. Overview of Multivariate Calibration
compare reference methods and analyzer methods, refer to
6.1 The practice of infrared multivariate quantitative analy-
Practice D4855.
sis involves the following steps:
6.1.8 Transfer of Calibrations—Transferable calibrations
6.1.1 Selecting the Calibration Set—This set is also termed
are equations that can be transferred from the original
thetrainingsetorspectrallibraryset.Thissetistorepresentall
instrument, where calibration data were collected, to other
of the chemical and physical variation normally encountered
instruments where the calibrations are to be used to predict
forroutineanalysisforthedesiredapplication.Selectionofthe
samples for routine analysis. In order for a calibration to be
calibration set is discussed in Section 17, after the statistical
transferable it must perform prediction after transfer without a
terms necessary to define the selection criteria have been
significantdecreaseinperformance,asindicatedbyestablished
defined.
statistical tests. In addition, statistical tests that are used to
6.1.2 Determination of Concentrations or Properties, or
detect extrapolation of the model must be preserved during the
Both, for Calibration Samples—The chemical or physical
transfer. Bias or slope adjustments, or both, are to be made
properties, or both, of samples in the calibration set must be
after transfer only when statistically warranted. Calibration
accurately and precisely measured by the reference method in
transfer, that is sometimes referred to as instrument
order to accurately calibrate the infrared model for prediction
standardization, is discussed in Section 21.
of the unknown samples. Reference measurements are dis-
cussed in Section 9.
7. Infrared Instrumentation
6.1.3 The Collection of Infrared Spectra—The collection of
optical data must be performed with care so as to present 7.1 A complete description of all applicable types of infra-
calibration samples, validation samples, and prediction (un- red instrumentation is beyond the scope of these practices.
known) samples for analysis in an alike manner. Variation in Only a general outline is given here.
E1655 − 17
7.2 The IR instrumentation is comprised of two categories, canoperateinthemid-IRandnear-IRspectralregions.TheFT
including instruments that acquire continuous spectral data instruments use a single detector.
7.3.4 A second type of transformation spectrophotometer
over wavelength or frequency ranges (spectrophotometers),
and those that only examine one or several discrete wave- uses the Hadamard transformation. Light is initially dispersed
with a grating. Light then passes through a mask mounted on
lengths or frequencies (photometers).
or adjacent to a single detector.The mask generates a series of
7.2.1 Photometers may have one or a series of wavelength
patterns. For example, these patterns may be formed by
filters and a single detector. These filters are mounted on a
electronically opening and shutting various locations, such as
turretwheelsothattheindividualwavelengthsarepresentedto
in a liquid crystal display, or by moving an aperture or slit
a single detector sequentially. Continuously variable filters
through the beam. These modulations alter the energy distri-
may also be used in this fashion. These filters, either linear or
bution incident upon the detector.Amathematical transforma-
circular, are moved past a slit to scan the wavelength being
tionisthenusedtoconvertthesignalintospectralinformation.
measured.Alternatively, photometers may have several mono-
chromatic light sources, such as light-emitting diodes, that 7.4 Infrared instruments used in multivariate calibrations
should be installed and operated in accordance with the
sequentially turn on and off.
instructions of the instrument manufacturer.Where applicable,
7.3 Spectrophotometers can be classified, based upon the
the performance of the instrument should be tested at the time
procedure by which light is separated into component wave-
the calibration is conducted using procedures defined in the
lengths. Dispersive instruments generally use a diffraction
appropriate ASTM practice (see 2.1). The performance of the
grating to spatially disperse light into a continuum of wave-
instrument should be monitored on a periodic basis using the
lengths. In scanning-grating systems, the grating is rotated so
same procedures. The monitoring procedure should detect
that only a narrow band of wavelengths is transmitted to a
changes in the performance of the instrument (relative to that
single detector at any given time. Dispersion can occur before
seen during collection of the calibration spectra) that would
thesample(pre-dispersed)orafterthesample(post-dispersed).
affect the estimation made with the calibration model.
7.3.1 Spectrophotometers are also available where the
7.5 For most infrared quantitative applications involving
wavelength selection is accomplished without moving parts,
complex matrices, it is a general consensus that scanning-type
usingaphotodiodearraydetector.Post-dispersionisutilized.A
instruments (either dispersive or interferometer based) provide
grating can again provide this function, although other
the greatest performance, due to the stability and reproducibil-
methods, such as a linear variable filter (LVF) accomplish the
ity of modern instrumentation and to the greater amount of
same purpose (a LVF is a multilayer filter that has variable
spectral data provided for computer interpretation. These data
thickness along its length, such that different wavelengths are
allow for greater calibration flexibility and additional options
transmitted at different positions).The photodiode array detec-
for selections of spectral areas less sensitive to band shifts and
tor is used to acquire a continuous spectrum over wavelength
extraneous noise within the spectral signal. Scanning/
without mechanical motion. The array detector is a compact
interferometer-based systems also allow greater wavelength/
aggregate of up to several thousand individual photodiode
frequency precision between instruments due to internal
detectors. Each photodiode is located in a different spectral
wavelength/frequencystandardizationtechniques,andthepos-
region of the dispersed light beam and detects a unique range
sibilities of computer-generated spectral corrections. For
of wavelengths.
example, scanning instruments have received approval for
7.3.2 The acousto-optical tunable filter is a continuous
complex matrices, such as animal feed and forages (1, 2).
variant of the fixed filter photometer with no moving optical
7.6 Descriptions of instrumentation designs related to Refs
parts for wavelength selection. A birefrigent crystal (for
(1)and (2)arefoundinRefs (3)and (4).Otherinstrumentation
example, tellurium oxide) is used, in which acoustic waves at
similar in performance to that described in these references is
a selected frequency are applied to select the wavelength band
acceptable for all near-infrared techniques described in these
of light transmitted through the crystal. Variations in the
practices.
acoustic frequency cause the crystal lattice spacing to change,
7.7 For information describing the measurement of perfor-
that in turn, causes the crystal to act as a variable transmission
mance of ultraviolet, visible, and near infrared
diffraction grating for one wavelength (that is, a Bragg diffrac-
spectrophotometers, refer to Practice E275. For information
tor). A single detector is used to analyze the signal.
describing the measurement of performance of dispersive
7.3.3 An additional category of spectrophotometers uses
infrared spectrophotometers, refer to Practice E932. For infor-
mathematical transformations to convert modulated light sig-
mation describing the measurement performance of Fourier
nals into spectral data. The most well-known example is the
Transform mid-infrared spectrophotometers, refer to Practice
Fourier transform, that when applied to infrared (IR) is known
E1421. For information describing the measurement perfor-
asFT-IR.Lightisdividedintotwobeamswhoserelativepaths
mance of FourierTransform near-infrared spectrophotometers,
are varied by use of a moving optical element (for example,
refertoPracticeE1944.Forspectrophotometerstowhichthese
eitheramovingmirror,oramovingwedgeofahighrefractive
practice do not apply, refer to Guide E1866.
index material). The beams are recombined to produce an
interference pattern that contains all of the wavelengths of
interest. The interference pattern is mathematically converted 4
The boldface numbers in parentheses refer to a list of references at the end of
into spectral data using the Fourier transform. The FT method this standard.
E1655 − 17
8. Infrared Spectral Measurements 8.2 Traditionally, a sample is manually brought to the
instrument and placed in a cell or cuvette with windows that
8.1 Multivariate calibrations are based on Beer’s Law,
transmit in the region of interest. Alternatively, transfer pipes
namely, the absorbance of a homogeneous sample containing
canbeusedtoallowliquidtoflowthroughanopticalcellinthe
an absorbing substance is linearly proportional to the concen-
instrument for continuous analysis. With optical fibers, the
tration of the absorbing species.The absorbance of a sample is
sample can be analyzed remotely from the instrument. Radia-
defined as the logarithm to the base ten of the reciprocal of the
tion is sent to the sample through an optical fiber or bundle of
transmittance, (T).
fibers and returned to the instrument by means of another fiber
A 5log ~1/T!
or bundle of fibers. Instruments have been developed that use
single fibers to transmit and receive the radiation, as well as
The transmittance, T, is defined as the ratio of radiant
those using bundles of fibers for this purpose. Detectors and
power transmitted by the sample to the radiant power inci-
radiationsourcesexternaltotheinstrumentcanalsobeused,in
dent on the sample.
which case only one fiber or bundle is needed. For spectral
8.1.1 For measurements conducted by reflectance, the
regions where transmitting fibers do not exist, the same
reflectance,R,issometimessubstitutedforthetransmittanceT.
function can be performed over limited distances using appro-
The reflectance is defined as the ratio of the radiant power
priate transfer optics.
reflected by the sample to the radiant power incident on the
NOTE5—Iftheinstrumentusespredispersionofthelight,somecaution
sample.
mustbeexercisedtoavoidintroducingambientlightintothesystematthe
NOTE 2—The relationship A=log (1/R) is not a definition, but rather sampleposition,sincesuchlightmaybedetected,givingrisetoerroneous
an approximation designed to linearize the relationship between the absorbance measurements.
measured reflectance, R, and the concentration of the absorbing species.
8.3 Although most multivariate calibrations for liquids in-
For some applications, other linearization functions (for example,
volve the direct measurement of transmitted light, alternative
Kubelka-Munk) may be more appropriate (5).
sampling technologies (for example, attenuated total reflec-
8.1.2 For most types of instrumentation, the radiant power
tance) can also be employed.Transmittance measurements can
incident on the sample cannot be measured directly. Instead, a
be employed for some types of solids (for example, polymer
reference (background) measurement of the radiant power is
films),whereasothersolids(forexample,powderedsolids)are
made without the sample being present in the light beam.
more commonly measured by diffuse reflectance techniques.
NOTE 3—To avoid confusion, the reference measurement of the radiant
8.4 For most infrared instrumentation, a variety of adjust-
power will be referred to as a background measurement, and the word
able parameters are available to control the collection and
reference will only be used to refer to measurements made by the
computation of the spectral data.These parameters control, for
reference method against which the infrared is to be calibrated. (See
Section 9.)
instance, the optical and digital resolution, and the rate of data
acquisition (scan speed).Adetailed description of the spectral
8.1.3 A measurement is then conducted with the sample
acquisition parameters and their effect on multivariate calibra-
present, and the ratio, T, is calculated. The background
tions is beyond the scope of these practices. However, it is
measurementmaybeconductedinavarietyofwaysdepending
essential that all adjustable parameters that control the collec-
on the application and the instrumentation. The sample and its
tion and computation of spectral data be maintained constant
holder may be physically removed from the light beam and a
for the collection of spectra of calibration samples, validation
background measurement made on the “empty beam”. The
samples, and unknown samples for which estimates are to be
sample holder (cell) may be emptied, and a background
made.
measurement may be taken through the “empty cell.”
8.5 For definitions and further description of general infra-
NOTE4—Foropticallythincells,caremaybenecessarytoavoidoptical
interferences resulting from multiple internal reflections within the cell. red quantitative measurement techniques, refer to Practices
Forverythickcells,differencesintherefractiveindexbetweenthesample
E168. For a description of general techniques of infrared
and the empty cell may change properties of the optical system, for
microanalysis, refer to Practice E334.
example, shift focal points.
8.1.4 The sample holder (cell) may be filled with a liquid
9. Reference Method and Reference Values
that has minimal absorption in the spectral range of interest,
9.1 Infrared spectroscopy requires calibration to determine
and the background measurement may be taken through the
the proportionality relationship between the signals measured
“backgroundliquid.”Alternatively,thelightbeammaybesplit
and the component concentrations or properties that are to be
or alternately passed through the sample and through an
estimated. During the calibration, spectra are measured for
“empty beam,” an “empty cell,” or a “background liquid.” For
samples for which these reference values are known, and the
reflectance measurements, the reflectance of a material having
relationshipbetweenthesampleabsorbancesandthereference
minimal absorbance in the region of interest is generally used
values is determined. The proportionality relationship is then
as the background measurement.
applied to the spectra of unknown samples to estimate the
8.1.5 The particular background referencing scheme that is
concentration or property values for the sample.
used may vary among instruments, and among applications.
The same background referencing scheme must be employed 9.2 For simple mixtures containing only a few chemical
for the measurement of all spectra of calibration samples, components, it is generally possible to prepare mixtures that
validation samples, and unknown samples to be analyzed. can serve as standards for the multivariate calibration of an
E1655 − 17
infraredanalysis.Becauseofpotentialinterferencesamongthe cluded in the method. In this case, it is only necessary to
absorbances of the components, it is not sufficient to vary the demonstrate that the reference measurement is being practiced
concentration of only some of the mixture components, even in accordance with the procedure described in the method, and
when analyses for only one component are being developed. thattherepeatabilityobtainedisstatisticallycomparabletothat
Instead, all components should be varied over a range repre- published in the method. Data from established quality control
sentative of that expected for future unknown samples that are procedures can be used to demonstrate that the repeatability of
to be analyzed. Since infrared measurements are conducted on the reference method is within ASTM specifications. If such
afixedvolumeofsample(forexample,afixedcellpathlength), dataisnotavailable,thenrepeatabilitydatashouldbecollected
itispreferablethatconcentrationreferencevaluesbeexpressed on at least three of the samples that are to be used in the
involumetricterms,forexample,involumepercentage,grams calibration. These samples should be chosen to span the range
per millilitre, moles per cubic centimetre, and so forth. Devel- of values over which the calibration is to be developed, one
oping multivariate calibrations for reference concentrations sample having a reference value in the bottom third of the
expressed in other terms (for example, weight percentage) can range, one sample having a value in the middle third of the
lead to models that are linear approximations to what is really range, and one sample having a value in the upper third of the
a nonlinear relationship and can lead to less accurate estimates range.At least six reference measurements should be made on
of the concentrations. each sample. The standard deviation among the measurements
should be calculated and compared to that expected based on
9.3 For complex mixtures, such as those obtained from
the published repeatability.
petrochemical processes, preparation of reference standards is
9.5 If the reference method to be used for the multivariate
generally impractical, and the multivariate calibration of an
calibrationisanestablishedASTMmethod,andthesamplesto
infraredanalysismusttypicallybeperformedonactualprocess
be used in the calibration have been analyzed by a cooperative
samples. In this case, the reference values used to calibrate the
testing program (for example, octane values obtained from
infraredanalysisareobtainedbyareferenceanalyticalmethod.
recognized exchange groups), then the reference values ob-
The accuracy of a component concentration or property value
tained by the cooperative testing program can be used directly,
estimated by a multivariate infrared analysis is highly depen-
and the standard deviations established by the cooperative
dentontheaccuracyandprecisionofthereferencevaluesused
testing program can be used as the estimate of the precision of
in the calibration. The expected agreement between the infra-
the reference data.
redestimatedvaluesandthoseobtainedfromasinglereference
measurement can never exceed the repeatability of the refer-
9.6 Reference methods that are not ASTM methods can be
ence method, since, even if the infrared estimated the true
usedforthemultivariatecalibrationofinfraredanalyses,butin
value, the measurement of agreement is limited by the preci-
this case, it is the responsibility of the method developer to
sion of the reference values. Knowledge of the precision
establish the precision of the reference method using proce-
(repeatability) of the reference method is critical in the
dures similar to those detailed in Practice E691,inthe Manual
development of an infrared multivariate calibration. The pre-
for Determining Precision for ASTM Methods on Petroleum
cisionofthereferencedatausedindevelopingamodel,andthe
Products and Lubricants and in Practice D6300.
accuracy of the model can be improved by averaging repeated
9.7 When multiple reference measurements are made on an
reference measurements.
individualcalibrationorvalidationsample,aDixon’sTest(see
NOTE6—Ifthereferencevaluesusedtocalibrateamultivariateinfrared A1.1) should be applied to the values to determine if all of the
analysis are generated in a single laboratory, it is essential that the
reference values came from the same population, or if one or
measurement process used to generate these values be monitored for bias
more of the values is suspect and should be rejected.
and precision using suitable quality assurance procedures (see for
example, Practice D6299. If primary standards are not available to allow
10. Simple Procedure to Develop a Feasibility
the bias of the reference measurement process to be established, it is
recommended that the laboratory participate in an interlaboratory cross- Calibration
check program as a means of demonstrating accuracy.
10.1 For new applications, it is generally not known
NOTE 7—Samples like hydrocarbons from petrochemical process
whether an adequate IR multivariate model can be developed.
streamscandegradewithtimeunlesscarefulsamplingandsamplestorage
In this case, feasibility studies can be performed to determine
procedures are followed. It is critical that the composition of samples
taken for laboratory or at-line infrared analysis, or for laboratory mea-
if there is a relationship between the IR spectra and the
surement of the reference data be representative of the process at the time
component/property of interest, and whether a model of
the samples are taken, and that composition is maintained during storage
adequate precision could possibly be built. If the feasibility
and transport of the samples either to the analyzer or to the laboratory.
calibrationissuccessful,thenitcanbeexpandedandvalidated.
Sampling should be done in accordance with methods like Practices
A feasibility calibration involves the following steps:
D1265 and D4057, or Practice D4177, whichever are applicable. When-
ever possible, sample storage for extended time periods is not recom-
10.1.1 Approximately 30 to 50 samples are collected cov-
mended because of the likelihood of samples degrading with time in spite
ering the entire range for the constituent/property of interest.
ofsamplingprecautionstaken.Degradationofsamplescancausechanges
Care should be exercised to avoid intercorrelations among
in the spectra measured by the analyzer and thus in the values estimated,
and in the property or quality measured by the reference method.
9.4 If the reference method used to obtain reference values
Manual on Determining Precision Data for ASTM Methods on Petroleum
for the multivariate calibration is an established ASTM
Products and Lubricants, which has been filed atASTM International Headquarters
method, then repeatability and reproducibility data are in- and may be obtained by requesting Research Report RR:D02-1007.
E1655 − 17
majorconstituentsunlesssuchintercorrelationsalwaysexistin for the calibration samples. The model is then built on the
the materials being analyzed. The range in the concentration/ mean-centereddata.Ifthespectralandreferencevaluedataare
propertyshouldbepreferablyfivetimes,butnotlessthanthree mean-centered prior to the development of the model, then:
times, the standard deviation of the reproducibility 11.2.1 When an unknown sample is analyzed, the average
(reproducibility/2.77) of the reference analysis.
spectrum for the calibration site must be subtracted from the
10.1.2 When collecting spectral data on these samples, spectrum of the unknown prior to applying the mean-centered
variations in particle size, sample presentation, and process
model, and the average reference value for the calibration set
conditions which are expected during analysis must be repro- mustbeaddedtotheestimatefromthemean-centeredmodelto
duced. Multiple spectra of the same sample under different
obtain the final estimate; and
conditionscanbeemployedifsuchvariationsinconditionsare 11.2.2 The degrees of freedom used in calculating the
anticipated during analysis.
standard error of calibration must be diminished by one to
10.1.3 Reference analyses on these samples are conducted account for the degree of freedom used in calculating the
using the accepted reference method. If the range for the
average (see 15.2).
component/property is not at least five times the standard
deviation of the reproducibility for the reference analysis, then
12. Multivariate Calibration Mathematics
r replicate analyses should be conducted on each sample such
12.1 Multivariatemathematicaltechniquesareusedtorelate
thatthe =r timestherangeispreferablyfivetimes,butatleast
the spectra measured for a set of calibration samples to the
three times, the standard deviation of the reference analysis.
reference values (property or component concentration values)
10.1.4 A calibration model is developed using one or more
obtained for this set of samples from a reference test. The
of the mathematical techniques described in Sections 11 and object is to establish a multivariate calibration model that can
12. The calibration model is preferably tested using cross- be applied to the spectra of future, unknown, samples to
validation methods such as SECV or PRESS (see 15.3.6). estimate values (property or component concentration values).
Other statistics can also be used to judge the overall quality of Only linear multivariate techniques are described in these
the calibration. practices; that is, it is assumed that the property or component
10.1.5 IftheSECVvalueobtainedfromthecrossvalidation concentrationvaluescanbemodeledasalinearfunctionofthe
suggests that a model of adequate precision can be built, then samplespectra.Variousnonlinearmultivariatetechniqueshave
additional samples are collected to round out the calibration been developed, but have generally not been as widely used as
set,andtoserveasavalidationset,spectraofthesesamplesare the following linear techniques. These practices are not in-
collected, a final model is developed, and validated as de- tended to compare or contrast among these techniques. For the
scribed in Sections 13, 14, and 15. purpose of these practices, the suitability of any specific
mathematical technique should be judged only on the follow-
11. Data Preprocessing
ing two criteria:
11.1 Various types of data preprocessing algorithms can be 12.1.1 The technique should be capable of producing a
applied to the spectral data prior to the development of a calibration model that can be validated as described in Section
multivariatecalibrationmodel.Forexample,numericalderiva-
18; and
tives of the spectra may be calculated using digital filtering
12.1.2 The technique should be capable of providing statis-
algorithms to remove varying baselines. Such filtering gener-
tics suitable for identifying if samples being analyzed are
ally causes a significant decrease in the spectral signal-to-
outside the range for which the model was developed; that is,
noise. Digital filters may also be employed to smooth data,
whentheestimatedvaluesrepresentextrapolationofthemodel
improving signal to noise at the expense of resolution. A
(see 16.3).
complete description of all possible preprocessing methods is
NOTE 8—In the following derivations, matrices are indicated using
beyond the scope of these practices. For the purpose of these
boldface capital letters, vectors are indicated using boldface lowercase
practices, preprocessing of the spectral data can be used if it
letters, and scalars are indicated using lowercase letters. Vectors are
produces a model which has acceptable precision and which column vectors, and their transposes are row vectors. Italicized lowercase
letters indicate matrix or vector dimensions.
passes the validation test described in Section 21. In addition,
any spectral preprocessing method must be automated so as to
12.1.3 All linear, multivariate techniques are designed to
provide an exactly reproducible result, and must be applied
solve the same generic problem. If n calibration spectra are
consistentlytoallcalibrationspectra,validationspectra,andto
measuredatfdiscretewavelengths(orfrequencies),then X,the
spectra of unknowns which are to be analyzed.
spectral data matrix, is defined as an f by n matrix containing
the spectra (or some function of the spectra produced by
11.2 One type of preprocessing requires special mention.
preprocessing,asdescribedinSection9)ascolumns.Similarly
Mean-centering refers to a procedure in which the average of
y is a vector of dimension n by 1 containing the reference
the calibration spectra (average absorption over the calibration
values for the calibration samples. The object of the linear,
spectra as a function of wavelength or frequency) is calculated
multivariate modeling is to calculate a prediction vector p of
and subtracted from the spectra of the individual calibration
dimension f by 1 that solves Eq 1:
samples prior to the development of the model. The average
t
reference value among the calibration samples is also
y 5 X p1e (1)
calculated, and subtracted from the individual reference values
E1655 − 17
t
where X is the transpose of the matrix X obtained by inter-
reference values (see Section 11) so that the statistics for the
changing the rows and columns of X. The error vector, e,is
model can be compared to those for a single reference value
a vector of dimension n by 1, that is the difference between
determination. The specific method in which the weighting is
the reference values y and their estimates, ŷ,
applied depends on the specific multivariate mathematics
where:
that are employed.
t
yˆ 5 X p (2)
12.1.7 Formostcases,ifthecalibrationspectraarecollected
12.1.4 For some applications, it may be useful to combine
overanextendedwavelength(orfrequency)range,thenumber
the spectral data with other measured variables (for example,
ofindividualabsorptionvaluesperspectrum, f,willexceedthe
sample temperature, pH, mixing rates, etc.). These additional
number of calibration spectra, n. In this case, the matrices
t t
heterogeneous variable may simply be appended to the spec-
(XX ) and (XRX ) are rank deficient and cannot be directly
trum of each sample as if they were additional measured
inverted. Even in cases where f < n, colinearity among the
t t
wavelengths. When heterogeneous data is used, it is important
calibration spectra can cause (XX ) and (XRX ) to be nearly
to consider the possibility that it may be appropriate to apply
singular(tohaveadeterminantthatisnearzero),andthedirect
weighting factors to the heterogeneous variables in order to
use of Eq 4 and Eq 6 can produce an unstable model, that is,
appropriately balance their influence on the calibration with
a model for which changes on the order of th
...

This document is not an ASTM standard and is intended only to provide the user of an ASTM standard an indication of what changes have been made to the previous version. Because
it may not be technically possible to adequately depict all changes accurately, ASTM recommends that users consult prior editions as appropriate. In all cases only the current version
of the standard as published by ASTM is to be considered the official document.
Designation: E1655 − 05 (Reapproved 2012) E1655 − 17
Standard Practices for
Infrared Multivariate Quantitative Analysis
This standard is issued under the fixed designation E1655; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope
1.1 These practices cover a guide for the multivariate calibration of infrared spectrometers used in determining the physical or
chemical characteristics of materials. These practices are applicable to analyses conducted in the near infrared (NIR) spectral
−1
region (roughly 780 to 2500 nm) through the mid infrared (MIR) spectral region (roughly 4000 to 400 cm ).
NOTE 1—While the practices described herein deal specifically with mid- and near-infrared analysis, much of the mathematical and procedural detail
contained herein is also applicable for multivariate quantitative analysis done using other forms of spectroscopy. The user is cautioned that typical and
best practices for multivariate quantitative analysis using other forms of spectroscopy may differ from practices described herein for mid- and
near-infrared spectroscopies.
1.2 Procedures for collecting and treating data for developing IR calibrations are outlined. Definitions, terms, and calibration
techniques are described. Criteria for validating the performance of the calibration model are described.
1.3 The implementation of these practices require that the IR spectrometer has been installed in compliance with the
manufacturer’s specifications. In addition, it assumes that, at the times of calibration and of validation, the analyzer is operating
at the conditions specified by the manufacturer.
1.4 These practices cover techniques that are routinely applied in the near and mid infrared spectral regions for quantitative
analysis. The practices outlined cover the general cases for coarse solids, fine ground solids, and liquids. All techniques covered
require the use of a computer for data collection and analysis.
1.5 These practices provide a questionnaire against which multivariate calibrations can be examined to determine if they
conform to the requirements defined herein.
1.6 For some multivariate spectroscopic analyses, interferences and matrix effects are sufficiently small that it is possible to
calibrate using mixtures that contain substantially fewer chemical components than the samples that will ultimately be analyzed.
While these surrogate methods generally make use of the multivariate mathematics described herein, they do not conform to
procedures described herein, specifically with respect to the handling of outliers. Surrogate methods may indicate that they make
use of the mathematics described herein, but they should not claim to follow the procedures described herein.
1.7 The values stated in SI units are to be regarded as standard. No other units of measurement are included in this standard.
1.8 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility
of the user of this standard to establish appropriate safety safety, health, and healthenvironmental practices and determine the
applicability of regulatory limitations prior to use.
1.9 This international standard was developed in accordance with internationally recognized principles on standardization
established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued
by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
2. Referenced Documents
2.1 ASTM Standards:
D1265 Practice for Sampling Liquefied Petroleum (LP) Gases, Manual Method
D4057 Practice for Manual Sampling of Petroleum and Petroleum Products
D4177 Practice for Automatic Sampling of Petroleum and Petroleum Products
These practices are under the jurisdiction of ASTM Committee E13 on Molecular Spectroscopy and Separation Science and are the direct responsibility of Subcommittee
E13.11 on Multivariate Analysis.
Current edition approved April 1, 2012Dec. 1, 2017. Published May 2012January 2018. Originally approved in 1997. Last previous edition approved in 20052012 as
E1655 – 05.E1655 – 05(2012). DOI: 10.1520/E1655-05R12.10.1520/E1655-17.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or contact ASTM Customer Service at [e-mail]. For Annual Book of ASTM Standards
volume information, refer to the standard’s Document Summary page on the ASTM website.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E1655 − 17
D4855 Practice for Comparing Test Methods (Withdrawn 2008)
D6122 Practice for Validation of the Performance of Multivariate Online, At-Line, and Laboratory Infrared Spectrophotometer
Based Analyzer Systems
D6299 Practice for Applying Statistical Quality Assurance and Control Charting Techniques to Evaluate Analytical Measure-
ment System Performance
D6300 Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products and Lubricants
E131 Terminology Relating to Molecular Spectroscopy
E168 Practices for General Techniques of Infrared Quantitative Analysis
E275 Practice for Describing and Measuring Performance of Ultraviolet and Visible Spectrophotometers
E334 Practice for General Techniques of Infrared Microanalysis
E456 Terminology Relating to Quality and Statistics
E691 Practice for Conducting an Interlaboratory Study to Determine the Precision of a Test Method
E932 Practice for Describing and Measuring Performance of Dispersive Infrared Spectrometers
E1421 Practice for Describing and Measuring Performance of Fourier Transform Mid-Infrared (FT-MIR) Spectrometers: Level
Zero and Level One Tests
E1866 Guide for Establishing Spectrophotometer Performance Tests
E1944 Practice for Describing and Measuring Performance of Laboratory Fourier Transform Near-Infrared (FT-NIR)
Spectrometers: Level Zero and Level One Tests
3. Terminology
3.1 Definitions—For terminology related to molecular spectroscopic methods, refer to Terminology E131. For terminology
relating to quality and statistics, refer to Terminology E456.
3.2 Definitions of Terms Specific to This Standard:
3.2.1 analysis, n—in the context of this practice, the process of applying the calibration model to a spectrum, preprocessed as
required, so as to estimate a component concentration value or property.
3.2.2 calibration, n—a process used to create a model relating two types of measured data. In the context of this practice, a
process for creating a model that relates component concentrations or properties to spectra for a set of known reference samples.
3.2.3 calibration model, n—the mathematical expression or the set of mathematical operations that relates component
concentrations or properties to spectra for a set of reference samples.
3.2.4 calibration samples, n—the set of reference samples used for creating a calibration model. Reference component
concentration or property values are known (measured by reference method) for the calibration samples and a calibration model
is found which relates these values to the spectra during the calibration.
3.2.5 estimate, n—the value for a component concentration or property obtained by applying the calibration model for the
analysis of an absorption spectrum.
3.2.6 model validation, n—the process of testing a calibration model with validation samples to determine bias between the
estimates from the model and the reference method, and to test the agreement between estimates made with the model and the
reference method.
3.2.7 multivariate calibration, n—a process for creating a model that relates component concentrations or properties to the
absorbances of a set of known reference samples at more than one wavelength or frequency.
3.2.8 reference method, n—the analytical method that is used to estimate the reference component concentration or property
value which is used in the calibration and validation procedures.
3.2.9 reference values, n—the component concentrations or property values for the calibration or validation samples which are
measured by the reference analytical method.
3.2.10 spectrometer/spectrophotometer qualification, n—the procedures by which a user demonstrates that the performance of
a specific spectrometer/spectrophotometer is adequate to conduct a multivariate analysis so as to obtain precision consistent with
that specified in the method.
3.2.11 surrogate calibration, n—a multivariate calibration that is developed using a calibration set which consists of mixtures
which contain substantially fewer chemical components than the samples which will ultimately be analyzed.
3.2.12 surrogate method, n—a standard test method that is based on a surrogate calibration.
3.2.13 validation samples—a set of samples used in validating the model. Validation samples are not part of the set of calibration
samples. Reference component concentration or property values are known (measured by reference method), and are compared to
those estimated using the model.
The last approved version of this historical standard is referenced on www.astm.org.
E1655 − 17
4. Summary of Practices
4.1 Multivariate mathematics is applied to correlate the spectra measured for a set of calibration samples to reference component
concentrations or property values for the set of samples. The resultant multivariate calibration model is applied to the analysis of
spectra of unknown samples to provide an estimate of the component concentration or property values for the unknown sample.
4.2 Multilinear regression (MLR), principal components regression (PCR), and partial least squares (PLS) are examples of
multivariate mathematical techniques that are commonly used for the development of the calibration model. Other mathematical
techniques are also used, but may not detect outliers, and may not be validated by the procedure described in these practices.
4.3 Statistical tests are applied to detect outliers during the development of the calibration model. Outliers include high leverage
samples (samples whose spectra contribute a statistically significant fraction of one or more of the spectral variables used in the
model), and samples whose reference values are inconsistent with the model.
4.4 Validation of the calibration model is performed by using the model to analyze a set of validation samples and statistically
comparing the estimates for the validation samples to reference values measured for these samples, so as to test for bias in the
model and for agreement of the model with the reference method.
4.5 Statistical tests are applied to detect when values estimated using the model represent extrapolation of the calibration.
4.6 Statistical expressions for calculating the repeatability of the infrared analysis and the expected agreement between the
infrared analysis and the reference method are given.
5. Significance and Use
5.1 These practices can be used to establish the validity of the results obtained by an infrared (IR) spectrometer at the time the
calibration is developed. The ongoing validation of estimates produced by analysis of unknown samples using the calibration
model should be covered separately (see for example, Practice D6122).
5.2 These practices are intended for all users of infrared spectroscopy. Near-infrared spectroscopy is widely used for quantitative
analysis. Many of the general principles described in these practices relate to the common modern practices of near-infrared
spectroscopic analysis. While sampling methods and instrumentation may differ, the general calibration methodologies are equally
applicable to mid-infrared spectroscopy. New techniques are under study that may enhance those discussed within these practices.
Users will find these practices to be applicable to basic aspects of the technique, to include sample selection and preparation,
instrument operation, and data interpretation.
5.3 The calibration procedures define the range over which measurements are valid and demonstrate whether or not the
sensitivity and linearity of the analysis outputs are adequate for providing meaningful estimates of the specific physical or chemical
characteristics of the types of materials for which the calibration is developed.
6. Overview of Multivariate Calibration
6.1 The practice of infrared multivariate quantitative analysis involves the following steps:
6.1.1 Selecting the Calibration Set—This set is also termed the training set or spectral library set. This set is to represent all of
the chemical and physical variation normally encountered for routine analysis for the desired application. Selection of the
calibration set is discussed in Section 17, after the statistical terms necessary to define the selection criteria have been defined.
6.1.2 Determination of Concentrations or Properties, or Both, for Calibration Samples—The chemical or physical properties,
or both, of samples in the calibration set must be accurately and precisely measured by the reference method in order to accurately
calibrate the infrared model for prediction of the unknown samples. Reference measurements are discussed in Section 9.
6.1.3 The Collection of Infrared Spectra—The collection of optical data must be performed with care so as to present calibration
samples, validation samples, and prediction (unknown) samples for analysis in an alike manner. Variation in sample presentation
technique among calibration, validation, and prediction samples will introduce variation and error which has not been modeled
within the calibration. Infrared instrumentation is discussed in Section 7 and infrared spectral measurements in Section 8.
6.1.4 Calculating the Mathematical Model—The calculation of mathematical (calibration) models may involve a variety of data
treatments and calibration algorithms. The more common linear techniques are discussed in Section 12. A variety of statistical
techniques are used to evaluate and optimize the model. These techniques are described in Section 15. Statistics used to detect
outliers in the calibration set are covered in Section 16.
6.1.5 Validation of the Calibration Model—Validation of the efficacy of a specific calibration model (equation) requires that the
model be applied for the analysis of a separate set of test (validation) samples, and that the values predicted for these test samples
be statistically compared to values obtained by the reference method. The statistical tests to be applied for validation of the model
are discussed in Section 18.
6.1.6 Application of the Model for the Analysis of Unknowns—The mathematical model is applied to the spectra of unknown
samples to estimate component concentrations or property values, or both, (see Section 13). Outlier statistics are used to detect
when the analysis involves extrapolation of the model (see Section 16).
6.1.7 Routine Analysis and Monitoring—Once the efficacy of one or more calibration equations is established, the equations
must be monitored for continued accuracy and precision. Simultaneously, the instrument performance must be monitored so as to
E1655 − 17
trace any deterioration in performance to either the calibration model itself or to a failure in the instrumentation performance.
Procedures for verifying the performance of the analysis are only outlined in Section 22. For petrochemicals, these procedures are
covered in detail in Practice D6122. The use of Practice D6122 requires that a quality control procedure be established at the time
the model is developed. The QC check sample is discussed in Section 22. For practices to compare reference methods and analyzer
methods, refer to PracticesPractice D4855.
6.1.8 Transfer of Calibrations—Transferable calibrations are equations that can be transferred from the original instrument,
where calibration data were collected, to other instruments where the calibrations are to be used to predict samples for routine
analysis. In order for a calibration to be transferable it must perform prediction after transfer without a significant decrease in
performance, as indicated by established statistical tests. In addition, statistical tests that are used to detect extrapolation of the
model must be preserved during the transfer. Bias or slope adjustments, or both, are to be made after transfer only when statistically
warranted. Calibration transfer, that is sometimes referred to as instrument standardization, is discussed in Section 21.
7. Infrared Instrumentation
7.1 A complete description of all applicable types of infrared instrumentation is beyond the scope of these practices. Only a
general outline is given here.
7.2 The IR instrumentation is comprised of two categories, including instruments that acquire continuous spectral data over
wavelength or frequency ranges (spectrophotometers), and those that only examine one or several discrete wavelengths or
frequencies (photometers).
7.2.1 Photometers may have one or a series of wavelength filters and a single detector. These filters are mounted on a turret
wheel so that the individual wavelengths are presented to a single detector sequentially. Continuously variable filters may also be
used in this fashion. These filters, either linear or circular, are moved past a slit to scan the wavelength being measured.
Alternatively, photometers may have several monochromatic light sources, such as light-emitting diodes, that sequentially turn on
and off.
7.3 Spectrophotometers can be classified, based upon the procedure by which light is separated into component wavelengths.
Dispersive instruments generally use a diffraction grating to spatially disperse light into a continuum of wavelengths. In
scanning-grating systems, the grating is rotated so that only a narrow band of wavelengths is transmitted to a single detector at
any given time. Dispersion can occur before the sample (pre-dispersed) or after the sample (post-dispersed).
7.3.1 Spectrophotometers are also available where the wavelength selection is accomplished without moving parts, using a
photodiode array detector. Post-dispersion is utilized. A grating can again provide this function, although other methods, such as
a linear variable filter (LVF) accomplish the same purpose (a LVF is a multilayer filter that has variable thickness along its length,
such that different wavelengths are transmitted at different positions). The photodiode array detector is used to acquire a continuous
spectrum over wavelength without mechanical motion. The array detector is a compact aggregate of up to several thousand
individual photodiode detectors. Each photodiode is located in a different spectral region of the dispersed light beam and detects
a unique range of wavelengths.
7.3.2 The acousto-optical tunable filter is a continuous variant of the fixed filter photometer with no moving optical parts for
wavelength selection. A birefrigent crystal (for example, tellurium oxide) is used, in which acoustic waves at a selected frequency
are applied to select the wavelength band of light transmitted through the crystal. Variations in the acoustic frequency cause the
crystal lattice spacing to change, that in turn, causes the crystal to act as a variable transmission diffraction grating for one
wavelength (that is, a Bragg diffractor). A single detector is used to analyze the signal.
7.3.3 An additional category of spectrophotometers uses mathematical transformations to convert modulated light signals into
spectral data. The most well-known example is the Fourier transform, that when applied to infrared (IR) is known as FT-IR. Light
is divided into two beams whose relative paths are varied by use of a moving optical element (for example, either a moving mirror,
or a moving wedge of a high refractive index material). The beams are recombined to produce an interference pattern that contains
all of the wavelengths of interest. The interference pattern is mathematically converted into spectral data using the Fourier
transform. The FT method can operate in the mid-IR and near-IR spectral regions. The FT instruments use a single detector.
7.3.4 A second type of transformation spectrophotometer uses the Hadamard transformation. Light is initially dispersed with a
grating. Light then passes through a mask mounted on or adjacent to a single detector. The mask generates a series of patterns.
For example, these patterns may be formed by electronically opening and shutting various locations, such as in a liquid crystal
display, or by moving an aperture or slit through the beam. These modulations alter the energy distribution incident upon the
detector. A mathematical transformation is then used to convert the signal into spectral information.
7.4 Infrared instruments used in multivariate calibrations should be installed and operated in accordance with the instructions
of the instrument manufacturer. Where applicable, the performance of the instrument should be tested at the time the calibration
is conducted using procedures defined in the appropriate ASTM practice (see 2.1). The performance of the instrument should be
monitored on a periodic basis using the same procedures. The monitoring procedure should detect changes in the performance of
the instrument (relative to that seen during collection of the calibration spectra) that would affect the estimation made with the
calibration model.
E1655 − 17
7.5 For most infrared quantitative applications involving complex matrices, it is a general consensus that scanning-type
instruments (either dispersive or interferometer based) provide the greatest performance, due to the stability and reproducibility
of modern instrumentation and to the greater amount of spectral data provided for computer interpretation. These data allow for
greater calibration flexibility and additional options for selections of spectral areas less sensitive to band shifts and extraneous noise
within the spectral signal. Scanning/interferometer-based systems also allow greater wavelength/frequency precision between
instruments due to internal wavelength/frequency standardization techniques, and the possibilities of computer-generated spectral
corrections. For example, scanning instruments have received approval for complex matrices, such as animal feed and forages (1,
2).
7.6 Descriptions of instrumentation designs related to Refs (1) and (2) are found in Refs (3) and (4). Other instrumentation
similar in performance to that described in these references is acceptable for all near-infrared techniques described in these
practices.
7.7 For information describing the measurement of performance of ultraviolet, visible, and near infrared spectrophotometers,
refer to Practice E275. For information describing the measurement of performance of dispersive infrared spectrophotometers,
refer to Practice E932. For information describing the measurement performance of Fourier Transform mid-infrared
spectrophotometers, refer to Practice E1421. For information describing the measurement performance of Fourier Transform
near-infrared spectrophotometers, refer to Practice E1944. For spectrophotometers to which these practice do not apply, refer to
Guide E1866.
8. Infrared Spectral Measurements
8.1 Multivariate calibrations are based on Beer’s Law, namely, the absorbance of a homogeneous sample containing an
absorbing substance is linearly proportional to the concentration of the absorbing species. The absorbance of a sample is defined
as the logarithm to the base ten of the reciprocal of the transmittance, (T).
A 5 log 1/T
~ !
The transmittance, T, is defined as the ratio of radiant power transmitted by the sample to the radiant power incident on the
sample.
8.1.1 For measurements conducted by reflectance, the reflectance, R, is sometimes substituted for the transmittance T. The
reflectance is defined as the ratio of the radiant power reflected by the sample to the radiant power incident on the sample.
NOTE 2—The relationship A = log (1/R) is not a definition, but rather an approximation designed to linearize the relationship between the measured
reflectance, R, and the concentration of the absorbing species. For some applications, other linearization functions (for example, Kubelka-Munk) may be
more appropriate (5).
8.1.2 For most types of instrumentation, the radiant power incident on the sample cannot be measured directly. Instead, a
reference (background) measurement of the radiant power is made without the sample being present in the light beam.
NOTE 3—To avoid confusion, the reference measurement of the radiant power will be referred to as a background measurement, and the word reference
will only be used to refer to measurements made by the reference method against which the infrared is to be calibrated. (See Section 9.)
8.1.3 A measurement is then conducted with the sample present, and the ratio, T, is calculated. The background measurement
may be conducted in a variety of ways depending on the application and the instrumentation. The sample and its holder may be
physically removed from the light beam and a background measurement made on the “empty beam”. The sample holder (cell) may
be emptied, and a background measurement may be taken through the “empty cell.”
NOTE 4—For optically thin cells, care may be necessary to avoid optical interferences resulting from multiple internal reflections within the cell. For
very thick cells, differences in the refractive index between the sample and the empty cell may change properties of the optical system, for example, shift
focal points.
8.1.4 The sample holder (cell) may be filled with a liquid that has minimal absorption in the spectral range of interest, and the
background measurement may be taken through the “background liquid.” Alternatively, the light beam may be split or alternately
passed through the sample and through an “empty beam,” an “empty cell,” or a “background liquid.” For reflectance
measurements, the reflectance of a material having minimal absorbance in the region of interest is generally used as the background
measurement.
8.1.5 The particular background referencing scheme that is used may vary among instruments, and among applications. The
same background referencing scheme must be employed for the measurement of all spectra of calibration samples, validation
samples, and unknown samples to be analyzed.
8.2 Traditionally, a sample is manually brought to the instrument and placed in a cell or cuvette with windows that transmit in
the region of interest. Alternatively, transfer pipes can be used to allow liquid to flow through an optical cell in the instrument for
continuous analysis. With optical fibers, the sample can be analyzed remotely from the instrument. Radiation is sent to the sample
The boldface numbers in parentheses refer to a list of references at the end of this standard.
E1655 − 17
through an optical fiber or bundle of fibers and returned to the instrument by means of another fiber or bundle of fibers. Instruments
have been developed that use single fibers to transmit and receive the radiation, as well as those using bundles of fibers for this
purpose. Detectors and radiation sources external to the instrument can also be used, in which case only one fiber or bundle is
needed. For spectral regions where transmitting fibers do not exist, the same function can be performed over limited distances using
appropriate transfer optics.
NOTE 5—If the instrument uses predispersion of the light, some caution must be exercised to avoid introducing ambient light into the system at the
sample position, since such light may be detected, giving rise to erroneous absorbance measurements.
8.3 Although most multivariate calibrations for liquids involve the direct measurement of transmitted light, alternative sampling
technologies (for example, attenuated total reflectance) can also be employed. Transmittance measurements can be employed for
some types of solids (for example, polymer films), whereas other solids (for example, powdered solids) are more commonly
measured by diffuse reflectance techniques.
8.4 For most infrared instrumentation, a variety of adjustable parameters are available to control the collection and computation
of the spectral data. These parameters control, for instance, the optical and digital resolution, and the rate of data acquisition (scan
speed). A detailed description of the spectral acquisition parameters and their effect on multivariate calibrations is beyond the scope
of these practices. However, it is essential that all adjustable parameters that control the collection and computation of spectral data
be maintained constant for the collection of spectra of calibration samples, validation samples, and unknown samples for which
estimates are to be made.
8.5 For definitions and further description of general infrared quantitative measurement techniques, refer to Practices E168. For
a description of general techniques of infrared microanalysis, refer to Practice E334.
9. Reference Method and Reference Values
9.1 Infrared spectroscopy requires calibration to determine the proportionality relationship between the signals measured and
the component concentrations or properties that are to be estimated. During the calibration, spectra are measured for samples for
which these reference values are known, and the relationship between the sample absorbances and the reference values is
determined. The proportionality relationship is then applied to the spectra of unknown samples to estimate the concentration or
property values for the sample.
9.2 For simple mixtures containing only a few chemical components, it is generally possible to prepare mixtures that can serve
as standards for the multivariate calibration of an infrared analysis. Because of potential interferences among the absorbances of
the components, it is not sufficient to vary the concentration of only some of the mixture components, even when analyses for only
one component are being developed. Instead, all components should be varied over a range representative of that expected for
future unknown samples that are to be analyzed. Since infrared measurements are conducted on a fixed volume of sample (for
example, a fixed cell pathlength), it is preferable that concentration reference values be expressed in volumetric terms, for example,
in volume percentage, grams per millilitre, moles per cubic centimetre, and so forth. Developing multivariate calibrations for
reference concentrations expressed in other terms (for example, weight percentage) can lead to models that are linear
approximations to what is really a nonlinear relationship and can lead to less accurate estimates of the concentrations.
9.3 For complex mixtures, such as those obtained from petrochemical processes, preparation of reference standards is generally
impractical, and the multivariate calibration of an infrared analysis must typically be performed on actual process samples. In this
case, the reference values used to calibrate the infrared analysis are obtained by a reference analytical method. The accuracy of
a component concentration or property value estimated by a multivariate infrared analysis is highly dependent on the accuracy and
precision of the reference values used in the calibration. The expected agreement between the infrared estimated values and those
obtained from a single reference measurement can never exceed the repeatability of the reference method, since, even if the
infrared estimated the true value, the measurement of agreement is limited by the precision of the reference values. Knowledge
of the precision (repeatability) of the reference method is critical in the development of an infrared multivariate calibration. The
precision of the reference data used in developing a model, and the accuracy of the model can be improved by averaging repeated
reference measurements.
NOTE 6—If the reference values used to calibrate a multivariate infrared analysis are generated in a single laboratory, it is essential that the measurement
process used to generate these values be monitored for bias and precision using suitable quality assurance procedures (see for example, Practice D6299.
If primary standards are not available to allow the bias of the reference measurement process to be established, it is recommended that the laboratory
participate in an interlaboratory crosscheck program as a means of demonstrating accuracy.
NOTE 7—Samples like hydrocarbons from petrochemical process streams can degrade with time unless careful sampling and sample storage procedures
are followed. It is critical that the composition of samples taken for laboratory or at-line infrared analysis, or for laboratory measurement of the reference
data be representative of the process at the time the samples are taken, and that composition is maintained during storage and transport of the samples
either to the analyzer or to the laboratory. Sampling should be done in accordance with methods like Practices D1265 and D4057, or Practice D4177,
whichever are applicable. Whenever possible, sample storage for extended time periods is not recommended because of the likelihood of samples
degrading with time in spite of sampling precautions taken. Degradation of samples can cause changes in the spectra measured by the analyzer and thus
in the values estimated, and in the property or quality measured by the reference method.
9.4 If the reference method used to obtain reference values for the multivariate calibration is an established ASTM method, then
repeatability and reproducibility data are included in the method. In this case, it is only necessary to demonstrate that the reference
E1655 − 17
measurement is being practiced in accordance with the procedure described in the method, and that the repeatability obtained is
statistically comparable to that published in the method. Data from established quality control procedures can be used to
demonstrate that the repeatability of the reference method is within ASTM specifications. If such data is not available, then
repeatability data should be collected on at least three of the samples that are to be used in the calibration. These samples should
be chosen to span the range of values over which the calibration is to be developed, one sample having a reference value in the
bottom third of the range, one sample having a value in the middle third of the range, and one sample having a value in the upper
third of the range. At least six reference measurements should be made on each sample. The standard deviation among the
measurements should be calculated and compared to that expected based on the published repeatability.
9.5 If the reference method to be used for the multivariate calibration is an established ASTM method, and the samples to be
used in the calibration have been analyzed by a cooperative testing program (for example, octane values obtained from recognized
exchange groups), then the reference values obtained by the cooperative testing program can be used directly, and the standard
deviations established by the cooperative testing program can be used as the estimate of the precision of the reference data.
9.6 Reference methods that are not ASTM methods can be used for the multivariate calibration of infrared analyses, but in this
case, it is the responsibility of the method developer to establish the precision of the reference method using procedures similar
to those detailed in Practice E691, in the Manual for Determining Precision for ASTM Methods on Petroleum Products and
Lubricants and in Practice D6300.
9.7 When multiple reference measurements are made on an individual calibration or validation sample, a Dixon’s Test (see
A1.1) should be applied to the values to determine if all of the reference values came from the same population, or if one or more
of the values is suspect and should be rejected.
10. Simple Procedure to Develop a Feasibility Calibration
10.1 For new applications, it is generally not known whether an adequate IR multivariate model can be developed. In this case,
feasibility studies can be performed to determine if there is a relationship between the IR spectra and the component/property of
interest, and whether a model of adequate precision could possibly be built. If the feasibility calibration is successful, then it can
be expanded and validated. A feasibility calibration involves the following steps:
10.1.1 Approximately 30 to 50 samples are collected covering the entire range for the constituent/property of interest. Care
should be exercised to avoid intercorrelations among major constituents unless such intercorrelations always exist in the materials
being analyzed. The range in the concentration/property should be preferably five times, but not less than three times, the standard
deviation of the reproducibility (reproducibility/2.77) of the reference analysis.
10.1.2 When collecting spectral data on these samples, variations in particle size, sample presentation, and process conditions
which are expected during analysis must be reproduced. Multiple spectra of the same sample under different conditions can be
employed if such variations in conditions are anticipated during analysis.
10.1.3 Reference analyses on these samples are conducted using the accepted reference method. If the range for the
component/property is not at least five times the standard deviation of the reproducibility for the reference analysis, then r replicate
analyses should be conducted on each sample such that the =r times the range is preferably five times, but at least three times,
the standard deviation of the reference analysis.
10.1.4 A calibration model is developed using one or more of the mathematical techniques described in Sections 11 and 12. The
calibration model is preferably tested using cross-validation methods such as SECV or PRESS (see 15.3.6). Other statistics can
also be used to judge the overall quality of the calibration.
10.1.5 If the SECV value obtained from the cross validation suggests that a model of adequate precision can be built, then
additional samples are collected to round out the calibration set, and to serve as a validation set, spectra of these samples are
collected, a final model is developed, and validated as described in Sections 13, 14, and 15.
11. Data Preprocessing
11.1 Various types of data preprocessing algorithms can be applied to the spectral data prior to the development of a multivariate
calibration model. For example, numerical derivatives of the spectra may be calculated using digital filtering algorithms to remove
varying baselines. Such filtering generally causes a significant decrease in the spectral signal-to-noise. Digital filters may also be
employed to smooth data, improving signal to noise at the expense of resolution. A complete description of all possible
preprocessing methods is beyond the scope of these practices. For the purpose of these practices, preprocessing of the spectral data
can be used if it produces a model which has acceptable precision and which passes the validation test described in Section 21.
In addition, any spectral preprocessing method must be automated so as to provide an exactly reproducible result, and must be
applied consistently to all calibration spectra, validation spectra, and to spectra of unknowns which are to be analyzed.
11.2 One type of preprocessing requires special mention. Mean-centering refers to a procedure in which the average of the
calibration spectra (average absorption over the calibration spectra as a function of wavelength or frequency) is calculated and
Manual on Determining Precision Data for ASTM Methods on Petroleum Products and Lubricants, which has been filed at ASTM International Headquarters and may
be obtained by requesting Research Report RR:D02-1007.
E1655 − 17
subtracted from the spectra of the individual calibration samples prior to the development of the model. The average reference
value among the calibration samples is also calculated, and subtracted from the individual reference values for the calibration
samples. The model is then built on the mean-centered data. If the spectral and reference value data are mean-centered prior to
the development of the model, then:
11.2.1 When an unknown sample is analyzed, the average spectrum for the calibration site must be subtracted from the spectrum
of the unknown prior to applying the mean-centered model, and the average reference value for the calibration set must be added
to the estimate from the mean-centered model to obtain the final estimate; and
11.2.2 The degrees of freedom used in calculating the standard error of calibration must be diminished by one to account for
the degree of freedom used in calculating the average (see 15.2).
12. Multivariate Calibration Mathematics
12.1 Multivariate mathematical techniques are used to relate the spectra measured for a set of calibration samples to the
reference values (property or component concentration values) obtained for this set of samples from a reference test. The object
is to establish a multivariate calibration model that can be applied to the spectra of future, unknown, samples to estimate values
(property or component concentration values). Only linear multivariate techniques are described in these practices; that is, it is
assumed that the property or component concentration values can be modeled as a linear function of the sample spectra. Various
nonlinear multivariate techniques have been developed, but have generally not been as widely used as the following linear
techniques. These practices are not intended to compare or contrast among these techniques. For the purpose of these practices,
the suitability of any specific mathematical technique should be judged only on the following two criteria:
12.1.1 The technique should be capable of producing a calibration model that can be validated as described in Section 18; and
12.1.2 The technique should be capable of providing statistics suitable for identifying if samples being analyzed are outside the
range for which the model was developed; that is, when the estimated values represent extrapolation of the model (see 16.3).
NOTE 8—In the following derivations, matrices are indicated using boldface capital letters, vectors are indicated using boldface lowercase letters, and
scalars are indicated using lowercase letters. Vectors are column vectors, and their transposes are row vectors. Italicized lowercase letters indicate matrix
or vector dimensions.
12.1.3 All linear, multivariate techniques are designed to solve the same generic problem. If n calibration spectra are measured
at f discrete wavelengths (or frequencies), then X, the spectral data matrix, is defined as an f by n matrix containing the spectra
(or some function of the spectra produced by preprocessing, as described in Section 9) as columns. Similarly y is a vector of
dimension n by 1 containing the reference values for the calibration samples. The object of the linear, multivariate modeling is to
calculate a prediction vector p of dimension f by 1 that solves Eq 1:
t
y 5 X p1e (1)
E1655 − 17
t
where X is the transpose of the matrix X obtained by interchanging the rows and columns of X. The error vector, e, is a vec-
tor of dimension n by 1, that is the difference between the reference values y and their estimates, ŷ,
where:
t
yˆ 5 X p (2)
12.1.4 For some applications, it may be useful to combine the spectral data with other measured variables (for example, sample
temperature, pH, mixing rates, etc.). These additional heterogeneous variable may simply be appended to the spectrum of each
sample as if they were additional measured wavelengths. When heterogeneous data is used, it is important to consider the
possibility that it may be appropriate to apply weighting factors to the heterogeneous variab
...