ASTM E2171-02(2013)
(Practice)Standard Practice for Rating-Scale Measures Relevant to the Electronic Health Record (Withdrawn 2017)
Standard Practice for Rating-Scale Measures Relevant to the Electronic Health Record (Withdrawn 2017)
SIGNIFICANCE AND USE
4.1 The simplicity and practicality of Rasch's probabilistic scale-free measurement models have brought within reach universal metrics for educational and psychological tests, and for rating scale-based instruments in general. There are at least 3 implications to the application of Rasch's models to the health-related calibration of universal metrics for each of the variables relevant to the Electronic Health Record (EHR) that are typically measured using rating scale instruments.
4.1.1 First, establishing a single metric standard with a defined range and unit will arrest the burgeoning proliferation of new scale-dependent metrics.
4.1.2 Second, the communication of the information pertaining to patient status represented by these measures (physical, cognitive, and psychosocial health status, quality of life, satisfaction with services, etc.) will be simplified.
4.1.3 Third, common standards of data quality will be used to evaluate and improve instrument performance. The vast majority of test and survey data quality is currently almost completely unknown, and when quality is evaluated, it is via many different methods that are often insufficient to the task, misapplied, misinterpreted, or even contradictory in their aims.
4.1.4 Fourth, currently unavailable economic benefits will accrue from the implementation of measurement methods based on quality-assessed data and widely accepted reference standard metrics. The potential magnitude of these benefits can be seen in an assessment of 12 different metrological improvement studies conducted by the National Science and Technology Council (Subcommittee on Research, 1996). The average return on investment associated with these twelve studies was 147 %. Is there any reason to suppose that similar instrument improvement efforts in the psychosocial sciences will result in markedly lower returns?
4.2 Until now, it has been assumed that the Practice E1384 would necessarily have to stipulate fields for the EHR th...
SCOPE
1.1 This standard addresses the identification of data elements from the EHR definitions in Practice E1384 that have ordinal scale value sets and which can be further defined to have scale-free measurement properties. It is applicable to data recorded for the Electronic Health Record and its paper counterparts. It is also applicable to abstracted data from the patient record that originates from these same data elements. It is applicable to identifying the location within the EHR where the observed measurements shall be stored and what is the meaning of the stored data. It does not address either the uses or the interpretations of the stored measurements.
WITHDRAWN RATIONALE
Formerly under the jurisdiction of Committee E31 Healthcare Informatics, this practice was withdrawn in March 2017. This standard is being withdrawn without replacement due to its limited use by industry.
General Information
Relations
Standards Content (Sample)
NOTICE: This standard has either been superseded and replaced by a new version or withdrawn.
Contact ASTM International (www.astm.org) for the latest information
Designation: E2171 − 02 (Reapproved 2013) An American National Standard
Standard Practice for
Rating-Scale Measures Relevant to the Electronic Health
Record
This standard is issued under the fixed designation E2171; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision.Anumber in parentheses indicates the year of last reapproval.A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope 3.2.3 bias analysis—investigation of considerations relative
to subject or area of performance.
1.1 This standard addresses the identification of data ele-
3.2.4 calibration—process of establishing additivity and
ments from the EHR definitions in Practice E1384 that have
reproducability of a data set.
ordinal scale value sets and which can be further defined to
havescale-freemeasurementproperties.Itisapplicabletodata
3.2.5 concatenation—processofmeasurementusesenumer-
recorded for the Electronic Health Record and its paper
ated physical unit quantities equal to the magnitude of the
counterparts. It is also applicable to abstracted data from the
measured item.
patientrecordthatoriginatesfromthesesamedataelements.It
3.2.6 construct—name of the conceptual domain measured.
is applicable to identifying the location within the EHR where
3.2.7 convergence—closing of the differences in sequential
the observed measurements shall be stored and what is the
measure estimates.
meaning of the stored data. It does not address either the uses
3.2.8 counting—basic activity upon which measurement is
or the interpretations of the stored measurements.
based and utilizes enumeration.
2. Referenced Documents
3.2.9 data—observation made in such a way that they lead
2.1 ASTM Standards:
to generalization.
E177Practice for Use of the Terms Precision and Bias in
3.2.10 data quality/ statistical consistency/ model fit—
ASTM Test Methods
establishment of whether the measuring instrument is affected
E456Terminology Relating to Quality and Statistics
by the object of measurement.
E691Practice for Conducting an Interlaboratory Study to
3.2.11 determinism—measurement model that requires
Determine the Precision of a Test Method
counts to be sufficient for reproducing the pattern of the
E1169Practice for Conducting Ruggedness Tests
responses over the length of the instrument.
E1384Practice for Content and Structure of the Electronic
3.2.12 dimensionality—property of having multiple compo-
Health Record (EHR)
nents of a measured value.
3. Terminology
3.2.13 equality/cocalibration—process of ensuring that dif-
ferent instruments measure the same property.
3.1 Definitions—Full definitions and discussion of Scale-
Free Measurement Terms are given in Annex A1.
3.2.14 error—uncertainty of measured properties.
3.2 Definitions of Terms Specific to This Standard:
3.2.15 estimation algorithms—mathematical specification
3.2.1 adaptive measurement—advantage of measurement to of an observational framework.
account for missing data.
3.2.16 incommensurable/commensurable—measure value
3.2.2 additivity—rating scale adherence to associativity and of the same quantity does/does not depend upon rating/
commutability. responses of the rating construct and does not/does remain
constant.
3.2.17 instrument—sensing device having a defined scale.
This practice is under the jurisdiction ofASTM Committee E31 on Healthcare
Informatics and is the direct responsibility of Subcommittee E31.25 on Healthcare
3.2.18 intra and inter-laboratory testing—variabilitytesting
Data Management, Security, Confidentiality, and Privacy.
usingthesamesetting/measure/operatorasopposedtodifferent
Current edition approved March 1, 2013. Published March 2013. Originally
setting/measure/operators.
approved in 2002. Last previous edition approved in 2008 as E2171–02(2008).
DOI: 10.1520/E2171-02R13.
3.2.19 item response/latent trait theory—analytic models
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
that forego prescriptive parameter separation, sufficiency and
contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
scale and sample free data standards for additional descriptive
Standards volume information, refer to the standard’s Document Summary page on
the ASTM website. parameters.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E2171 − 02 (2013)
3.2.20 items/item-bank—part of survey statements/test 3.2.43 software—packages of machine code used for data
questions for adaptive administration. analysis.
3.2.44 specific objectivity—data satisfying the separability
3.2.21 levels of measurement—nature of scale of measure-
theorem.
ment.
3.2.45 standardized—common conventions for instruments,
3.2.22 logit—scale unit using logarithms of odds ratios.
reference measurement material, scales and units of measure
(P/1−P).
for a measurement process.
3.2.23 mathematicalentities—conceptsthatcanbetaughtor
3.2.46 suffıciency—statistics that extract all available infor-
learned through what is already known.
mation from the data.
3.2.24 measurement—determining in units the value of a
3.2.47 targeting—lack of floor an/or ceiling effects in mea-
property in a scale having magnitude (that is, ratio or differ-
surement.
ence).
3.2.48 transparency—ability to “look through” raw scores
3.2.25 metaphor in measurement—suspension of disbelief
to the composite ratings producing that score (see also suffı-
of some areas or properties in the name of estimating magni-
ciency).
tude.
3.2.49 unit of measurement—common conventions for the
3.2.26 metric—measure of a property in defined units.
appropriate smallest basic measures for a given construct.
3.2.27 missing data—use of uncalibrated data in instru-
3.2.50 validity/construct/content—both content and con-
ments with varying numbers of items.
struct must make sound theoretical sense to be considered
3.2.28 multi-faceted measurement—use of measurement valid.
models that have more than two basic parameters.
3.2.51 variable—attribute of the property being measured.
3.2.29 ordinal data—one scale for measurement.
4. Significance and Use
3.2.30 population—universe of elements relevant to mea-
4.1 The simplicity and practicality of Rasch’s probabilistic
surement of a particular construct.
scale-free measurement models have brought within reach
3.2.31 probabilistic conjoint measurement—framework for
universal metrics for educational and psychological tests, and
demonstrating data quality, statistical consistency, and model
for rating scale-based instruments in general.There are at least
fit of non-deterministic measures with a stable order of facets.
3 implications to the application of Rasch’s models to the
3.2.32 quantification—cocalibration of different constructs
health-related calibration of universal metrics for each of the
with respect to the same property (variable) in a common
variables relevant to the Electronic Health Record (EHR) that
metric.
are typically measured using rating scale instruments.
4.1.1 First, establishing a single metric standard with a
3.2.33 Rasch analysis measurement and models—analytic
defined range and unit will arrest the burgeoning proliferation
modelspecifyingtheobservationalframeworkanddataquality
of new scale-dependent metrics.
measures for quantification.
4.1.2 Second, the communication of the information per-
3.2.34 raw score—sum of ratings or count of direct re-
taining to patient status represented by these measures
sponses in a given measurement event.
(physical, cognitive, and psychosocial health status, quality of
3.2.35 reliability—ratio of variation to error or signal to
life, satisfaction with services, etc.) will be simplified.
noise.
4.1.3 Third, common standards of data quality will be used
to evaluate and improve instrument performance. The vast
3.2.36 repeatability—variability of measurements in a
majority of test and survey data quality is currently almost
single setting by a single operator using the same measuring
completely unknown, and when quality is evaluated, it is via
instrument.
many different methods that are often insufficient to the task,
3.2.37 reproducibility—variability of measurements in dif-
misapplied,misinterpreted,orevencontradictoryintheiraims.
ferent settings.
4.1.4 Fourth, currently unavailable economic benefits will
3.2.38 root mean square error—mathematical algorithm for
accrue from the implementation of measurement methods
determining the variation due to error of the estimates.
based on quality-assessed data and widely accepted reference
standardmetrics.Thepotentialmagnitudeofthesebenefitscan
3.2.39 sample—subset of measured population.
be seen in an assessment of 12 different metrological improve-
3.2.40 sample size—magnitude of the measured population.
ment studies conducted by the National Science and Technol-
3.2.41 scale-free/scale-dependent—measures not affected ogy Council (Subcommittee on Research, 1996). The average
bytheinstrumentemployedasopposedtomeasuresthatareso
return on investment associated with these twelve studies was
affected. 147%. Is there any reason to suppose that similar instrument
improvement efforts in the psychosocial sciences will result in
3.2.42 separability theorem/parameter separation—ability
markedly lower returns?
of measures to be independent of the instrument selected and
ability of the instrument’s item calibrations to be independent 4.2 Until now, it has been assumed that the Practice E1384
of the sample measured. would necessarily have to stipulate fields for the EHR that
E2171 − 02 (2013)
would contain summary scores from commonly used func- 4.4.1 The use of measures, not scores, in all capture of data
tionalassessment,healthstatus,qualityoflife,andsatisfaction from the EHR for statistical comparisons;
instruments. This is because standards for rating scale instru- 4.4.2 The reporting of both the traditional reliability statis-
ments to date have been entirely content-based. Those who tics (Cronbach’s alpha or the KR20) and the additive, linear
separationstatistics(Wright&Masters,1982),alongwiththeir
havesought“gold”orcriterionstandardsthatwouldcommand
universal respect and relevance have been stymied by the error and variation components, for both the measures and the
calibrations;
impossibility of identifying content (survey questions and
rating categories) capable of satisfying all users’ needs. Com- 4.4.3 Aqualitativeelaborationofthevariabledefinedbythe
order of the survey questions or test items on the measurement
munication of patient statistics between managers and
continuum, preferably in association with a figure displaying
clinicians, or payors and providers, may require one kind of
the variable;
information; between providers and referral sources, other
4.4.4 Reporting of means and standard deviations for each
kinds; between providers and accreditors, yet another; among
of the three essential measurement statistics, the measure, the
clinicians themselves, still another; and even more kinds of
error, and the model fit;
information may be required for research applications.
4.4.5 Statement of the full text of at least a significant
4.2.1 For instance, payors may want to know outcome
sample of the questions included on the instrument;
information that tells them what percentage of patients dis-
4.4.6 Specification of the mathematical model employed,
charged can function independently at home. A hospital
with a justification for its use;
manager, referral source, or accreditor might want to know
4.4.7 Specification of the error estimation and model fit
more detail, such as percentages of patients discharged who
estimationalgorithmsemployed,withmathematicaldetailsand
can dress, bathe, walk, and eat independently. Clinicians will
justification provided when they differ from those routinely
wanttoknowstillmoredetailaboutamountsofindependence,
used;
such as whether there are safety issues, needs for assistive
4.4.8 Evaluation of overall model fit, elaborated in a report
devices, or specific areas in which functionality could be
on the details of one or more of the least and most consistent
improved. Researchers may seek even more detail yet, as they
response patterns observed;
evaluate differences in outcomes across treatment programs,
4.4.9 Graphical comparison of at least two calibrations of
diagnostic groups, facilities, levels of care, etc.
newinstrumentsfromdifferentsamplesofthesamepopulation
4.2.1.1 Members of each of these groups have, at some
to establish the invariance of the item calibration order across
time, felt that their particular information needs have not been
samples;
met by the tools designed and developed by members of
4.4.10 Graphical comparison of measures produced by at
another group. Despite the fact that the information provided
least two subsets of items on new instruments to establish the
by these different tools appears in many different forms and at
invariance of the person measure order across scales (collec-
different levels of detail, to the extent that they can be shown
tions of items);
to measure the same thing, they can do so in the same metric.
4.4.11 Graphicalcomparisonofnewinstrumentcalibrations
This is the primary result of the introduction of Rasch’s
withthecalibrationsproducedbyotherinstrumentsintendedto
probabilistic scale-free measurement models. The different
measure the same variable in the same population, to establish
purposes guiding the design of the instruments will still
the potential for sample-free equating of the instruments and
continue to impact the two fundamental statistics associated
establishment of reference standards;
witheverymeasure:theerrorandmodelfit.Moregeneral,and
4.4.12 At least a useable prototype of the instrument
also less well-designed instruments, will measure with more
employed, with the worksheet laid out to produce informative
error than those that make more detailed and consistent
quantitative measures (not summed scores) as soon as it is
distinctions. Data consistency is the key to scale-free measure-
filled out; and
ment.
4.4.13 Graphical presentation of the treatment and control
groups’ measurement distributions, for the purpose of facili-
4.3 Theremainderofthisdocument(1)identifies,inSection
tating a substantive interpretations of differences’ significance.
5, the fields in the current Practice E1384 targeted for change
from a scale-dependent to a scale-free measurement orienta-
5. Applicable Data Elements
tion; (2) lists referenced ASTM documents; (3) defines scale-
5.1 The data elements in Practice E1384 which are affected
free measurement terms, often contrasting them with their
bythesuggestionsformeasurem
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.