ASTM E1948-98(2022)
(Guide)Standard Guide for Analytical Data Interchange Protocol for Chromatographic Data
Standard Guide for Analytical Data Interchange Protocol for Chromatographic Data
SIGNIFICANCE AND USE
6.1 General Coding Guidelines—The NetCDF libraries are supplied to developers as source code. End users receive the libraries in compiled binary form as part of a vendor’s application.
6.1.1 Some vendors found that compilation using the huge memory model under Microsoft Windows on MS-DOS was needed because of an array pointer passing its boundary. Many vendors chose to write a conversion program as a stand-alone DOS application, whereby the conversion program used only text-mode screen I/O. Some vendors are now using it in their MS-Windows applications.
6.1.2 Developers setting out to write a program to convert their data files to the Chromatographic Data Protocol should use the NetCDF utilities ncgen and ncdump. Applying the ncgen utility to the CHROMSTD.CDL template will generate the skeletal code needed to create the NetCDF file. The programmer can then modify the code to create a program that reads the netDCF file from another vendor. After developers create the NetCDF file they should use the ncdump program to generate the ASCII representation of the data files, and examine it to ensure the data is being correctly put into the file.
6.2 Make Files for NetCDF Libraries and Utilities—In general the compilation is straightforward. The make files were modified after they were received from the Unidata Corporation, because they did not compile the first time on PCs. The changes needed to get the Unidata distribution to run on DOS are (1) rename the file MAKEFILE to UNIX.MK, and (2) rename MSOFT.MK to MAKEFILE, and then run NMAKE. The default switches in the Unidata distribution use the switches for the floating point coprocessor and Microsoft Windows options.
6.2.1 The protocol contain some complete makefile examples for Microsoft C V6.0 running on DOS. The Microsoft C V6.0 compiler manual should be consulted for the exact meaning of the compiler and linker options.
6.2.2 The VMS and SunOS compilation instructions are in directories for those operatin...
SCOPE
1.1 This guide covers the implementation of the Chromatographic Data Protocol in analytical software applications. Implementation of this protocol requires:
1.1.1 Specification E1947, which contains the full set of data definitions. The chromatographic data protocol is not based upon any specific implementation; it is designed to be independent of any particular implementation, so that implementations can change as technology evolves. The protocol is implemented in stages, to speed its acceptance through actual use.
1.1.2 Specification E1947 contains a full description of the contents of the data communications protocol, including the analytical information categories with data elements and their attributes for most aspects of chromatographic tests.
1.2 The Analytical Information Categories are a practical convenience for breaking down the standardization process into smaller, more manageable pieces. It is easier for developers to build consensus and produce working systems based on smaller information sets, without the burden and complexity of the hundreds of data elements contained in all the categories. The categories also assist vendors and end users in using the guide in their computing environments.
1.3 The NetCDF Data Interchange System is the container used to communicate data between applications in a way that is independent of both computer architectures and end-user applications. In essence, it is a special type of application designed for data interchange.
1.4 The Common Data Language (CDL) Template for Chromatography is a language specification of the chromatography dataset being interchanged. With the use of the NetCDF utilities, this human-readable template can be used to generate an equivalent binary file and the software subroutine calls needed for input and output of data in analytical applications.
1.5 This international standard was developed in accordance with internationally recogn...
General Information
Relations
Standards Content (Sample)
This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the
Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
Designation: E1948 − 98 (Reapproved 2022)
Standard Guide for
Analytical Data Interchange Protocol for Chromatographic
Data
This standard is issued under the fixed designation E1948; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (´) indicates an editorial change since the last revision or reapproval.
1. Scope 1.5 This international standard was developed in accor-
dance with internationally recognized principles on standard-
1.1 This guide covers the implementation of the Chromato-
ization established in the Decision on Principles for the
graphic Data Protocol in analytical software applications.
Development of International Standards, Guides and Recom-
Implementation of this protocol requires:
mendations issued by the World Trade Organization Technical
1.1.1 Specification E1947, which contains the full set of
Barriers to Trade (TBT) Committee.
data definitions. The chromatographic data protocol is not
based upon any specific implementation; it is designed to be
2. Referenced Documents
independent of any particular implementation, so that imple-
2.1 ASTM Standard:
mentations can change as technology evolves. The protocol is
E1947 Specification for Analytical Data Interchange Proto-
implemented in stages, to speed its acceptance through actual
col for Chromatographic Data
use.
2.2 Other Standard:
1.1.2 Specification E1947 contains a full description of the
NetCDF User’s Guide
contents of the data communications protocol, including the
analytical information categories with data elements and their
3. Features of the Chromatographic Data Interchange
attributes for most aspects of chromatographic tests.
Implementation Guide
1.2 The Analytical Information Categories are a practical
3.1 The chromatographic data protocol consists of: (1) the
convenience for breaking down the standardization process
data contents (what is being transferred), and (2) the data
into smaller, more manageable pieces. It is easier for develop-
container (how it is being transferred).
ers to build consensus and produce working systems based on
3.2 Analytical Information Categories—To make the stan-
smaller information sets, without the burden and complexity of
the hundreds of data elements contained in all the categories. dardization process more manageable, fiveAnalytical Informa-
tion Categories were defined:
The categories also assist vendors and end users in using the
guide in their computing environments.
1. Raw Data
2. Final Results
1.3 The NetCDF Data Interchange System is the container
3. Full Data Processing Method
4. Full Chemical Method
used to communicate data between applications in a way that
5. Good Laboratory Practices Information
is independent of both computer architectures and end-user
3.2.1 This guide covers implementation of Raw Data (Cat-
applications. In essence, it is a special type of application
egory 1) and Final Results (Category 2). The development
designed for data interchange.
committee has completed implementation and testing of infor-
1.4 The Common Data Language (CDL) Template for
mation in these categories.
Chromatography is a language specification of the chromatog-
3.3 Chromatography Data Elements—Each data element
raphy dataset being interchanged. With the use of the NetCDF
specifies a piece of information that is to be stored or
utilities, this human-readable template can be used to generate
transferred to another system. A data element has a name, a
an equivalent binary file and the software subroutine calls
definition, and may have some default attributes, such as its
needed for input and output of data in analytical applications.
For referenced ASTM standards, visit the ASTM website, www.astm.org, or
This guide is under the jurisdiction of ASTM Committee E13 on Molecular contact ASTM Customer Service at service@astm.org. For Annual Book of ASTM
Spectroscopy and Separation Science and is the direct responsibility of Subcom- Standards volume information, refer to the standard’s Document Summary page on
mittee E13.15 on Analytical Data. the ASTM website.
Current edition approved Nov. 1, 2022. Published November 2022. Originally Available from Russell K. Rew, Unidata Program Center, University Corpora-
approved in 1998. Last previous edition approved in 2014 as E1948 – 98 (2014). tion for Atomospheric Research, P. O. Box 3000, Boulder, CO 80307-3000,
DOI: 10.1520/E1948-98R22. http://www2.ucar.edu.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States
E1948 − 98 (2022)
datatype, the analytical information category to which it wrapper code needed for FORTRAN programs. The ncdump
belongs, and whether it is mandatory or recommended. A data utility reads the binary files generated by ncgen (which were
element may also reference other data elements. then populated by the application using the NetCDF libraries),
and then generates their ASCII-encoded, human-readable rep-
3.4 NetCDF Data Interchange System—NetCDF is a data
resentation. ncdump is routinely used to check the contents of
interchange system designed specifically for the requirements
NetCDF files, and debug applications that use NetCDF. It can
of scientific data. It was developed to meet the requirements of
also be used to export data to reporting programs. Fig. 1
high-performance, large dataset interchange over networks, in
illustrates the usage of the NetCDF utilities.
a machine- and network-independent way. NetCDF is imple-
3.6.2 Developers typically start with the CDL source code
mented in the portable C programming language, and is made
description of the data file they wish to interchange. Then the
to be available on all popular scientific computers and operat-
ncgen utility with the -n option generates the NetCDF binary
ing systems. It achieves platform-independence through the
file. Developers then use ncgen with the -c option to generate
use of the XDR protocol for byte-stream encoding. XDR
the C source code needed for the subroutines to read and write
(eXternal Data Representation) was developed by Sun Micro-
this particular dataset (specified by the CDL template.) These
systems as part of the Network File System (NFS) and is now
read/write subroutines make up the NetCDF Application Pro-
in the public domain. NetCDF has a particular advantage over
gramming Interface (API). These routines are simply cut and
many other approaches, such as ASCII flat files or structured
pasted into the target application(s).The programmer then uses
ASCII files, because it uses the abstraction of a data access
the API to read or write data in NetCDF datasets.
interface or abstractApplication Programming Interface (API).
3.6.3 If FORTRAN is the programming language being
This API lets the application developer deal with data more in
used, the ncgen utility with the -f option can generate the
terms of logical entities, instead of having to think about
FORTRAN source code wrappers needed to embed the
specific details of encoding and decoding data. This guide
NetCDF library code into FORTRAN applications.
specifies the use of version 2.0 or later, and the exchange
3.6.4 Once the application populates the data file with its
software should determine only that the version of NetCDF is
data, that file can be transferred to another system. If a
version 2.0 or greater.
developer or end user needs a human-readable version of the
3.5 Common Data Language—ANetCDF data file has both binary dataset, the binary file can be fed into the ncdump
binary and human-readable representations. The human-
utility, which regenerates the original CDL source code. The
readable form is anASCII-encoded file written in the NetCDF ncdump utility has an option switch to regenerate the CDL
system’s Common Data Language (CDL). CDL is a data
code with or without the actual experimental data. Some
description language, which is a simplified type of program- datasets can be enormous and there may be no reason to view
ming language without any action statements. The main
the entire dataset. In this way, data are easily encoded into
purpose of a data description language is the specification of NetCDF files and the required routines to read and write those
data structures. CDL is a generic data description language for
data are incorporated into applications.
scientificdata;itisnotapplication-specific.Thebasicconstruct
3.6.5 There is another way to build NetCDF data files.
in CDL is that of a multidimensional array, to which many Using the NetCDF Application Programming Interface, it is
types of metadata can be attached. Metadata is additional data
possible to build NetCDF binary files one data element at a
about the data, that is, attributes about the data that help turn it time.However,thatmethodisgenerallymoretime-consuming,
into useful information.
and it is not recommended except for those situations where
applications can fully automate the process.
3.5.1 For the protocol, a template for chromatography
written in CDL is supplied to developers. The CDL template
4. Chromatographic Data Protocol Distribution Kit
provides an easily comprehended text version of the structure
4.1 Information on how to obtain the kit will be posted on
and contents of a binary NetCDF file. Developers use this
template with certain NetCDF utilities to generate the software the ASTM web site (www.astm.org) under Committee E49.
code (subroutine calls) needed to use the NetCDF toolkit. The
generated code can then be cut and pasted in the end-user
application.
3.5.2 For a full description of the Common Data Language,
see the NetCDF User’s Guide.
3.6 NetCDF Utilities—The NetCDF data interchange sys-
tem has two very powerful utilities called “ncgen” and “nc-
dump” that eliminate much work for developers.
3.6.1 After the template for a particular analytical technique
dataset (in this case chromatography) is written in CDL, it is
fed into ncgen. There are several options with ncgen, the first
of which creates the binary file and populates it with fields
specified by the CDL template. Other ncgen options generate
code for the C language subroutines required for an application
to read and write data in the NetCDF file, and the FORTRAN FIG. 1 Usage of NetCDF Utilities
E1948 − 98 (2022)
4.2 TheAnalyticalDataInterchangeProtocolforChromato- 6.1.1 Some vendors found that compilation using the huge
graphic Data Distribution Kit contains: memory model under Microsoft Windows on MS-DOS was
4.2.1 Software Disks—NetCDF distribution kit from Uni- needed because of an array pointer passing its boundary. Many
data (with the modified makefile needed to make the kit vendors chose to write a conversion program as a stand-alone
compile out of the box), and the Chromatographic Data DOS application, whereby the conversion program used only
Protocol CDL Template. text-mode screen I/O. Some vendors are now using it in their
4.2.2 NetCDF User’s Guide—supplied by Unidata Program MS-Windows applications.
Center.
6.1.2 Developers setting out to write a program to convert
4.2.3 Specification E1947.
their data files to the Chromatographic Data Protocol should
4.2.4 Guide E1948.
use the NetCDF utilities ncgen and ncdump. Applying the
4.2.5 Registration Card, used for tracking and user updates.
ncgen utility to the CHROMSTD.CDL template will generate
the skeletal code needed to create the NetCDF file. The
5. Hardware and Software
programmer can then modify the code to create a program that
5.1 This section describes the hardware and software con-
reads the netDCF file from another vendor. After developers
figurations used for testing. In general, the NetCDF system
create the NetCDF file they should use the ncdump program to
puts very few requirements on the hardware, because most
generate the ASCII representation of the data files, and
routines are left on disk. Only routines being used at any
examine it to ensure the data is being correctly put into the file.
particulartimearekeptinmemory.Anylimitationsfoundwere
6.2 Make Files for NetCDF Libraries and Utilities—In
typically those not imposed by NetCDF but ones imposed by
generalthecompilationisstraightforward.Themakefileswere
the operating system or environment.
modified after they were received from the Unidata
5.1.1 Hardware (Personal Computers)—The personal com-
Corporation, because they did not compile the first time on
puter system hardware used for testing was:
PCs. The changes needed to get the Unidata distribution to run
5.1.1.1 Intel 80286 processor,
on DOS are (1) rename the file MAKEFILE to UNIX.MK, and
5.1.1.2 640K minimum,
(2) rename MSOFT.MK to MAKEFILE, and then run
5.1.1.3 monochrome, EGA, VGA graphics,
NMAKE. The default switches in the Unidata distribution use
5.1.1.4 20 megabyte minimum, 80 megabyte hard-disk is
the switches for the floating point coprocessor and Microsoft
typical, and
Windows options.
5.1.1.5 a mouse (optional).
6.2.1 The protocol contain some complete makefile ex-
5.1.1.6 NetCDF works well on AT-class machines and
amples for Microsoft C V6.0 running on DOS. The Microsoft
higher. NetCDF does not have the items in 5.1.1.1 – 5.1.1.5 as
C V6.0 compiler manual should be consulted for the exact
requirements. These are just the minimum, base-level systems
meaning of the compiler and linker options.
that were used.
6.2.2 The VMS and SunOS compilation instructions are in
5.1.2 Software—NetCDF runs on MS-DOS, OS/2,
directories for those operating systems.
Macintosh, Windows 95, and Windows NT operating systems
for personal computers. NetCDF was originally ported from
6.3 NetCDF Library Build Order—The NetCDF libraries
UNIX to DOS running on an IBM-PS/2 Model 80. It was
must be built in a specific order. The correct order to build the
recently ported to the Macintosh OS. NetCDF is written is the
NetCDF directories is:
C programming language, and there are FORTRAN jackets
UTIL
available for applications that want to use FORTRAN calls.
XDR
The personal computer software employed for testing and
SRC
NCDUMP
developing NetCDF applications were:
NCGEN
5.1.2.1 Microsoft DOS V3.3 or above,
NCTEST
5.1.2.2 Microsoft C Compiler V6.0,
6.3.1 The UTIL and XDR makefiles work as distributed
5.1.2.3 Microsoft Windows V3.0,
using NMAKE with Microsoft C V6.0.
5.1.2.4 Microsoft Windows SDK, and
5.1.2.5 NetCDF Version 2.0.1.
7. CDL Template Structure
5.1.3 Workstations and Servers—NetCDF runs easily on
UNIX workstations such as Sun 3, Sun 4, VAXstations,
7.1 A NetCDF template i
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.