ASTM D 6122 : 2023
Current
The latest, up-to-date edition.
Standard Practice for Validation of the Performance of Multivariate Online, At-Line, Field and Laboratory Infrared Spectrophotometer, and Raman Spectrometer Based Analyzer Systems
Hardcopy , PDF
English
01-03-2024
Committee |
D 02
|
DocumentType |
Standard Practice
|
Pages |
52
|
PublisherName |
American Society for Testing and Materials
|
Status |
Current
|
Supersedes |
1.1This practice covers requirements for the validation of measurements made by laboratory, field, or process (online or at-line) infrared (near- or mid-infrared analyzers, or both), and Raman analyzers, used in the calculation of physical, chemical, or quality parameters (that is, properties) of liquid petroleum products and fuels. The properties are calculated from spectroscopic data using multivariate modeling methods. The requirements include verification of adequate instrument performance, verification of the applicability of the calibration model to the spectrum of the sample under test, and verification that the uncertainties associated with the degree of agreement between the results calculated from the infrared or Raman measurements and the results produced by the PTM used for the development of the calibration model meets user-specified requirements. Initially, a limited number of validation samples representative of current production are used to do a local validation. When there is an adequate number of validation samples with sufficient variation in both property level and sample composition to span the model calibration space, the statistical methodology of Practice D6708 can be used to provide general validation of this equivalence over the complete operating range of the analyzer. For cases where adequate property and composition variation is not achieved, local validation shall continue to be used.
1.1.1For some applications, the analyzer and PTM are applied to the same material. The application of the multivariate model to the analyzer output (spectrum) directly produces a PPTMR for the same material for which the spectrum was measured. The PPTMRs are compared to the PTMRs measured on the same materials to determine the degree of agreement.
1.1.2For other applications, the material measured by the analyzer system is subjected to a consistent additive treatment prior to being analyzed by the PTM. The application of the multivariate model to the analyzer output (spectrum) produces a PPTMR for the treated material. The PPTMRs based on the analyzer outputs are compared to the PTMRs measured on the treated materials to determine the degree of agreement.
1.1.3 In some cases, a two-step procedure is employed. In the first step, the analyzer and PTM are applied to the measurement of a blendstock material. In a second step, the PPTMRs produced in Step 1 are used as inputs to a second model that predicts the results obtained when the PTM is applied to the analysis of the finished blended product produced by additivation to the blendstock. If the analyzer used in the first step is a multivariate spectroscopic based analyzer, then this practice is used to access the degree of agreement between PPTMRs and PTMRs. Otherwise, Practice D3764 is used to compare the PPTMRs to the PTMRs for this blendstock to determine the degree of agreement. Since this second step does not use spectroscopic data, the validation of the second step is done using Practice D3764. If the first step uses a multivariate spectrophotometric analyzer, then only samples for which the spectra are not outliers relative to the multivariate model are used in the second step. Note that the second model might accommodate variable levels of additive material addition to the blend stock.
1.2Multiple physical, chemical, or quality properties of the sample under test are typically predicted from a single spectral measurement. In applying this practice, each property prediction is validated separately. The separate validation procedures for each property may share common features, and be affected by common effects, but the performance of each property prediction is evaluated independently. The user will typically have multiple validation procedures running simultaneously in parallel.
1.3Results used in analyzer validation are for samples that were not used in the development of the multivariate model, and for spectra which are not outliers or nearest neighbor inliers relative to the multivariate model.
1.4When the number, composition range or property range of available validation samples do not span the model calibration range, a local validation is done using available samples representative of current production. When the number, composition range and property range of available validation samples becomes comparable to those of the model calibration set, a general validation can be done.
1.4.1Local Validation:
1.4.1.1The calibration samples used in developing the multivariate model must show adequate compositional and property variation to enable the development of a meaningful correlation, and must span the compositional range of samples to be analyzed using the model to ensure that such analyses are done via interpolation rather than extrapolation. The Standard Error of Calibration (SEC) is a measure of how well the PTMRs and PPTMRs agree for this set of calibration samples. SEC includes contributions from spectrum measurement error, PTM measurement error, and model error. Sample (type) specific biases are a part of the model error. Typically, spectroscopic analyzers are very precise, so that spectral measurement error is small relative to the other types of error.
1.4.1.2During initial analyzer validation, the compositional range of available samples may be small relative to the range of the calibration set. Because of the high precision of the spectroscopic measurement, the average difference between the PTMRs and PPTMRs may reflect a sample (type) specific bias which is statistically observable, but which are less than the uncertainty of PPTMR, U(PPTMR). Therefore, the bias and precision of the PTMR/PPTMR differences are not used as the basis for local validation.
1.4.1.3Based on SEC, and the leverage statistic, the uncertainty of each PPTMR, U(PPTMR) is calculated. During validation, for each non-outlier sample, a determination is made as to whether the absolute difference between PPTMR and PTMR, |δ|, is less than or equal to U(PPTMR). Counts are maintained as to the total number of non-outlier validation samples, and the number of samples for which |δ| is less than or equal to U(PPTMR). Given the total number of non-outlier validation samples, an inverse binomial distribution is used to calculate the minimum number of results for which |δ| must be less than U(PPTMR). If the number of results for which |δ| is less than U(PPTMR) is greater than or equal to this minimum, then the results are consistent with the expectations of the multivariate model, and the analyzer passes local validation. The calculations involved are described in detail in Section 11 and Annex A4.
1.4.1.4The user must establish that results that are consistent with the expectations based on the multivariate model will be adequate for the intended application. A 95 % probability is recommended for the inverse binomial distribution calculation. The user may adjust this based on the criticality of the application. See Annex A4 for details.
1.4.2General Validation:
1.4.2.1When the validation samples are of sufficient number, and their compositional and property ranges are comparable to that of the model calibration set, then a General Validation can be done.
1.4.2.2General Validation is conducted by doing a D6708 based assessment between results from the analyzer system (or subsystem) produced by application of the multivariate model, (such results are herein referred to as PPTMRs), versus the PTMRs for the same sample set. The system (or subsystem) is considered to be validated if the D6708 meets the following condition:
(1)No bias correction can statistically improve the agreement between the PPTMRs versus the PTMRs, and
(2)Rxy computed as per D6708 meets user-specified requirements.
1.4.2.3For analyzers used in product release or product quality certification applications, the precision and bias requirement for the degree of agreement are typically based on the site or published precision of the PTM.
Note 1:In most applications of this type, the PTM is the specification-cited test method.
1.4.2.4This practice does not describe procedures for establishing precision and bias requirements for analyzer system applications. Such requirements must be based on the criticality of the results to the intended business application and on contractual and regulatory requirements. The user must establish precision and bias requirements prior to initiating the validation procedures described herein.
1.5This practice does not cover procedures for establishing the calibration model (correlation) used by the analyzer. Calibration procedures are covered in Practice D8321 and references therein.
1.6This practice is intended as a review for experienced persons. For novices, this practice will serve as an overview of techniques used to verify instrument performance, to verify model applicability to the spectrum of the sample under test, and to verify that the degree of agreement between PPTMRs and PTMRs meet user requirements.
1.7This practice specifies appropriate statistical tools, outlier detection methods, for determining whether the spectrum of the sample under test is a member of the population of spectra used for the analyzer calibration. The statistical tools are used to determine if the infrared measurement results in a valid property or parameter estimate.
1.8The outlier detection methods do not define criteria to determine whether the sample or the instrument is the cause of an outlier measurement. Thus, the operator who is measuring samples on a routine basis will find criteria to determine that a spectral measurement lies outside the calibration, but will not have specific information on the cause of the outlier. This practice does suggest methods by which instrument performance tests can be used to indicate if the outlier methods are responding to changes in the instrument response.
1.9This practice is not intended as a quantitative performance standard for the comparison of analyzers of different design.
1.10Although this practice deals primarily with validation of infrared and Raman analyzers, the procedures and statistical tests described herein are also applicable to other types of analyzers which employ multivariate models.
1.11This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of regulatory limitations prior to use.
1.12This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
ASTM D 7825 : 2018 | Standard Practice for Generating a Process Stream Property Value through Application of a Process Stream Analyzer |
ASTM D 8470 : 2022 | Standard Practice for Development and Implementation of Instrument Performance Tests for Use on Multivariate Online, At-Line and Laboratory Spectroscopic Based Analyzer Systems |
ASTM D 7278 : 2021 | Standard Guide for Prediction of Analyzer Sample System Lag Times |
ASTM E 1655 : 2017 | Standard Practices for Infrared Multivariate Quantitative Analysis |
ASTM D 8321 : 2022 | Standard Practice for Development and Validation of Multivariate Analyses for Use in Predicting Properties of Petroleum Products, Liquid Fuels, and Lubricants based on Spectroscopic Measurements |
ASTM D 6624 : 2020 | Standard Practice for Determining a Flow-Proportioned Average Property Value (FPAPV) for a Collected Batch of Process Stream Material Using Stream Analyzer Data |
ASTM C 1307 : 2021 | Standard Test Method for Plutonium Assay by Plutonium (III) Diode Array Spectrophotometry |
ASTM D 7453 : 2022 | Standard Practice for Sampling of Petroleum Products for Analysis by Process Stream Analyzers and for Process Stream Analyzer System Validation |
ASTM D 7165 : 2022 | Standard Practice for Gas Chromatograph Based On-line/At-line Analysis for Sulfur Content of Gaseous Fuels |
ASTM D 8340 : 2022 | Standard Practice for Performance-Based Qualification of Spectroscopic Analyzer Systems |
ASTM D 7164 : 2021 | Standard Practice for On-line/At-line Heating Value Determination of Gaseous Fuels by Gas Chromatography |
ASTM D 7314 : 2021 | Standard Practice for Determination of the Heating Value of Gaseous Fuels using Calorimetry and On-line/At-line Sampling |
ASTM D 7166 : 2023 | Standard Practice for Total Sulfur Analyzer Based On-line/At-line for Sulfur Content of Gaseous Fuels |
ASTM D 3764 : 2023 | Standard Practice for Validation of the Performance of Process Stream Analyzer Systems |
ASTM D 7808 : 2022 | Standard Practice for Determining the Site Precision of a Process Stream Analyzer on Process Stream Material |
ASTM D 4814 : 2023 : REV A | Standard Specification for Automotive Spark-Ignition Engine Fuel |
ASTM D 6621 : 2021 | Standard Practice for Performance Testing of Process Analyzers for Aromatic Hydrocarbon Materials |
ASTM E 2898 : 2020 : REV A | Standard Guide for Risk-Based Validation of Analytical Methods for PAT Applications |
ASTM D 7235 : 2021 : REV A | Standard Guide for Establishing a Linear Correlation Relationship Between Analyzer and Primary Test Method Results Using Relevant ASTM Standard Practices |
ASTM D 86 : 2023 | Standard Test Method for Distillation of Petroleum Products and Liquid Fuels at Atmospheric Pressure |
ASTM D 86 : 2023 : REV A : EDT 1 | Standard Test Method for Distillation of Petroleum Products and Liquid Fuels at Atmospheric Pressure |
ASTM E 456 : 2013 : REV A : R2022 | Standard Terminology Relating to Quality and Statistics |
ASTM E 456 : 2013 : REV A : R2022 : EDT 1 | Standard Terminology Relating to Quality and Statistics |
ASTM D 1265 : 2023 : REV A | Standard Practice for Sampling Liquefied Petroleum (LP) Gases, Manual Method |
ASTM D 2699 : 2024 : REV A | Standard Test Method for Research Octane Number of Spark-Ignition Engine Fuel |
ASTM D 2699 : 2024 | Standard Test Method for Research Octane Number of Spark-Ignition Engine Fuel |
ASTM D 1265 : 2023 | Standard Practice for Sampling Liquefied Petroleum (LP) Gases, Manual Method |
ASTM D 2699 : 2023 : REV B | Standard Test Method for Research Octane Number of Spark-Ignition Engine Fuel |
Access your standards online with a subscription
Features
-
Simple online access to standards, technical information and regulations.
-
Critical updates of standards and customisable alerts and notifications.
-
Multi-user online standards collection: secure, flexible and cost effective.