The importance placed on fuel quality in the biofuels industry is intended to assure producers, distributors and end users that biodiesel will meet the expectation of trouble-free performance. The advocacy for a standard measure of quality increases the industry's protection against the risk of introducing unacceptable product into the market. Actualization of this risk management philosophy reduces the potential of costly liability assigned to those responsible for production and distribution.

Significant effort by committee members of ASTM International provides the basis for our confidence. Committee members work to develop effective standard methods legally recognized as the universal measure for product quality. Standard methods, such as those specified in ASTM D 6751 Standard Specification for Biodiesel Fuel Blend Stock (B100) for Middle Distillate Fuels, are utilized by scientists and laboratory staff to direct a reproducible, systematic approach to the instrumental analysis of biodiesel. The potential for erroneous analytical determinations, however, is significant when considering the challenge presented by systematic and random error inherent in the analytical process.

Representative samples, calibrated instrumentation, correct sample preparation, proper data interpretation and accurate reporting must successfully merge to achieve the intent of standard methods. Experienced instrumental analysts therefore invest considerable time and effort to perform quality control tests that offer quality assurance.

For perspective, a professor colleague once shared the fundamental question, "How do you know that your instrument is telling the truth?" Perhaps the more poignant question is how does a producer, distributor or consumer of biodiesel know the analytical results stated as a declaration of quality on a certificate of analysis are correct?

Standard methods are only effective if those responsible for conducting the analysis perform the analysis well. The result must be accurate. Particularly, if the tests to confirm fuel quality are performed well and accurate results are generated, our confidence increases.

How does one know if the test is performed well? How do producers, distributors or consumers know if laboratories are generating and propagating accurate results? Professional laboratories that value the need to produce credible analytical results accept the challenge to voluntarily participate in the ASTM Interlaboratory Crosscheck Program. The ILCP tests a laboratory's proficiency for a particular analytical capability. An identical sample is provided to each participating laboratory and the test results are subsequently collated to statistically assess each participant's performance.

Is the laboratory you trust routinely participating in the ASTM ILCP performance assessment? If so, how do you know if the laboratory is doing well? Fortunately, there is a useful indicator that can be evaluated to assess analytical performance.

Z-Score: A Laboratory Performance Index
As with the need to assess performance in other industries, a statistical index or performance indicator exists to assess laboratory performance such as the Dow index used for financial performance or the consumer price index used to measure inflation. This laboratory performance index is known as the z-score and is computed for each test result submitted by a laboratory participating in an ASTM ILCP. The z-score enables a reviewer to consistently evaluate and compare the analytical performance of participating laboratories.

To calculate this index, the ASTM committee responsible for administering the ILCP collects, analyzes, interprets and presents the data of those who participate in the proficiency test. During this process, a robust statistical evaluation of the data set is performed to identify outliers and to calculate a robust mean value and robust standard deviation. An explanation of robust statistics is beyond the scope of this article. However, the calculations and theory of robust statistics can be better understood through review of the journal article titled "Robust Statistics-How Not to Reject Outliers," by the Analytical Methods Committee of the Royal Society of Chemistry in the December 1989 issue of Analyst.

The z-score index is the ratio of the laboratory's deviation from the robust mean value of the data set over the standard deviation within the data set. The closer the z-score value is to zero the greater the confidence that the laboratory can determine the correct result. A z-score of zero indicates there is no deviation between the statistical true value and the laboratory's result and validates that the laboratory is capable of performing the test well enough to generate accurate results.

A z-score greater than two or less than negative two should cause a laboratory to review their test data for any possible systematic error. Z-scores outside this range should occur only one time in 20 if a laboratory has average capability running the method. A z-score calculation equal to or greater than the range plus or minus three is rejected. Z-scores within the range plus or minus 1 demonstrate exceptional performance. Laboratories should strive to obtain z-score values close to zero.

For an example calculation, consider a data set of 54 results for the measurement of flash point by standard method ASTM D 93, whereby the calculated robust mean regarded as the true value is equal to 157.12 and the robust standard deviation is equal to 3.83. If a laboratory's independent result is equal to 155.5, their deviation from the mean is equal to minus-1.62 (i.e. 155.5 minus 157.12 equals minus-1.62). The z-score for this laboratory's performance is therefore calculated by dividing its deviation from the mean by the robust standard deviation (i.e., minus-1.62 divided by 3.83 equals minus-0.4). A z-score of minus-0.4 is within the range plus or minus 1 and would be regarded as a desirable outcome.

Improving the Z-Score
A z-score close to zero reflects a successful quality assurance program and demonstrates the ability to report accurate results. Accuracy is a measure of precision and bias. To improve or maintain accuracy and subsequently improve the z-score, a laboratory must sustain systems for the evaluation of precision and bias in their analytical technique.

To monitor precision, particularly the ability to replicate a measured value, replicate samples are analyzed on a routine basis according to the standard method. The agreement between the replicate results is calculated and expressed as either a percent difference in the case of duplicate analyses or as a relative standard deviation in the case of three or more replicate analyses.

To monitor for bias, a standard reference material such as a National Institute of Standards and Technology traceable standard with a certified known value is measured to determine if the correct value can be achieved. The agreement between the measured value and the certified known value is also expressed as a percent difference or as a percent recovery. The analysis of a standard reference material will reveal the possibility of a systematic error that shifts the measured value away from the certified known value. Additional performance of these tests occasionally using a second source of similar standard reference material improves the likelihood that systematic bias, if present, is discovered and addressed.

Maintaining quality control charts for precision and bias data enable analysts to discover performance perturbations that if not otherwise addressed pose the risk of reporting incorrect values. These observation techniques and activities help to fulfill requirements for quality. When combined with participation in the ASTM ILCP, the fundamentals of a quality assurance program are practiced.

If those engaged in the industry become familiar with indices used to assess quality performance, such as the z-score, the efficacy of a risk management philosophy rooted in the advocacy of a standard measure of quality is achieved. If we work as partners to add the discussion of z-score to our lexicon, we demonstrate an understanding of what quality can look like. The success of our partnership to promote accurate analytical work builds confidence for others.

Ultimately, a laboratory's proficiency can be assessed by checking their z-score. Currently, the ASTM committee administers the ILCP three times per year for the biodiesel industry. If you are assessing your own laboratory's performance, seeking independent third-party analytical support, or measuring confidence in existing certificates of analysis, ask to check the index. Check the Z.

Jeff Fetkenhour is the president and founder of Gorge Analytical LLC. Reach him at gorgeanalytical@embarqmail.com or (541) 980-7168.