Performance Verification Statement for NOC Phosphate Analyzer.

Abstract

The Alliance for Coastal Technology (ACT) conducted a sensor verification study of in situ nutrient analyzers during 2016 to characterize performance measures of accuracy, precision and reliability. The verification including a week of laboratory testing along with three moored field deployments in freshwater, estuarine, and oceanic coastal environments. Laboratory tests of accuracy, precision, and range were conducted at the University of Maryland’s Chesapeake Biological Laboratory (CBL) in Solomons, MD. A series of five tests were conducted to evaluate performance under controlled challenge conditions including: concentration range, temperature, salinity, turbidity, and dissolved organic carbon. All laboratory tests were conducted in 250 L polypropylene tank using RO water as the initial matrix, within a temperature controlled room. Instruments sampled from a common, well-mixed, test tank maintained at a documented level of known challenge condition. Instruments were set-up by the manufacturer daily prior to the start of each individual laboratory test, exposed to each test condition for a period of three hours, and programmed to sample at a minimum frequency of 30 minutes. Reference samples were collected every 30 minutes for five timepoints during corresponding instrument sampling times for each test. For the laboratory concentration range challenge the absolute difference between the NOCPO4 and reference measurement across all timepoints for trials C0 – C5 ranged from -0.0153 to 0.0025 mgP/L, with a mean of -0.0027 ±0.0043 mgP/L. There was a small but significant increase in the measurement difference with increasing concentration as determined by linear regression (p=0.008; r2=0.27). However, the change in accuracy mostly occurred at the highest test concentration (0.406 mgP/L) with measurement difference of -0.0103 mgP/L. An assessment of precision was performed by computing the standard deviations and coefficients of variation of the five replicate measurements for C1 – C5 concentration trials. The standard deviation of the mean ranged from 0.0002 to 0.0050 mgP/L across the five trials, and the coefficient of variation ranged from 1.25 to 5.25 percent. For the laboratory temperature challenge with testing at 5 oC, the absolute difference between instrument and reference measurement across all timepoints for trials C2 – C4 ranged from -0.0095 to -0.0004 mgP/L, with a mean of -0.0045 ±0.0031 mgP/L. There was no significant difference in measurement accuracy at the C2 concentration level. However, measurement differences were significantly more negative (under-predicted) for the C3 and C4 concentration trials at 5 oC then at 20 oC, with offsets of -0.0050 and -0.0071, respectively. For the laboratory salinity challenge performed at the C3 concentration level, the absolute difference between instrument and reference measurement across all timepoints for the three added salinity levels ranged from -0.0021 to 0.0125 mgP/L, with a mean of 0.004 ±0.0051 mgP/L. There was no statistically significant response between measurement accuracy and salinity across all trials (p=0.32; r2=.08). For the laboratory turbidity challenge, performed at the C3 concentration level, the absolute difference between instrument and reference measurement across all timepoints for the two added turbidity levels ranged from -0.0033 to 0.0014 mgP/L, with a mean of -0.0007 ±0.0017 mgP/L. A linear regression of the measurement differences versus turbidity was not significant (p<0.12; r2=0.20). For the laboratory DOC challenge, performed at the C3 concentration level, the absolute difference between instrument and reference measurement across all timepoints for the two added DOC levels ranged from -0.0006 to 0.0098 mgP/L, with a mean of 0.0015 ±0.0034 mgP/L. A linear regression of the measurement differences versus DOC concentration was barely non-significant (p=0.056; r2=0.27). Measurement offset was 0.004 mgP/L more positive at 10 versus 1 mgC/L. A 32 day field deployment occurred from May 26 through June 27 in the Maumee River, at the facilities of the Bowling Green, Ohio Water Treatment Plant. The NOC-PO4 operatedsuccessfully during the entire 32 day deployment sampling at hourly intervals, but lost 12 days of data between 5/27 – 6/8 due to a problem writing results to the SD memory card. The NOC-PO4 generated 461 observations out of a possible 763 for a data completion result of 60.4%. The average and standard deviation of the measurement difference between instrument and reference PO4 measurements for each matched pair (n=28 of a possible 51 observations) over the total deployment was 0.034 ± 0.024 mgP/L with a total range of -0.033 to 0.079 mgP/L. There was a small but significant trend in measurement difference over time as estimated by linear regression (p= 0.03; r2=0.17) with a slope of 0.001 mgP/L/d. A linear regression of instrument versus reference measurement was highly significant (p<0.0001; r2 = 0.49) but with a slope of only 0.65 and intercept of 0.045. The NOC-PO4 handled the measurement range equally well, but was generally over-predicting concentrations as noted by the positive intercept of 0.045 mgP/L. An 84 day moored field test was conducted in Chesapeake Bay from July 18 to October 10, 2016. The NOC-PO4 operated continuously for the first 8 days of the deployment sampling at hourly intervals but stopped reporting on 7/31 when it appears to have fallen off the mooring when an attachment bolt was corroded away. A new instrument was deployed on 9/16 and operated until the end of the deployment reporting 874 of a possible 883 accepted values for a data completion result of 99%, but this represented only 43% of the total possible record. During the second unit’s operation, 9 values were flagged by the instrument as bad data. The average and standard deviation of the measurement difference between instrument and reference PO4 measurements for each matched pair (n=48 of a possible 103 observations) over the total deployment was 0.006 ±0.005 mgP/L, with the total range of differences between -0.003 to 0.015 mgP/L. There was a similar range of measurement offset during the two deployment periods; however the sharp rise in instrument values and offset during the initial 8 days may have indicated somethings was malfunctioning within the instrument, leading to the corrosion problem. A linear regression of NOC-PO4 versus reference measurements was highly significant (p<0001; r2 = 0.743), with a slope of 0.933 and intercept of 0.006. NOC-PO4 covered the range equally well, but in general over-predicted concentrations. A one month long moored field test was conducted in Kaneohe Bay from October 3, 2016 to November 2, 2016. The NOC-PO4 operated successfully for the entire 30 days of the deployment, sampling at hourly intervals returning 718 of a possible 720 measurements for a data completion result of 99.7%. The average and standard deviation of the differences between instrument and reference readings over the entire deployment (n=73 out of a possible 73) was 0.0014 ± 0.0009 mgP/L, with a total range in the differences of -0.0034 to -0.0001 mgP/L. There was a small but statistically significant trend in the measurement difference over time (p=0.0001; r2 = 0.233) during the deployment, with a slope of -0.00004 mgP/L/d. A linear regression of instrument versus reference measurements was significant (p=0.014; r2 = 0.103), but with a slope of only 0.149 and intercept of 0.002. The NOC-PO4 under-predicted throughout the measurement range and was marginally responsive to concentrations above 0.004 mgP/L.