USGS - science for a changing world

Hypoxia in the Gulf of Mexico

USGS Info on MRB Nutrients
Gulf of Mexico Hypoxic Zone
Hypoxia Task Force
Other Agency Info
USGS Publications

Report Site Map > Methods Used to Estimate Nutrient Fluxes

USGS Open-File Report 2007-1080 - Streamflow and Nutrient Fluxes of the Mississippi-Atchafalaya River Basin and Subbasins for the Period of Record Through 2005

Methods Used to Estimate Nutrient Fluxes


Stream mass flux, often referred to as load, is the mass of chemical constituents or sediment transported at a point in a stream during a period of time. Mass flux (phi = mass flux) is the product of constituent concentration (C) and discharge (Q) integrated over time (t):

phi = mass flux   =   C(t)Q(t)dt

Flux estimation using the integral in equation 1 requires a continuous record of concentration and discharge. Although discharge can be easily measured at a sufficiently high frequency, constituent concentration typically is measured less frequently due to the expense of collecting and analyzing water-quality samples. Therefore, concentration must be estimated between relatively infrequent samples. Several approaches have been developed to estimate concentrations continuously through time, and two approaches, the regression-model method and the composite method, are used herein for estimating fluxes.

Regression-Model Method

Methodology: The regression-model method, also known as the rating-curve method, is a standard statistical technique that can be used to estimate concentration continuously, thus enabling a direct calculation of mass flux. This method uses a regression model relating concentration to continuous variables such as discharge and day of year (for example, Johnson, 1979; Cohn and others, 1992). Fluxes were estimated using a seven-parameter regression model equation of the form:

ln(Li) = a0 + a1lnQ + a2lnQ2 + a3 sin(2πdtime) + a4 cos(2πdtime) + a5dtime + a6dtime2 + e (2)


ln is the natural logarithm (log base e);
Li is the calculated flux for sample i;
lnQ is ln(daily average streamflow) – center of ln(daily average streamflow);
dtime is decimal time minus the center of decimal time (as defined by Cohn and others, 1992);
e is error; and
a0...a6 are the fitted parameters in the multiple regression model.

Note that equation 2 is for flux (Li), not concentration, as sample concentrations have been already multiplied by the daily average discharge, but that this modification has no effect on the resulting flux estimates. Average daily discharge was used instead of the instantaneous discharge when the sample was collected because instantaneous discharge was not always available.

The flux for any period is estimated in a discrete manner using the model in equation 2 with a daily time step, which was then summed over the period of interest:

equation 3 (3)


LT is the total flux;
Li is the predicted flux for day i from equation 2;
n is the number of days;
Δt is the daily time step; and
Σ is a summation.

The average daily discharge was used to estimate Li. The use of a daily time-step should be adequate for the calculation of annual fluxes of large rivers because streamwater concentration and discharge do not change radically within a given day.

Flux Estimation Implementation: Fluxes were typically estimated for periods in which there were five or more consecutive years with at least four water-quality analyses available for each year. Exceptions to the four water-quality analyses per year rule were sometimes made to prevent a gap in a long-term continuous flux estimation period. Flux estimates were made across these potential gaps for cases where there were three or less consecutive years with three analyses per year, or a single year with two analyses available.

Fluxes were estimated with a regression-model method using Load Estimator (LOADEST), a FORTRAN-based load estimation program (Runkel and others, 2004). LOADEST estimates fluxes using various algorithms for different statistical distributions of the data and correction factors for back-transformation bias corrections from a log model back to linear space. Results from the adjusted maximum likelihood estimates (AMLE) are used in this analysis, which modifies equation 2 to correct for transformation bias. The AMLE approach also handles censored water-quality data (concentrations below the reporting limit) by inferring the censored sample concentrations from the statistical distribution of sample concentrations above the reporting limit. The LOADEST option for which LOADEST determines the optimal model form from a selection of nine model forms based on equation 2 (model forms include equation 2 and eight subsets of the parameters in equation 2) was employed for this analysis.

Model Calibration Procedure: The first 5 years of fluxes are estimated for each water-quality constituent by calibrating the regression model using the first 5 years of available water-quality data. Each subsequent year of flux estimates were calculated by calibrating the regression model using samples from the current year and the previous 4 years. This "moving window" approach allows a sufficient number of samples in each model run to represent the full range of flow and nutrient concentration conditions.

Flux Estimation Reporting: Nutrient fluxes are reported on annual (water year) and monthly time steps. Note that flux estimates on a monthly time-step can be quite inaccurate and should be used with caution. They are provided with the intent to allow the user to sum up fluxes on either a seasonal basis or an annual basis other than water year.

Flux Error Estimation: LOADEST estimates the standard error (SE), the standard error of the prediction (SEP), and the lower and upper 95 percent confidence intervals of the SEP. The SEP is preferable to the SE because it contains the effects of random error in addition to the error due to the model calibration (parameter uncertainty). Hence SEP is somewhat higher than the SE. SEP bounds are not equal above and below the mean. Therefore, the 95 percent confidence intervals better express the range in errors, and are the errors that are reported for this analysis. The error depends on the estimation period dataset. Therefore, error estimates need to be calculated for each estimation period of interest. LOADEST automatically estimates the errors on a monthly basis and for the entire estimation period. Errors for individual water years were estimated by running LOADEST with the same calibration dataset, but for individual water year estimation files.

Composite Method

Methodology: The composite method is a hybrid flux estimation approach that combines the regression-model method for predicting concentrations continuously with a period-weighted approach used to apply structure present in residual concentrations over time (residual concentration defined as the regression-model predicted concentration minus the observed concentration; Aulenbach and Hooper, 2006). In this manner, the composite method adjusts the regression model predicted concentration to the observed concentration on days when samples are collected, and applies the residuals to the fluxes between samples in a piece-wise linear fashion. This approach improves flux estimates when there is serial autocorrelation in the residual concentrations, which is an indication that there is unmodeled structure in the residuals that was not captured by the regression model. Serial autocorrelation in residual concentrations typically increases as sampling frequency increases. Assessments of the composite method indicate that when serial autocorrelation is about 0.2 or greater, the composite method is useful in improving fluxes. Serial autocorrelations of this level have been observed for sampling frequencies of monthly or better.

Flux Estimation Implementation: The composite method is employed using a modified version of LOADEST based on the AMLE estimates with a routine for period-weighting the residual concentrations. The composite method is employed for flux estimates for stations used to estimate the delivery to the Gulf of Mexico, which typically have the higher sampling frequency necessary for using the composite method. There is significant interest for higher frequency flux estimates at the mouth for investigation of the relation between nutrient delivery and the development of the hypoxic zone.

Model Calibration Procedure: The composite method uses the same 5-year moving window approach used for the regression-model method. The calibration datasets were modified slightly because the composite method requires a sample either on the first day or before the beginning of the estimation period, and a sample either on the last day or after the end of the estimation period. Hence, there are typically two additional samples included in the calibration dataset. Also, it was occasionally necessary to extrapolate sample concentrations at the beginning and end of the entire flux estimation period when there were no concentrations available within either 60-days before the beginning or 60-days after the end of the flux estimation period available for a particular water-quality constituent.

Flux Estimation Reporting: Composite method fluxes are reported on annual (water year) and monthly time steps, but only for water years that have at least 10 samples per year. This is about the sampling frequency in which the serial autocorrelation in the residual concentrations were generally high enough such that the composite method would improve the flux estimates. Although the serial autocorrelation in the residuals is a better measure for determining whether or not the composite method would improve the flux estimates, because the sampling can vary from year to year and the estimate of autocorrelation is for the entire 5-year calibration period, the number of samples was chosen as the default for determining the use of the composite method. Occasionally, negative daily fluxes were estimated from the LOADEST composite method routine. These fluxes were set to zero before summing fluxes by month or water year.

Flux Error Estimation: Error estimates cannot be calculated for the composite method. It can be assumed that the error estimates for composite method are similar to or better than the AMLE method because the composite method should generally be improving the flux estimates toward the observed water-quality concentrations. Occasionally, the composite method estimates were observed to fall outside the AMLE 95 percent confidence limits.

Specific Procedures for Computing Fluxes and 95 Percent Confidence Intervals for the Entire Mississippi-Atchafalaya River Basin

Mississippi River at St. Francisville: Nutrient fluxes estimated for the mainstem Mississippi River represent streamflow from Mississippi River at Tarbert Landing, Miss. and water quality from Mississippi River near St. Francisville, Louisiana. For modeling purposes, the at-site flow used for the Mississippi River models was a combination of flows from the Mississippi River at Tarbert Landing and Mississippi River flows diverted upstream of Tarbert Landing to the Atchafalaya River via the Old River Outflow Channel as determined from the station near Knox Landing, Louisiana; see monitoring network. (On average, 23 percent (based on water years 1968 through 2005) of the Mississippi River flow was diverted via the Old River Outflow Channel.) This combined flow was used in the flux models because it represents the natural flow associated with the variations in nutrient concentrations.

The model structure in equation 2 was modified to include additional flow terms (both flow and flow-squared terms) from two upstream stations (Mississippi River at Thebes and Ohio River at Metropolis; see monitoring network). Upstream flows were lagged 10 days to account for travel time between the streamflow stations and the sampling station. These upstream flow terms improved model predictions because of the different nutrient transport characteristics that exist between the Ohio River and the combined Missouri and Upper Mississippi River Basins. The LOADEST feature which chooses the best model from nine different model forms could not be implemented for this site because of the custom model form used.

The portion of the flux transported in the Mississippi River at Tarbert Landing to the Gulf of Mexico is determined by multiplying the fluxes estimated using the combined at-site flow by the portion of total Mississippi River flow at Tarbert Landing. This partitioning was done on a daily basis, as the percentage of flow varied daily. Note that the flow and fluxes diverted by the Old River Diversion are included in the Atchafalaya River at Melville, Louisiana, flux estimates.

Error estimates for fluxes from Mississippi River at St. Francisville are not exact because of the way the flux estimates are done for this site. AMLE flux 95 percent confidence intervals were estimated on a monthly and water year basis. The confidence intervals could not be scaled by the portion of flow on a daily basis; as was done for the daily fluxes. Therefore, the confidence intervals had to be scaled by the portion of total Mississippi River flow at Tarbert Landing on a monthly and water year basis. But since errors cannot be estimated on a daily basis and then somehow combined to a larger time period, errors have to be estimated at monthly and annual time steps and partioned by flow at these larger time steps. This is not ideal, but should be adequate.

Atchafalaya River at Melville: Nutrient fluxes were estimated from flow at the Atchafalaya River at Simmesport, Louisiana, and from water quality at the Atchafalaya River at Melville, Louisiana. The estimates include the fluxes from the Red River and the Old River Outflow Channel as well. Water-quality data was not available as early on as for the Mississippi at St. Francisville for most constituents. Hence, flux estimates for individual water-quality constituents generally have a later start date for the Atchafalaya River than for the Mississippi River.

Procedures for Computing Aggregate Fluxes and Associated 95 Percent Confidence Intervals: The fluxes for the entire Mississippi-Atchafalaya River Basin are the sum of the individual fluxes from the Mississippi River at St. Francisville and the Atchafalaya River at Melville. Fluxes were reported for years or months for which flux estimates from both Mississippi and Atchafalaya Rivers were available. The 95 percent confidence intervals for the entire Mississippi-Atchafalya River Basin is calculated by adding the size of the confidence interval above or below the flux estimate of each of the two fluxes in quadrature (that is, squared, added, and then square rooted; Kirchner, 2001).

This approach assumes that the flux estimates from the Mississippi and Atchafalaya Rivers are independent of each other (uncorrelated). It is logical to assume that the flux estimates are correlated due to similarities in flow and season for these adjacent basins. But the flux estimates are functions of both streamflow and season, resulting in errors in flux that are orthogonal to flow and season such that the errors in the fluxes are uncorrelated with flow and season. Only factors that are not included in the flux models can represent sources of potentially correlated error between station flux estimates. Therefore, it makes sense to treat the flux estimates as independent (Greg Schwarz, U.S. Geological Survey, written commun., February 4, 2007).


Aulenbach, B.T., and Hooper, R.P., 2006, The composite method: an improved method for stream-water solute load estimation: Hydrological Processes, v. 20, p. 3029–3047.

Cohn, T.A., Caulder, D.L., Gilroy, E.J., Zynjuk, L.D., Summers, R.M., 1992, The validity of a simple statistical model for estimating fluvial constituent loads—An empirical study involving nutrient loads entering Chesapeake Bay: Water Resources Research, v. 28, no. 9, p. 2353–2363.

Johnson, A.H., 1979, Estimating solute transport in streams from grab samples: Water Resources Research, v. 15, no. 5, p. 1224–1228.

Kirchner, James. 2001. Data analysis toolkit #5—Uncertainty analysis and error propagation: Analysis of Environmental Data Course, University of California, Berkely, Ca., 8 p.

Runkel, R.L., Crawford, C.G., and Cohn, T.A., 2004, Load estimator (LOADEST)—A FORTRAN program for estimating constituent loads in streams and rivers: U.S. Geological Survey Techniques and Methods, book 4, chap. A5, 69 p.

Report Site Map > Methods Used to Estimate Nutrient Fluxes

Accessibility FOIA Privacy Policies and Notices

Take Pride in America logo logo U.S. Department of the Interior | U.S. Geological Survey
Page Contact Information: Webmaster
Page Last Modified: Thursday, 10-Jan-2013 17:13:10 EST