Detrend

The Detrend option in the Time menu or the Time toolbar is used to fit a trend to the overall data and remove it. There are 8 parametric models which can be fitted with linear least squares or by one of two non-linear robust options that should be less influenced by the signal components. In addition to subtracting the fit, this option can zero the mean and/or normalize to unit standard deviation.

This option presents an AutoSignal graph displaying the input and detrended data.

Options

When the Subtract Fit item is checked, the first step in the detrending will be to fit a trend to the data and then to subtract this model from the data. If Subtract Mean is checked, the next step is to subtract the mean of the data. Finally, if Divide by SD is checked, the each data element will be divided by the standard deviation. Standard anomalies (zero mean and unit standard deviation) are a common statistical time series normalization.

Model

The trend model selection will be available only if the Subtract Fit item is checked. The models are as follows:

· Constant (y=a0)

· Linear (y=a0+a1*x)

· Quadratic (y=a0+a1*x+a2*x^2)

· Cubic (y=a0+a1*x+a2*x^2+a3*x^3)

· Logarithmic (y=a0+a1*ln(x))

· Exponential (y=a0+a1*exp(-x/a2))

· Power (y=a0+a1*x^a2)

· Hyperbolic (y=a0 +a1*a2/(a2+x))

The exponential, power, and hyperbolic baselines are non-linear models and are fitted iteratively using the same Levenburg-Marquardt algorithm used in ARMA fitting. The constant, linear, quadratic, cubic, and logarithmic baselines are linear and are fitted in a single step matrix solution when a least-squares minimization is used. When a robust maximum likelihood minimization is made, all of the models must be fit non-linearly.

When a model is fitted non-linearly, it is possible that the true global minimum will not be achieved in the optimization. This is particularly a problem when the trend in the data is subtle and not readily apparent visually. Also, with oscillatory data there can be problems when using a model such as a cubic which can fit some portion of a low frequency harmonic.

Minimization

Least Squares is typically used. If there are significant outliers or a wide dynamic range in Y values, a robust minimization may be of value. A maximum likelihood minimization is more likely to represent the true data trend when outliers are present or when the Y-values span several orders of magnitude or more.

The Least Abs. Dev. (least absolute deviation) option will offer a medium level of immunity toward outliers. The Lorentzian minimization offers a high immunity from being adversely affected by outliers and non-Gaussian error distributions.

Because a robust minimization is non-linear, there is a chance of fitting to a local rather than a global minimum with any of the models.

To assess the merits of any given trend fit, there are three tools that may prove useful. The most obvious is to use a visual inspection of the data and trend model to assess whether the fit makes sense. The second item consists of goodness of fit statistics in the chiČ, rČ, SE, and Fstat informational fields. This should allow a fast comparison between the different models.

Numeric Summary

The Numeric Summary option offers a much more detailed fit analysis. This will include the goodness of fit statistics and coefficient values for all eight of the models.

List

The List Data option lists the index, x-values, and the detrended data. The listing uses the AutoSignal text viewer facility.

Copy

The Copy Data to Clipboard option copies the x values and detrended data to the clipboard. Formats include full precision binary (for spreadsheets such as Excel) and ASCII (for pasting into text editors).

Save

The Save Data to Disk option writes the x-values and detrended data to a supported file format. These formats include ASCII, Excel 97, Excel 95, Lotus WK3, Lotus WK1, SPSS, or Systat.

Production Facility

The AutoSignal Automation facility allows unattended processing of large numbers of data sets. The data sets can be consolidated in an Excel file or acquired using a DLL. The numeric summaries and graphs can be exported to a MS Word RTF file, while the detrended data can be exported to an Excel 95 or Excel 97 file.

Rich-Text Format Export

The Export Numeric Summary and Graph to RTF File option writes the numeric summary and the plot to an RTF file. The text data will be written to portrait orientation pages. The graph uses the current settings and size of the plot, and is inserted as a Windows Metafile. The graph always uses a landscape orientation. Beyond a certain size, the graph utilizes a full landscape page.

If an Automation Session is in progress, the Reset button can be used to terminate the automated processing.

When exiting this procedure with the OK button, an option will be presented to update AutoSignal's main data table with the detrended data.