ARMA (AutoRegressive Moving Average) Spectrum

The ARMA (AutoRegressive Moving Average) Spectrum option in the Spectral menu or the Spectral toolbar produces a combined pole-zero model that is capable of effectively describing both peaks and nulls. An ARMA model is generally regarded as superior for fitting both signal and noise. Unfortunately, an ARMA model is non-linear in nature, and requires an iterative procedure to resolve the parameters. Unless performance considerations are critical, it is recommended that ARMA fits use the true non-linear algorithms.

Dealing with both AR and MA parameters, whose orders can be independently set, adds a considerably complexity to the modeling and to the spectral interpretation. To embrace this additional complexity and yet accept a suboptimal ARMA parameter set seems incongruous, and for this reason we have largely chosen not to support the various suboptimal procedures. There is no shortage of ARMA algorithms where the coefficients are generated via fast non-iterative procedures. In the course of AutoSignal's development, 24 different ARMA algorithms were coded into the product. Only five were retained.

Algorithm

The Lsmywe,MA option uses the least-squares modified Yule-Walker equations to estimate the AR parameters. The residuals from this fit are then modeled using the non-iterative Durbin MA algorithm. As ARMA procedures go, this sequential algorithm is very fast. A stable filter is not assured, as this algorithm can produce roots outside the unit circle. For high AR and MA model orders, unstable filters are commonplace. The fit is appreciably suboptimal. A Lag value must be specified for the Lsmywe,MA algorithm. In this procedure, information from the higher order autocorrelations is used. This lag term should be set to the highest value in the autocorrelation series that can be said to accurately represent the autocorrelation function. This is likely to be some fraction of the data length. The minimum lag is the sum of AR and MA orders plus one. The maximum lag is the number of data points minus one. This algorithm is very sensitive to the selected lag. You may want to use n/2 as a starting point.

The four NL ARMA procedures consist of full non-linear Levenburg-Marquardt minimizations. Unlike many ARMA implementations, the AutoSignal ARMA filter in the NL algorithms first proceeds toward the initial data element with backward prediction/averaging and then forward across the full data sequence. Both the ARMA model and a partial derivative for each parameter must be computed point by point at each iteration. The fitting process can be very slow with large data sets and high AR, MA model orders.

The NL algorithm imposes no constraints as parameters are allowed to vary freely. The NL SF algorithm adds full spectral factorization so that both the AR and MA roots will lie within the unit circle. Although the unconstrained NL algorithm can sometimes offer a better goodness of fit, the NL procedure with spectral factorization is often close statistically. Despite the overhead of the spectral factorization, the NL SF algorithm can sometimes be faster since a good measure of a non-linear ARMA fit involves parameters wandering about in regions of instability.

AutoSignal also offers the NL Svd and NL Svd SF versions of the two NL procedures. Just as in AutoSignal’s AR SVD options, a signal space is selected that should contain the principal singular values of the least-squares problem. While one of the uses of ARMA models is to also characterize observation noise, there are still benefits to truncating eigenmodes with SVD. If considerable fitting time is expended wandering about in n-dimensional space fitting weak noise components, faster fits can be achieved by discarding these eigenmodes. Also, deep nulls and sharp peaks are treated equally in the least-squares problem. A principal eigenmode may be associated with a null if this MA component significantly impacts the least-squares fit merit function.

Order

One of the greatest obstacles to ARMA fitting is determining the AR and MA orders. Selecting optimum AR and MA orders is difficult. There is no reason that these should be the same; that is, that there should be one spectral null (from an MA root) for each spectral peak (from an AR root). However, to simplify order selection, it is common practice to set the AR and MA orders equal to one another. Simply check the AR=MA box if you want to do this. Otherwise, you must independently fill in the values of the AR and MA order fields.

To see only an MA fit, or only an AR fit, the appropriate coefficient count can be set to zero. Since the non-linear spectral factorization algorithms fit stable ARMA models with all roots within the unit circle, AR-only fits are likewise constrained. Bear in mind, however, that the linear Data algorithms in the AR (AutoRegressive) Spectrum option often achieve this stability. You should use a spectrally factored non-linear ARMA fit only if the linear AR procedures fail to generate the desired stability in the parameters.

If an AR-only fit is made, note that the results will not exactly match any of the linear AR procedures. An AR filter is defined using backward prediction from the model order down, and forward prediction from the model order up. This preserves the degrees of freedom, as an estimate is made for each of the input data elements (there is no gap at one end of the data stream). The statistics in the AR algorithms throughout AutoSignal reflect this approach, although none of the linear AR algorithms specifically optimizes this particular merit function. This means that the ARMA procedure’s results for an AR-only fit will in general show a slightly better goodness of fit than all of the AR linear fits, but only due to how this merit function is defined.

Signal Subspace Selection

The Graphically Select Signal and Noise Sub-Spaces signal selection is enabled only when the NL Svd or NL Svd SF algorithm is selected. Just as in AutoSignal’s AR SVD options, a signal space is selected that should contain the principal singular values of the least-squares problem. Here though, the principal eigenmodes may be associated with both AR and MA components.

The SVD signal subspace selection option will display the singular values present at convergence (or when iterations are manually terminated). Unlike the AR options, selecting a different signal subspace does not result in an immediate update of the solution vector. Instead, the iterative fit is reinitiated using the new subspace.

Since nulls are being modeled in addition to peaks, the optimum signal space will not automatically be twice the number of narrowband components present in the data. Since considerable fitting can be spent wandering about in n-dimensional space fitting weak noise components, very nearly the same goodness of fit can be achieved in a modest level of iterations by truncating the eigenspace at an appropriate singular value. Since observation noise is managed to some extent within an ARMA fit, the singular value plot will not necessarily reveal a sharp transition between signal and noise nor is a noise subspace floor necessarily apparent. Instead, the singular values may range across many orders of magnitude.

A full signal space SVD fit, one where the signal space equals the sum of the AR and MA model orders, produces the same results as the non-SVD NL algorithms.

Spectrum

An ARMA spectrum can be generated directly from the AR and MA coefficients, or with some performance benefits using an FFT. The Full Range option locks the 0-0.5 Nyquist range. It also causes the spectrum to be generated via an FFT if the Adaptive option is disabled. When the Full Range option is on, only the total spectral count (n) can be specified. Unlike the FFT options, which specify the length of the transform, the ARMA option specifies the total frequency count in the output spectrum. An FFT of 16384 points produces 8193 spectral frequencies from 0 to 0.5 normalized frequency. For the Full Range option, it will be fastest if the values in the drop down box for n are used, since these produce power of 2 FFTs. The Best Exact n FFT procedure is used.

If the Full Range option is off, you can select the desired start and end frequencies as well as the count of spectral frequencies (n) in this band. It is thus possible to generate a detailed spectrum only in the region of specific interest. This option uses a direct computation for the spectrum and any size can be used.

The Adaptive option always uses a direct computation for the spectrum. An ARMA spectrum can consist of astonishingly sharp peaks and nulls, especially in comparison with traditional FFT spectra. For uniform sampling, a size of 8193 uniformly spaced points is not unreasonable in order to get good representation of the peaks and nulls. Even with a large n, it is possible to miss some fraction of the power of a peak. As an alternative, AutoSignal can use a Runge-Kutta procedure to integrate the spectrum adaptively, saving the points used in the computation of the integral. This results not only in an adaptive frequency set containing frequencies concentrated near the peaks, but also in an accurate area under the spectrum.

If the Adaptive option is used, it is possible to Normalize the spectrum so that its integrated power matches that of the input data. The Adaptive integration seeks 1E-5 fractional convergence.

The areas under the peaks are indicative of estimated power, and as such, this adaptive integration uses partitions formed by midpoints of the local maxima-determined peak positions. The Numeric Summary offers this numeric integration.

Plot

For ARMA spectra, there are only four formats. The PSD can reflect the three different power normalizations, or it can be expressed in dB. There is no normalized dB scale where the highest peak is set to 0 dB; sharp peaks are likely to be poorly characterized for height and they will not linearly reflect the power of spectral components.

This is not to say that ARMA spectra should be regarded only as frequency estimators, but it should be kept in mind that power values from peak integration may be quite limited in accuracy. This is true even when the adaptive integration achieves the target fractional error since the envelope of the peak is highly sensitive to ARMA modeling errors. For true harmonics, the linear least-squares sinusoidal fit in the Numeric Summary will almost certainly be more accurate, and the Non-Linear Optimization better still.

ARMA peak labels consist of frequencies only. They are toggled on and off with the Display Maxima button. The spectral peaks are identified by first generating a spectrum with a count of 8193 equally spaced frequencies. These peaks are then further refined using a one-dimensional minimization procedure with the continuous ARMA spectral model. The peak frequencies are estimated to 1E-15 precision. For the non-SVD algorithms, each local maximum in the 8193 frequency count spectrum is treated as a valid spectral peak. The spectral peak count can therefore be as high as half the model order. For the SVD procedures, the spectral peak count should be half the signal subspace value.

Add Noise

It may be instructive to see where a given procedure starts to break down as a consequence of temporarily adding white observation noise to the input data. The zero noise level is S/N=300dB (fractional noise=1E-15, the IEEE double precision threshold for addition). At this value, no noise is added to the data. A value of 280 would add noise in the 14th significant figure, 260 in the 13th, 240 in the 12th and so on. This option assumes that the current data set is entirely signal, and adds noise accordingly. Typical test values are 40dB(1% noise), 20(10%), 10(31.6%), 6(50.1%), 3(70.8%), and 0(100%).

List

The List Data option lists the index, frequency, and the spectral quantity currently plotted. The listing uses the AutoSignal text viewer facility.

Copy

The Copy Data to Clipboard option copies the frequency and the spectral quantity currently plotted to the clipboard. Formats include full precision binary (for spreadsheets such as Excel) and ASCII (for pasting into text editors). You can generally find a Paste As option in most applications if you want specific control over the format imported.

Save

The Save Data to Disk option writes the frequency and and the spectral quantity currently plotted to a supported file format. These formats include ASCII, Excel 97, Excel 95, Lotus WK3, Lotus WK1, SPSS, or Systat.

Production Facility

The AutoSignal Automation facility allows unattended processing of large numbers of data sets. The data sets can be consolidated in an Excel file or acquired using a DLL. The numeric summaries and graphs can be exported to a MS Word RTF file, while the extended data summaries or the current spectra can be exported to an Excel 95 or Excel 97 file.

Numeric Summary

The Numeric Summary offers a full ARMA spectral analysis report. The report optionally includes a listing of the coefficients, component powers by numeric integration, ARMA fit estimation details, and a linear sinusoidal least-squares fit summary.

Non-Linear Optimization

The Non-Linear Optimization offers the means to refine the parameter estimates given in the linear sinusoidal fit that is reported in the Numeric Summary. Constrained least-squares and robust (maximum likelihood) non-linear fitting is available with either sinusoid or damped sinusoid models.

Rich-Text Format Export

The Export Numeric Summary and Graph to RTF File option writes the numeric summary and spectral plot to an RTF file. The numeric portion of the file is based upon the current settings in the Numeric Summary option. The text data will be written to portrait orientation pages. The graph uses the current settings and size of the spectral plot, and is inserted as a Windows Metafile. The graph will always use a landscape orientation. Beyond a certain size, the graph will utilize a full landscape page.

View Residuals

Because a fitting occurs, the residuals can be inspected to see if they are normally distributed. The SNP plot is particularly useful.

Plot Roots

The poles and zeros of the ARMA model can be inspected with the Plot Roots option. Roots consisting of signal will rest on or close to the unit circle while those corresponding with noise will tend to be found in the interior of the unit circle.

Toggle Popup Information Window

Because the ARMA spectrum is a fitting procedure, a host of statistics are available to describe the ARMA model fit. The Toggle Popup Information Window is used to show or hide this information. The r² goodness of fit index may be particularly useful, since spectra that visually appear to be well fitted may be the result of a poor deterministic fit. A smooth ARMA spectrum is not an indicator of an accurate model fit. A high r² (0.95+) is not needed for good frequency estimation, but it is necessary for accuracy in the numerical integrations.

The statistical results reported are based on an ARMA filter that consists of forward prediction/averaging with partial backward prediction/averaging. This simplifies the goodness of fit statistics since a single estimate is made for each of the input data elements.

Local Options

A local option changes the data set for the duration of the current procedure only. The main data table is not altered. AutoSignal offers four local options in most of the spectral procedures.

Section the data to isolate specific regions for processing.

Detrend for removing mean or subtracting a least-squares trend model.

Fourier Filtration for isolating spectral components by frequency.

Eigendecomposition Filtration for isolating spectral components by signal strength.

The Reset button restores the data to its state when first entering the procedure. Note that if you implement sequential local procedures, all of the revisions are discarded upon reset. If an Automation Session is in progress, the Reset button can be used to terminate the automated processing.