ARMA Algorithms

ARMA (Autoregressive-moving average) algorithms are used in the ARMA (AutoRegressive Moving Average) Spectrum option.

Lsmywe, MA

The simplest and fastest ARMA models are sequential algorithms. In a sequential linear procedure an AR model is first fitted to the data, and then an MA model is fitted to the residuals. The AR and MA coefficients are not simultaneously estimated and the overall solution will be suboptimal. Of this type of ARMA algorithm, the Lsmywe,MA procedure produces respectable peak frequency estimation and spurious peaks tend to be less a factor than with other algorithms.

The Lsmywe,MA option uses the least-squares modified Yule-Walker equations to estimate the AR parameters. The residuals from this fit are then modeled using the non-iterative Durbin MA algorithm. As ARMA procedures go, this algorithm is very fast. A stable filter is not assured, as this algorithm can produce roots outside the unit circle. For high AR and MA model orders, unstable filters are commonplace. The fit is appreciably suboptimal.

A lag term must be specified for this algorithm. In this procedure, information from the higher order autocorrelations is used. This term should be set to the highest value in the autocorrelation series that can be said to accurately represent the autocorrelation function. This is likely to be some fraction of the data length. The minimum lag is the sum of AR and MA orders plus one. The maximum lag is the number of data points minus one. This algorithm is very sensitive to the selected lag. You may want to try n/2 as a starting point.

Excellent coverage of ARMA spectral algorithms can be found in the following references:

· Steven M. Kay, "Modern Spectral Estimation", Prentice Hall, 1988, p.290-369.

· S. Lawrence Marple, Jr., "Digital Spectral Analysis with Applications", Prentice Hall, 1987, p.285-302.

A version of the Lsmywe,MA procedure can be found in the Kay (p.361) reference. AutoSignal's procedure follows this implementation.

NL

The four non-linear ARMA procedures consist of full iterative Levenburg-Marquardt minimizations. Unlike many ARMA implementations, the AutoSignal ARMA filter in the NL algorithms first proceeds toward the initial data element with backward prediction/averaging and then forward across the full data sequence. Both the ARMA model and a partial derivative for each parameter must be computed point by point at each iteration. The fitting process can be very slow with large data sets and high AR, MA model orders.

The NL algorithm imposes no constraints as parameters are allowed to vary freely. A non-linear update is furnished to report the progress of the fit as well as to offer user termination of fits that are slow to achieve the 1E-6 target convergence in the sum of squares merit function.

AutoSignal's ARMA NL algorithms use the non-linear curve-fitting technology implemented in TableCurve 2D. The four NL ARMA procedures are original algorithms authored for AutoSignal.

NL SF

The NL SF algorithm adds full spectral factorization to the problem. The ARMA filter will be minimum phase as both the AR and MA roots will lie within the unit circle. Although the unconstrained NL algorithm can sometimes offer a better goodness of fit, the NL procedure with spectral factorization is often close statistically.

Despite the overhead of the spectral factorization, the NL SF algorithm can sometimes be faster. Since a good measure of a non-linear ARMA fit involves parameters wandering about in regions of instability, spectral factorization can significantly improve fitting times. The resultant fit will usually have poorer goodness of fit statistics, however, as the true global least-squares minimum often has one or more roots outside the unit circle.

NL Svd and NL Svd SF

AutoSignal also offers the NL Svd and NL Svd SF versions of the two NL procedures. Just as in AutoSignal’s AR SVD options, a signal space is selected that should contain the principal singular values of the least-squares problem. It is true that one of the uses of ARMA models is to also characterize observation noise, and the SVD procedures are often used to discard contributions from the noise-bearing eigenvectors. Still, there are benefits to the SVD options. One obvious matter is that much of the fitting time is spent wandering about in n-dimensional space fitting weak noise components. By truncating the eigenspace at an appropriate singular value, very nearly the same goodness of fit can be achieved in a modest level of iterations. Also, deep nulls and sharp peaks are treated equally in the least-squares problem. A principal eigenmode may be associated with a null if this MA component significantly impacts the least-squares fit merit function.

ARMA Filter

In the ARMA procedure, The ARMA filter used in minimizing the least-squares merit function uses backward prediction/averaging from an initial data position down, and forward prediction/averaging from the model order up. The initial position for the backward prediction/averaging is the smaller of (1) the data count minus the AR model order and (2) the sum of the AR and MA model orders, or 100, whichever is larger. This approach conserves the degrees of freedom, as an estimate is made for each of the input data elements (there is no gap at one end of the data stream). The statistics reported for the ARMA procedure reflect this definition.

If the MA order is set to 0 and an AR-only fit is made, the non-linear parameter estimates will not exactly match the linear AR procedures. The AR filter is defined using backward prediction from the model order down, and forward prediction from the model order up. The statistics reported for the AR algorithms throughout AutoSignal reflect this approach, although none of the linear AR algorithms specifically optimizes this particular merit function. This means that the ARMA procedure’s results for an AR-only fit will in general show a slightly better goodness of fit than all of the AR linear least-squares fits, but only due to how this merit function is defined.