AR and EigenAnalysis Spectra

This tutorial covers the spectral analysis capabilities of AutoSignal that rely on autoregressive parametric modeling and eigenanalysis procedures. These advanced procedures are particularly important for frequency estimation when data records are very short. Regardless of the data record size, these procedures generally offer the highest frequency estimation accuracy.

Generating a Test Signal

Select the Generate Signal option in the Edit menu or Main toolbar.

For this tutorial, we will create a challenging data set that contains three closely spaced sinusoids. The data record is only 50 points long. One percent random Gaussian noise is added.

Click Read and select the file tutor4.sig from the Signals subdirectory.

The following signal expression is imported:

SRATE=5000

NYQ=SRATE/2

AMP1=1

AMP2=SQRT(2)/2

AMP3=1

FREQ1=NYQ*0.2

FREQ2=NYQ*0.225

FREQ3=NYQ*0.25

PHASE1=PI/2

PHASE2=PI

PHASE3=3*PI/2

F1=AMP1*SIN(2*PI*X*FREQ1+PHASE1)

F2=AMP2*SIN(2*PI*X*FREQ2+PHASE2)

F3=AMP3*SIN(2*PI*X*FREQ3+PHASE3)

Y=F1+F2+F3

The X (time) values vary from 0 to 0.0098 with a 0.0002 sample increment. The Nyquist frequency is thus 2500 (half the 5000 sampling rate). The first component is at frequency 500, the second is at frequency 562.5, and the third is at frequency 625. The three components use only 5% of the Nyquist band. Further, the middle component is half the power of the other components so that it can be masked by the two adjacent peaks of greater power. The object of this tutorial is to highlight those procedures that can effectively identify the presence of the three components as well as to accurately estimate the frequencies.

Click OK to process the current signal.

An AutoSignal graph is presented containing the generated data.

Click OK to accept the generated data. Click Yes when asked to update the main data table with the revised data.

AutoRegressive Spectrum

The first type of algorithm uses AR (autoregressive) parametric modeling. An AR spectrum models a given data value by a linear combination of prior and/or subsequent data values. The AR coefficients are used to create a continuous parametric spectrum that is capable of exceptional frequency resolution.

For virtually all of the methods in this tutorial, the SVD (singular value decomposition) methods offer the best component isolation and frequency estimation. This is because an eigendecomposition that removes most of the noise influence is intrinsic to the coefficient computations.

Select the AR (AutoRegressive) Spectrum option in the Spectral menu or toolbar. Select the AutoCorr algorithm and set the order to 25. Be sure the Full Range and Adaptive n options are checked and that the dB plot format is selected.

This algorithm's spectrum is based on the estimated autocorrelation series. This is the AR method in most statistical packages. It is unable to resolve the three components. The frequency estimates are nowhere close.

Select the Burg algorithm.

The Burg procedure is probably the most widely used AR algorithm. It is sometimes known as the MEM or Maximum Entropy spectral procedure. The procedure computes the AR coefficients directly from the data by estimating the reflection coefficients (partial autocorrelations) at successive orders. It is also unable to resolve the three components.

Select the Data FB algorithm.

This algorithm is also known as the modified covariance method. The coefficients are determined by a least squares procedure whose data matrix consists of both forward and backward predictions. This procedure is successful in resolving the three spectral components. The frequency estimates are also good.

Select the Data Svd FB algorithm. Be sure the Signal Subspace is set to 6.

The three harmonic components are now cleanly isolated and the frequencies are very accurately estimated. The SVD procedure uses an eigendecomposition to discard the noise eigenmodes when the AR coefficients are computed. This in-situ noise filtration results in superb frequency estimation. This algorithm is similar to the PCAR (Principal Component AutoRegressive) method.

Optimum AR order selection for the non-SVD algorithms is normally a difficult task. It is generally much simpler to fit a sufficiently high model order and use SVD to truncate the eigenmodes at a signal-noise threshold.

Click the Graphically Select Signal and Noise Subspaces button.

Two eigenmodes are needed to capture a single oscillatory component. The first four eigenmodes represent the two higher power spectral peaks. The fifth and sixth eigenmodes represent the lower power spectral peak. The remaining eigenmodes capture the noise within the signal. By selecting the sixth eigenmode as the last element of signal space, the noise component is effectively removed from the analysis. If the model order is high enough to enable proper partitioning of signal and noise, and provided the signal elements of interest have sufficient signal strength, the signal subspace should be readily identified graphically.

The singular value plot not only suggests three oscillatory components, but also indicates that two are of approximately equal power while the third is substantially lower in power.

Even when the number of components is known, it is good practice to confirm a sufficient model order by inspecting the singular value plot.

Click on the second eigenmode in the plot and then click OK.

All of the spectral power is shown at the middle of the three frequencies.

Click the Graphically Select Signal and Noise Subspaces button, click on the fourth eigenmode in the plot, and click OK.

Note that the two spectral peaks are not identified at the locations of the two higher power components. It is thus important that the signal subspace be specified correctly.

Set the Signal Subspace value back to 6.

Click on the Numeric Summary button. Be sure the Add Power [Numeric Integration], Add Estimation Details, and Add Sine Component Linear Fit Summary items in the Format menu are checked. Note these three sections.

Power [Numerical Integration-Frequency Domain]

   Frequency       Power       %

  500.20264700      0.0001045112      36.369004257

  560.35852376      7.922194e-05      27.568556411

  625.87851653      0.0001031587      35.898342157

The TISA power values come from a numeric integration of the AR frequency spectrum. An AR spectrum's peaks are not linearly proportional to power. Power estimates are computed along with the adaptive spectrum. An adaptive spectrum catches the sharp AR spectral peaks with a modest number of spectral frequencies. Note that the power estimates are approximations only since the partitions are defined by the midpoints between the frequencies determined by the AR polynomial's roots.

AR Fit Details

   rē       StdErr       F-stat       SSE/n      

  0.9997350332      0.0154958228      3773.0582985      0.0001200603      

   AIC       MDL       Pwr,Data       Pwr,Spec       %

  -401.3758377      -353.5752626      0.0045826854      0.0002873634      6.2706335166

Note the excellent goodness of fit statistics. Also note the difference between the power in the data and the integrated power from the AR spectrum. The power values are highly susceptible to errors in the computation of the driving white noise variance that is part of the AR model. When the Normalize option is checked, the spectral magnitudes are adjusted so that power is conserved.

Sine Component Fit (Suboptimal)

   Frequency       Amplitude       Phase       Power       %       Rel %

  500.202647      1.01401210      1.56386623      0.00496191      41.3607250      100.000000

  560.358524      0.69036199      3.19124861      0.00233894      19.4965819      47.1379113

  625.878517      0.97237467      4.68896054      0.00469582      39.1426931      94.6373476

                    0.01199668      100.000000      

   r2       DOF r2       Std Err       F-stat

  0.9998811820      0.9998544480      0.0081028548      43128.087138

   Data Power       Model Power       Error Power       Ratio

  0.0045311443      0.0045821470      5.383813e-07      8510.9699328

In this section, the frequencies from the AR spectrum are used in a linear least-squares fit to the time domain data that resolves the amplitudes and phases. Note that the components are recovered with a high measure of accuracy. The least-squares goodness of fit values are even better than those in the AR model fit. Further, the sinusoidal model achieves this with 9 parameters rather than 25.

Close the Numeric Summary window.

Note that Non-Linear Optimization can be used to further refine the frequency estimates. This will be done near the conclusion of this tutorial.

Click the Plot Roots option.

When the complex roots from the AR model are plotted, those associated with signal should be on or near the unit circle while those associated with noise should be in the interior. In this case, the six roots associated with signal are readily apparent. For real data, the roots are symmetric about the imaginary axis. There is thus a negative frequency peak for each positive frequency peak, and a total of three harmonic components that correspond with the six signal roots.

One pair of roots is slightly outside the unit circle. If the AR coefficients were being used in an FIR filter rather than in a spectral estimator, some form of spectral stabilization would normally be employed.

Note that the ARMA (AutoRegressive Moving Average) Spectrum option can be used with the MA order set to zero in order to produce non-linear spectrally-factored AR model fits.

An ARMA model with both AR and MA coefficients can effectively fit data whose frequency spectrum consists of both peaks and nulls. This is useful for fitting noise, although better spectral results are generally achieved by discarding the noise component.

Click OK to close Roots window and OK again to close the AR Spectrum procedure.

EigenAnalysis Spectra

AutoSignal offers a number of eigendecomposition based spectral procedures for highly accurate frequency estimation. These are non-parametric procedures whose spectrum utilizes the noise eigenmodes from an eigendecomposition of the data. Two widely respected procedures are the MUSIC (Multiple Signal Classification) and Eigenvector algorithms.

Select the EigenAnalysis Spectrum option from the Spectral menu or toolbar. Select the MUSIC FB algorithm, set the order to 25, and set the Signal Subspace to 6. Be sure the Full Range and Adaptive n options are checked and that the dB(Spec) plot format is selected.

The MUSIC algorithm successfully resolves the three components and offers very good frequency estimates.

Click the Numeric Summary option and inspect the linear sinusoidal fit.

Sine Component Fit (Suboptimal)

   Frequency       Amplitude       Phase       Power       %       Rel %

  499.963254      1.01537173      1.57124593      0.00497360      42.1831065      100.000000

  559.343899      0.67668651      3.21997245      0.00225125      19.0937764      45.2640357

  626.291791      0.95871906      4.68046156      0.00456565      38.7231171      91.7976894

                    0.01179051      100.000000      

   r2       DOF r2       Std Err       F-stat

  0.9998718476      0.9998430133      0.0084151229      39986.309790

   Data Power       Model Power       Error Power       Ratio

  0.0045311443      0.0045821047      5.806772e-07      7890.9670858

The frequency estimates are very good, though not quite as accurate as the Data Svd FB algorithm in the AR (AutoRegressive) Spectrum option.

Close the Numeric Summary and select the EigVec FB algorithm.

The Eigenvector algorithm equally resolves the three components.

Again click the Numeric Summary option and inspect the linear sinusoidal fit.

Sine Component Fit (Suboptimal)

   Frequency       Amplitude       Phase       Power       %       Rel %

  499.052679      0.98308999      1.59193143      0.00465758      40.7717999      99.6456033

  561.948457      0.65379559      3.15038664      0.00209181      18.3113923      44.7527393

  626.286599      0.97001924      4.67875818      0.00467414      40.9168078      100.000000

                    0.01142352      100.000000      

   r2       DOF r2       Std Err       F-stat

  0.9998575370      0.9998254828      0.0088725423      35969.125114

   Data Power       Model Power       Error Power       Ratio

  0.0045311443      0.0045820399      6.455205e-07      7098.2101016

Although the Eigenvector algorithm is reputed to be more accurate than the MUSIC algorithm, it proved slightly less effective in this instance. The F-statistic is often helpful in this type of comparison. The higher the F-statistic, the better the parametric fit.

Close the Numeric Summary.

Non-Linear Frequency Refinement

Click the Non-Linear Optimization option. Non-linearly fit the 3 sinusoids by clicking the OK button and then the Review Fit button when the iterations are complete.

Click on the Set Confidence/Prediction Intervals button. Be sure Prediction Intervals is checked and that a 95% Confidence is selected. Click OK to close the Intervals dialog.

Note the exceptionally tight 95% prediction intervals about the composite curve.

Click the Numeric Summary button and inspect the fit statistics.

Fitted Parameters

rē Coef Det      DF Adj rē      Fit Std Err      F-value

0.99988373      0.99985757      0.00801550      44073.4141

  Data Power       Model Power       Error Power

0.0045311443      0.0045821585      5.268349e-07

 Comp      Type       Frequency       Amplitude       Phase       Power       %

    1      Sine      500.521961      1.02056485      1.55626043       0.00502822      41.0442050

    2      Sine      560.500550      0.70652849      3.18814022       0.00244915      19.9918438

    3      Sine      625.491093      0.98048061      4.69868524      0.00477337      38.9639512

The non-linear optimization resulted in only a modest refinement, suggesting that all three of the procedures that successfully isolated the three spectral components achieved very accurate frequency estimates. Compare the F-Statistic for the different procedures:

Non-Linear Optimization            44073
AR Data Svd FB 25/6            43128
MUSIC FB 25/6                  39986
EigenVector FB 25/6            35969

Close the Numeric Summary.

Click OK to close the Non-Linear Optimization. Click OK once again to exit the EigenAnalysis procedure.

Other Options

The Data Svd algorithms in the AR (AutoRegressive) Spectrum option and the MUSIC and EigVec algorithms in the EigenAnalysis Spectrum option are particularly effective high-resolution harmonic frequency estimators. The signal subspace adjustment is graphical which for the AR procedure is generally much easier than trying to find an optimum model order in a non-SVD algorithm. Although none of the procedures can provide accurate power estimates, the excellent frequency estimates make it a simple matter to derive the amplitudes, phases, and component powers from a fast linear least-squares procedure or from a non-linear parametric optimization.

The Prony Spectrum procedure offers an undamped sinusoid option with SVD to minimize the impact of observation noise. It can successfully isolate the three components although the Prony fit uses the same Data Svd FB AR algorithm for isolating frequencies, and thus the same frequency estimates arise. The Prony procedure in AutoSignal is primarily intended for fitting exponentially-damped sinusoids.

The ARMA (AutoRegressive Moving Average) Spectrum option can also isolate the three spectral components. The addition of MA coefficients makes it possible to model the nulls that arise in observation noise, although it also adds considerable complexity (non-linear iterative fitting) to the procedure. In general, you will probably find it simpler to discard the noise in an AR SVD procedure rather than fit it in a non-linear ARMA algorithm. ARMA spectral factorization does make it possible to fit an AR only model with stable coefficients (no roots outside the unit circle).