Eigendecomposition Smoothing and Denoising

The Eigendecomposition Smoothing and Denoising option in the Process menu or the Process toolbar is a specialized Eigendecomposition filtration procedure that sets a signal strength threshold for retaining all signal-bearing eigenmodes and filtering out all noise-containing ones. This option is used exclusively to remove noise. In general, eigenfiltration is the most effective method for denoising.

The option presents a dual AutoSignal graph with the eigenvalues in the upper graph and the input and output time domain data in the lower graph.

The graph's toolbar has a button that can toggle on and off and change the proportions of the two graphs in the dialog. The default has the eigenvalue graph using the upper third of the region, and the output graph using the lower two-thirds.

Algorithm

The eigendecomposition algorithms vary only by how a lagged covariance matrix is constructed:

· CovM Fwd - the covariance matrix equivalent of processing a forward prediction data matrix

· CovM FB - the covariance matrix equivalent of processing a forward-backward prediction data matrix

· CovM Tpltz - a Toeplitz symmetric covariance matrix constructed using the Burg procedure

In general, the differences between the methods will be small. Because it imposes a matrix structure, the CovM Tpltz algorithm is typically the least effective in mapping data trends. If the AR algorithm properties apply, the CovM FB algorithm should be the most effective of the procedures.

Full data matrix procedures are not needed since no parametric model is computed. The decomposition of a forward or forward-backward data matrix produces the same eigenvalues and eigenvectors as the equivalent covariance matrix, and requires more processing time with large data sets. Similarly, no backward prediction data/covariance matrix algorithm is needed since it would produce the same decomposition as the forward data/covariance matrix.

In this procedure, the main task is to set a sorted eigenvalue threshold whereby the noise that is present in the signal can be removed. All of the eigenmodes are used in the reconstruction. The principal components of the signal will be captured in the initial eigenmodes. The noise is broken into low power elements that generally form a sloping floor beyond the signal eigenmodes in the eigenvalue plot.

Order

Establishing the eigendecomposition order for harmonic signals in the absence of noise is a simple matter. A model order of two is needed to fully describe one oscillation. Similarly an order of four is needed to fully model two oscillatory components. For noise-free data, the minimum order needed will be twice the number of signal components introducing oscillations in the data. These oscillatory components can be harmonic, such as undamped or damped sinusoids, sawtooth, and others. Or they can be anharmonic, oscillations where the model is not readily apparent.

Since there is usually some level of noise present in the data, a higher order model is needed to also account for the oscillations introduced by noise. To achieve a reasonable signal-noise separation within an eigendecomposition, it is necessary to use a high enough order so that the primary eigenvectors span only signal space. In general, the order is not particularly critical so long as it is sufficient to produce an effective partitioning of the signal and noise. What is crucial is the accurate specification of the signal space threshold.

Signal Subspace Selection

The number of principal eigenmodes comprising signal components can be specified numerically if you know with a certainty the number of spectral components present in the data. Again, you must enter a value that is twice the number of components. If three spectral components are known to exist, the signal subspace must be set to six. Even when the spectral component count is known, you should use the graphical inspection of eigenvalues to insure that a high enough order is being used to achieve the desired signal-noise separation.

When there is sufficient signal-noise separation in the eigenmodes, the eigenvalue plot reveals one or more sharp transitions between the signal subspace and the noise subspace floor. The last eigenmode before the long sloping noise floor represents the last element of signal space. Assuming a high-enough eigendecomposition order is used, this signal-noise separation does not become difficult until the noise level approaches that of the signal. At this point, the sharp characteristic transition may disappear.

To select the signal space graphically, left click on the last of the eigenvalues judged to represent signal space. All of the eigenvalues used in the reconstruction will be circled.

Estimated Noise Reduction

AutoSignal offers a robust noise estimation procedure that may be of some value for low-frequency signals. A cubic polynomial interpolation is made for each point using the two points to the left and the two to the right (excluding the current point). The difference between the interpolated and signal values is used to generate a measure of the white noise present in the signal. This assumes that the signal can be locally characterized by a smooth cubic interpolant. Also, the signal component(s) should exist only in the lower quarter of the Nyquist range. If a high frequency signal component is present, these estimates of noise will be invalid.

The In value reports the estimated white noise in the incoming data, the Out value the estimated white noise for the filtered signal. The percent is given as the amount of estimated noise remaining after filtration.

Power Reduction

In this section the In value reports the TISA power in the incoming data. The Out value is the TISA power for the filtered signal. The percent is given as the amount of power remaining after filtration. For most S/N ratios, the reduction in power should be minimal. These value can alert you when signal components are being discarded.

Correlation Coefficient

The r-squared correlation coefficient should also remain high. An rē of 1 is a perfect correlation while a value of 0 means the filtered and unfiltered signals are uncorrelated. Low rē values are also indicative of signal components being lost in the thresholding.

List

The List Data option lists the index, time, and output signal in a three column table. The listing uses the AutoSignal text viewer facility.

Copy

The Copy Data to Clipboard option copies the time and output signal values to the clipboard. Formats include full precision binary (for spreadsheets such as Excel) and ASCII (for pasting into text editors).

Save

The Save Data to Disk option writes the time and output values to a supported file format. These formats include ASCII, Excel 97, Excel 95, Lotus WK3, Lotus WK1, SPSS, or Systat.

Production Facility

The AutoSignal Automation facility allows unattended processing of large numbers of data sets. The data sets can be consolidated in an Excel file or acquired using a DLL. The graphs can be exported to an MS Word RTF file, while the processed data can be exported to an Excel 95 or Excel 97 file.

Local Options

A local option changes the data set for the duration of the current procedure only. The main data table is not altered. AutoSignal offers four local options in most of the spectral procedures.

Section the data to isolate specific regions for processing.

Detrend for removing mean or subtracting a least-squares trend model.

Fourier Filtration for isolating spectral components by frequency.

The Reset button restores the data to its state when first entering the procedure. Note that if you implement sequential local procedures, all of the revisions are discarded upon reset. If an Automation Session is in progress, the Reset button can be used to terminate the automated processing.

When exiting this procedure with the OK button, an option will be presented to update AutoSignal's main data table with the denoised data.