AutoSignal Overview

Program Design

AutoSignal consists of many important leading edge signal analysis, filtration, prediction, and noise removal technologies. Instead of combining these into a single interface, each of these has been assembled into separate mini-applications. This approach allows each procedure to be custom tailored and streamlined for ease of use. The program consists of four basic sections:

· Data Import, Generation, and Basic Modification (File, Edit, Data menus; "Main" toolbar)

· Time Domain Analysis Procedures (Time menu, "Time" toolbar)

· Frequency and Time-Frequency Spectral Analysis Procedures (Spectral menu, "Spectral" toolbar)

· Advanced Processing Procedures in Fourier, Wavelet, and Eigendecomposition Domains (Process menu, "Process" toolbar)

Data Import, Generation, and Basic Modification

These consist of the basic import, save, list, edit, and calculation (transform) facilities. Since one is only interested in 2 columns of data at a time, AutoSignal does not hold open the various data files. For larger Excel files, for example, it is counterproductive to use RAM to store data that are not going to be used until a different data series is selected.

The pertinent data are thus Imported to in-memory vectors and the file is then closed. Aside from the memory considerations, this offers you the means to hold open and edit Excel files within Excel while using this same file as the AutoSignal data source. The primary AutoSignal sample data file is sample.xls. It contains a variety of interesting data sets as well as some used for testing AutoSignal's algorithms.

Since AutoSignal does not hold open any given data source (except in its automation), there is no Save option. The Save As option saves the in-memory data vectors to the file format of choice. Although AutoSignal does not use a proprietary format, it supports most common data formats, such as Excel, which are of interest to engineers and scientists.

The program includes a mini-spreadsheet editor, the AutoSignal XY Editor option in the Edit menu. AutoSignal also includes the ASCII Editor in the Edit menu. Either can also be used for numerically viewing the imported data.

The Generate Signal option in the Edit menu offers a powerful parser and expression compiler for generating user-defined signals. The program includes a variety of sample signal files useful for exploring the various capabilities of the program.

The View option in the Data menu offers a simple graph of the data as well as a very fast numeric list facility.

AutoSignal uses timers for edit fields. When a given timer interval has passed without additional input, an update automatically occurs. If you find this recomputation occurring too quickly or too slowly, it can be adjusted in the Edit menu's General Preferences.

Data transforms are accomplished using the Enter Calculation option in the Data menu. A calculation is a global transform that will be offered for use on all subsequent imports until the Cancel Calculation option is used.

The Section option is important for isolating portions of a long signal stream for analysis. You can isolate sections graphically by an XY sectioning mode where you box the region of interest, or an X sectioning mode where the left mouse is held down and slid right to disable points, and left to enable. You can also numerically specify ranges to include or exclude.

The Compare Imported Reference offers the means to graphically compare the current data set with any imported reference. Imported references can also be added to or subtracted from the current data set. The Subtract Imported Reference is used for removing backgrounds that can be measured separately.

The Reset XY Data will restore modified data back to its original import or generated state.

Time Domain Analysis Procedures

The Detrend option is used to fit a trend to the overall data and remove it. There are 8 parametric models which can be fitted with linear least squares or by one of two non-linear robust options that should be less influenced by the signal components. In addition to subtracting the fit, this option can zero the mean and/or normalize to unit standard deviation.

This Difference, Cumulative, Normalize option can difference the data with adjustable order and lag, compute various cumulatives, and normalize for unit area, unit power, unit standard deviation, and zero mean.

The Savitzky-Golay Smoothing Filter procedure offers effective time-domain smoothing for data sets with uniform X-spacing. The algorithm offers adjustable order, automatic sequential passes, and optional first through fourth smoothed derivatives.

The Spline Estimation option offers seven important spline procedures including variable order B-splines and a constrained cubic for pure interpolation, and least-squares B-splines, a cross-validation smoothing cubic spline, and the smoothing non-uniform rational B-spline (NURBS). Primarily for the procedures that perform smoothing, first and second derivatives are available. This procedure is useful for both upsampling and downsampling since the range and number of output points are specified. Uniform data are not required and AutoSignal will automatically average samples with identical x values.

An alternative interpolation procedure, the Non-Parametric Estimation option offers an adjustable order Loess-type (locally-weighted least-squares) procedure with two different weighting functions. This procedure can sometimes extract an underlying data pattern in extremely noisy data.

The Autocorrelation option offers the means to inspect the estimated ACS.

The AR Linear Prediction procedure offers effective forecasting and extrapolation. The AR model is either forward (subsequent points are predicted), or backward (prior points are predicted). The algorithms are available either as basic AR or where SVD (singular value decomposition) is used for in-situ noise removal. The two standard stabilizations are available for roots that lie outside the unit circle. The points that are to be processed can be specified, allowing predictions based on a data segment to be compared with actual subsequent data. The extent of the prediction is variable and noise can be added to see how well the algorithm's prediction stands up when white noise is added.

The Fractal Dimension option is included for checking spectra which produce a flat frequency response. The procedure computes the Hurst exponent, a measure of the fractal dimension of a data series. This algorithm is useful for determining if a data set is truly distinguishable from Gaussian or white noise.

Spectral Analysis Procedures

The Fourier Spectrum is the basic FFT option in AutoSignal. You can specify one of four different algorithms or the Best Exact n option. Zero padding is as simple as specifying the minimum FFT size. All Fourier options offer a variety of display options such as several power normalizations, dB, normalized dB, and amplitude. Peak detection uses a spline-based bin interpolation method in order to achieve some frequency refinement. AutoSignal also offers critical limits for most Fourier procedures.

Further, these can be expressed as white noise (AR1 Bkgrnd=0), or as red noise (power decreases with frequency). The light bulb option computes the lag-1 autocorrelation, a good starting point for red noise estimation. It is generally more accurate, however, to filter out signal components and fit an AR order 1 model. The limits are toggled on and off from the last button in the graph's toolbar.

A numeric report offers a tabulation of interpolated peaks, a frequency analysis with component amplitudes, phases, and powers based upon the FFT, and a linear least-squares sinusoidal fit where the component amplitudes, phases and powers are found independently.

The NL Optimization option offers refinement of the frequencies as well as the amplitudes and phases of the components. This procedure fits multiple sinusoids or damped sinusoids using constrained non-linear fitting. The optimization can be least-squares, or one of these robust minimizations.

The Fourier Spectrum with Data Window option adds data windowing. You will find AutoSignal's data tapering window list extensive. There are twenty fixed width windows, and nine adjustable ones including the Chebyshev, VanderMaas, and Slepian windows. An Explore Data Tapering Windows option simplifies window selection.

The Fourier Spectra with Data Window Comparison procedure enables up to three windowed spectra to be simultaneously graphed.

The Fourier Spectra of Segmented Data option averages FFTs from overlapping segments. Rather than assume stationarity, AutoSignal retains the spectrum of each overlapping segment. This makes it possible to visually check the assumption of stationarity in both 2D and 3D plots, as well as to generate full error bars for the averaging across segments.

The Fourier Multitaper Spectra option uses the series of orthogonal Slepian data tapers so that information at the edges of the data is utilized, and the variance of the spectral estimate is reduced. The algorithm generates F-values that are useful in assessing the significance of spectral peaks.

The Fourier Spectrum of Unevenly Sampled Data option generates a Lomb-Scargle periodogram. Its primary use in AutoSignal is for data with unevenly spaced X values. The algorithm was extended to use all of the data tapering windows that can be used with unevenly spaced data. A Chebyshev approximation window was created primarily to service this algorithm.

The AR (AutoRegressive) Spectrum option offers a selection of state-of-the-art algorithms. An AR model is fitted to the data and its coefficients are used to generate a continuous spectrum. The best AR spectral methods are excellent frequency estimators, offering this accuracy with quite short data sets. Least-squares methods that offer in-situ separation of signal and noise through singular value decomposition (SVD) are the most robust of AutoSignal’s AR methods. Selection criteria are available as is a graphical signal-noise SVD thresholding.

An AR spectrum can be generated for any starting and ending frequency and with any desired frequency spacing. Since AR spectral peaks can be exceedingly sharp, AutoSignal offers an Adaptive option which uses a Runge-Kutta procedure to integrate the spectrum adaptively. This offers a frequency set containing frequencies concentrated near the peaks and an accurate area under the spectrum (power). White noise can also be added to the data to observe its influence on the spectral estimates. The AR residuals can be inspected to see if they are normally distributed.

The AR Spectrum with Order Exploration allows multiple AR orders to be processed as spectra, plotted, and averaged. This option should be used to optimize the order. A 3D plotting option is often useful for ascertaining the optimum order.

The AR Spectrum with Algorithm Comparison allows up to three AR procedures to be simultaneously processed and plotted.

The MA (Moving Average) Spectrum option is used to model broadBand spectra containing nulls (zones where spectral content is absent). A true non-linear MA algorithm is available and to preserve stability, spectral factorization is available as a non-linear fit constraint.

The ARMA (AutoRegressive Moving Average) Spectrum is viewed as a good model for signals with noise since both peaks and nulls can be described. A pole-zero non-linear model is required. True non-linear fits are available, and these can include spectral factorization for stability and SVD for faster fitting.

The Prony Spectrum option fits complex exponentials to data. Using this method, it is possible to fit exponentially damped sines, undamped sines, and damped exponentials. For stability, SVD procedures are available. The Prony procedure can be used for fitting multicomponent exponential decays.

The Minimum Variance Spectrum option offers a low variance and the ability to graphically compare component powers. The frequency resolution lies somewhere between the FFT and the AR methods. Critical limits are available for all of the minimum variance spectral algorithms.

The EigenAnalysis Spectrum option offers the MUSIC (Multiple Signal Classification) and EV (eigenvector) high-performance frequency estimation algorithms. Since these algorithms can produce exceedingly sharp spectral peaks, AutoSignal's Adaptive spectrum is particularly valuable. The frequency of each spectral component is automatically refined to full machine precision.

The Short Time Fourier Transform Spectrum option produces a 3D time-frequency plot based upon a segmented overlapped FFT. Windowing is normally used to sharpen the resolution in time and minimize spectral leakage. The STFT spectrum is modeled and plotted with a bicubic spline. The 3D graphics options include four predefined profiles and surface animation. Since the STFT has a uniform time-frequency resolution, amplitudes can be directly read from the spectrum.

The Continuous Wavelet Spectrum (3D Surface) is the primary CWT (continuous wavelet transform) procedure. The CWT spectrum is rendered as a 3D surface. AutoSignal offers three adjustable mother wavelets in both real and complex forms. The number of frequencies is adjustable, as is whether or not the spacing is logarithmic. There is no need to use logarithmic scales in this procedure since the power is computed from surface integrals of a bicubic spline interpolant. 3D gradient critical limits are available for all wavelets and settings. Both white and AR-1 red noise critical limits are offered. A quick evaluation option offers fast interpolated surface maximums and time-frequency point evaluations.

The Continuous Wavelet Spectrum (2D Contour) option is a convenience for those preferring to view wavelet spectra strictly as contour plots.

The Continuous Wavelet Spectrum Frequency Range option is a specialized wavelet procedure to compute the power across time for a specified frequency band. The number of evaluated power points is adjustable.

The Continuous Wavelet Spectrum Time Range option is similar except the power is computed across all frequencies for a specified range in time. The global wavelet spectrum, which is similar to a smoothed FFT, is given by using the full time range.

Advanced Processing Procedures

The Fourier Smoothing and Denoising option is a specialized Fourier filtration procedure that sets either a frequency threshold for low pass frequency-domain filtration, or a signal threshold for zeroing all spectral elements below a given dB (normalized) value. The time domain data is reconstructed using the inverse FFT.

The Eigendecomposition Smoothing and Denoising option accomplishes a similar function except that the filtration occurs by zeroing those eigenmodes in an eigendecomposition of the data matrix that lie beyond a specified signal-noise threshold. By using a high order decomposition, it is often possible to remove nearly all of the noise within a signal.

The Wavelet Smoothing and Denoising option is similar to the Fourier procedure except that the thresholding is done in the time-frequency domain. The analyzing wavelet is fully adjustable, although the size and scales are fixed for reconstruction and the format is limited to an adjustable dB contour. Although frequency thresholding is offered, the main functionality comes from setting a wavelet spectrum threshold below which all values are zeroed prior to a CWT reconstruction. This option is effective in smoothing and denoising non-stationary data.

The Fourier Filtering and Reconstruction option is an extensive Fourier domain filtering and component isolation procedure. It is possible to set lower and upper frequency thresholds as well as lower and upper spectrum thresholds. A given portion of the spectrum can be included or excluded by graphical sectioning. Individual frequencies can also be toggled on and off.

Data windows are available, primarily to minimize spectral leakage so that very low power components can be isolated and reconstructed. When some frequencies are zeroed within an exact n non-windowed FFT, and the reconstructed data is again processed by an exact n non-windowed FFT, there will be true brickwall transitions at the zeroed frequencies.

The Eigendecomposition Filtering and Reconstruction option offers full eigenmode filtering and reconstruction. This allows components to be isolated by signal strength. In addition to reconstructing the data, the reconstruction can optionally consist of the eigenvectors, the principal components, the data components, FFTs of the data components, an FFT of the data, AR spectra of the components, or an AR spectrum of the data.

Two eigenmodes are required to capture an oscillation. For this reason eigenmodes often appear in pairs. Unlike the FFT, eigendecomposition is non-parametric. A non-sinusoidal anharmonic oscillation is captured as easily as a sinusoidal harmonic.

The Wavelet Filtering and Reconstruction option offers the means to reconstruct signals from spectral components that have been isolated in the time-frequency domain. Time, frequency, or spectral ranges can be set numerically or regions can be included or excluded graphically. The analyzing wavelet is fully adjustable, but the other elements are fixed by the reconstruction requirement. When a signal's spectral content varies across time, this option can readily isolate components that appear and disappear. Components that undergo changes in amplitude and frequency with time can also be characterized.

The Fourier Interpolation option is similar to the Fourier Filtering and Reconstruction option except that the reconstruction is computed directly from the amplitude, frequency, and phase of the sine components rather than by an inverse FFT. This offers a reconstruction where the data count and limits are variable. This option thus offers interpolation based upon the frequency spectrum. This means that any size reconstruction will generate this same spectrum. When the data count is increased, the frequencies beyond the original Nyquist can be zeroed using a low pass filter option. In addition to reconstructing the basic data, this option also makes it possible to reconstruct the first, second, third, or fourth derivatives.

The Fourier Upsampling option uses the traditional zero-insertion approach to interpolate data. This procedure is limited to integer upsampling ratios and all frequencies beyond the original Nyquist will always be zeroed. Since this procedure uses the inverse FFT for the reconstruction, it is very fast.

The Parametric Interpolation and Prediction option is a powerful composite algorithm that generates a parametric (sinusoids or damped sinusoids) model of the signal. The algorithm has three stages. In the first stage, a procedure is used to estimate the frequencies and component count. There are five algorithms for equally spaced data, and three ranges of the Lomb algorithm for data that are not uniformly sampled. The best algorithms use SVD for removing the influence of observation noise.

In the second stage a linear fit is made to determine the amplitudes, phases, and damping factors. These are the starting estimates for the third stage, the non-linear optimization that fully refines the parametric estimates.

To facilitate prediction tests, it is possible to specify that only a portion of the data set be processed. Subsequent data can then be compared with the predicted points. The generated data bounds and count are fully variable.

The Deconvolve Gaussian Response Function option manages the instance where a signal is smeared by a Gaussian response function. The deconvolution seeks to recover the true signal that would have been measured using an ideal sensing system.

The Deconvolve Exponential Response Function option is for instances where a signal is smeared by a first order or exponential response function. This is always a one-sided deconvolution that seeks to recover the true signal absent the delay of the measurement system.

AutoSignal Automation

Most algorithms within AutoSignal offer a powerful automation facility that enables the processing of large numbers of data sets in an Excel file in the same manner as the one currently being processed. It also offers the means to tie in real-time acquisition and processing using a custom automation DLL.

Most options offer an RTF export which generally consists of a numeric summary as well as optional graphs. RTF files are the portable MS Word format. You can select landscape or portrait mode for both the text and graphs.

All options offer Excel export. Some offer extended data, such as the FFT spectral options, while others will write only basic (two or three column) data. The export can be written using one worksheet for each data set, packing as many data sets as possible per worksheet, or if possible, the export will be written to match the input Excel format. Both Excel 95 and Excel 97 formats are supported, although Excel 95 is limited to 16384 rows.