Significance Levels

Critical Limits

AutoSignal exclusively uses peak-based critical limit significance levels. This type of confidence limit is of particular merit in ascertaining the significance of the largest spectral component. In this type of test, one seeks to disprove the null hypothesis where one postulates either a white noise signal (AR(1)=0.0), or a red noise signal (AR(1)>0.0). Red noise is present when the background power decreases with increasing frequency.

It is important to understand the difference between the more traditional confidence limit and the peak-type critical limit used within AutoSignal. A 95% critical limit is that level where in only 1 of 20 separate random noise signals would the largest peak present achieve this height strictly due to random chance. A 99.9% critical limit is similarly that level where in only 1 of 1000 separate random noise sets would the largest peak attain this height.

The traditional confidence or significance level applies to a single set. For example, a standard 95% confidence limit would specify that level where 5% of the points in a single spectrum would be expected to lie above this height strictly due to random chance. This type of significance limit is not used within AutoSignal.

Levels

For frequency spectra, five different levels are plotted: 50%, 90%, 95%, 99%, and 99.9%. By the default colors, the 50% levels will be white, the 90% levels will be cyan, the 95% levels will be green, the 99% levels will be yellow, and the 99.9% levels will be red.

The labels can be toggled on and off using the Toggle Significance Labels button. The curves are automatically labeled.

If the AR(1) Bkgrnd option is checked in the wavelet options, the critical limits for the time-frequency spectra are plotted as color gradients. These override any Z-gradient coloring. The wavelet critical limit gradients are the following colors by default: 8-level grayscale from 10 to 50%, 8-level cyanscale from 50% to 90%, 8-level greenscale from 90% to 95%, 8-level yellowscale from 95% to 99%, and 8-level redscale from 99% to 99.9%.

Monte Carlo Approximations

The peak-type critical limits were generated using extensive Monte Carlo trials with the algorithms exactly as implemented within AutoSignal. The Monte Carlo data were then fitted to effective parametric models. For those cases where the data size n is the only factor, univariate TableCurve 2D parametric models are used. When an additional factor influences significance, such as an adjustable data window or wavelet parameter, bivariate TableCurve 3D models are used. For the segmented FFT, where segment size and overlap are additional influences, trivariate Chebyshev polynomials are implemented.

Determining Significance

The first step in making a significance determination is to select an appropriate background spectrum. If the noise can be assumed to be Gaussian (normally) distributed, a white noise [AR(1)=0.0] background can be used. If the background decays with increasing frequency, you can seek to describe this profile with a lag-1 autoregressive [AR(1)] model. In nature, values tend to range between AR(1)=0.50 and 0.80. The lag-1 normalized autocorrelation is often a good preliminary estimate. For a more effective estimate of the background, you can use the Fourier Filtering and Reconstruction, Eigendecomposition Filtering and Reconstruction, or Wavelet Filtering and Reconstruction option to filter the spectral components from the signal, leaving only the background. Then use the AR (AutoRegressive) Spectrum option with an order of 1 to fit the AR model. The desired AR(1) coefficient is listed in AR procedure's numeric summary.

Once the background is set, the peak spectral power can be compared against the various critical limits. When the largest peak in a spectrum attains the level of only a 50% critical limit, there is a 50-50 probability the peak could have arisen strictly from chance. On the other hand, when the largest peak exceeds a 99.9% critical limit, there is less than a 1 in 1000 probability the peak arose from chance.