Transient Noise Sample Removal

The Transient noise sample removal operator identifies and adjusts sample values that are significantly higher than those of surrounding samples.

Based on the combination of settings applied, it can be used to adjust the value of samples that are affected by transient noise such as sound generated by wave-hull collisions (which manifest as long ‘spikes’ on an echogram). Care is required to prevent it from adjusting good data. Legitimate biological signal can be easily and mistakenly changed. Study your echogram data and use the settings Exclude above and Exclude below to protect legitimate data. The use of no data as a Replacement method may be able to highlight where legitimate data is adversely affected by this operator.

The operator is based on the “transient noise (TN)” algorithm and definitions described in Ryan et al (2015).

Settings are specified on the Transient Noise Sample Removal page.

Echoview accepts operands of the following data type:

  • Complex power dB
  • Complex Sv
  • Complex TS
  • Power dB
  • Pulse compressed complex power dB
  • Pulse compressed complex Sv
  • Pulse compressed complex TS
  • Sv
  • TS

For more information refer to the Transient Noise Sample Removal algorithm.

Settings

The Transient Noise Sample Removal Variable Properties dialog box pages include (common) variable pages and these operator pages:

Operands page

Transient Noise Sample Removal page

Exclusion

Setting

Description

Exclude above

Specifies the exclude above line.

Study your echogram data and use the settings Exclude above and Exclude below to protect legitimate data. The use of no data as a Replacement method may be able to highlight where legitimate data is adversely affected by this operator.

Exclude below

Specifies the exclude below line.

Study your echogram data and use the settings Exclude above and Exclude below to protect legitimate data. The use of no data as a Replacement method may be able to highlight where legitimate data is adversely affected by this operator.

Exclude below threshold (dB at 1m)

Specifies a sample Sv or TS exclusion threshold. This setting excludes the impulse noise algorithm from samples with values that are similar to no acoustic return. Any smoothed sample value below the threshold will remain unchanged.

The specified threshold value is the dB value at 1m for Constant in the Constant + TVG curve algorithm as described for the Data generator operator.

Smoothing

Setting

Description

Vertical window units

Specifies Vertical window units as Samples or Meters.

Vertical window size (samples)

Specifies the vertical size (samples) of the Vertical smoothing window. The Vertical smoothing window is one ping wide,

Context Window

Setting

Description

Horizontal size (pings)

Specifies the size of the Transient noise context window in the ping dimension.

Vertical size (samples)

Specifies the size of the Transient noise context window in the sample dimension.

Calculations per sample

Displays the number of calculations per sample for the operator. A high value indicates a longer operator evaluation time.

Percentile

Percentile specifies the value for the Nth Percentile algorithm used in the transient noise comparison.

Sample

Setting

Description

Threshold (dB)

Specifies the transient noise threshold value. Samples under the transient noise comparison that exceed the transient noise threshold are deemed to represent transient noise and are set to a Replacement value.

Noise sample replacement value

Specifies the Replacement method for samples deemed to represent transient noise.

Available methods are:

Percentile

Percentile specifies the value for the Nth Percentile algorithm when the Replacement method "Percentile" is selected.

For more information about the settings refer to the Transient Noise Sample Removal algorithm.

Transient noise sample removal algorithm

This operator identifies and adjusts sample values that are significantly higher than those of surrounding samples. The center sample value in a Context window is compared to a metric calculated with the remaining Context window sample values.

You can specify settings on the Transient Noise Sample Removal page of the Variable Properties dialog box.

Notes

  • It is advised to use great care with this operator because legitimate biological and bottom signal can be mistakenly changed. Study your echogram data and use the Exclusion settings Exclude above and Exclude below to protect legitimate data. The use of no data as a Replacement method may be able to visually highlight where legitimate data is adversely affected by this operator. After studying their data Ryan et al reason that TVG amplification of transient noise is not significant at depths above 250m and that the biological echo is strong above 250m and may be mistakenly identified as transient noise.
  • The operator algorithm may fail to identify noise in the set of start and end pings and set of top and bottom samples of the echogram. This is due to the handling of boundary samples with respect to the Context window. To address this issue, you can inspect the set of first and last pings and set of top and bottom samples and use regions and/or advanced operators to deal with manually identified noise.
  • In Ryan et al (2015), transient noise sample values are replaced with "no data". The algorithm offers two noise sample replacement methods, No data or Percentile of samples in the Context Window.
  • In Ryan et al (2015), a window of 50 pings and 20 meters is used for the calculation of lower percentile of surrounding samples. The algorithm uses default values of 7 pings and 9 samples. The Context Window has a size limit 101 pings x 501 samples. A larger Context Window results in a longer operator processing time.
  • Samples in adjacent pings may shift in depth under heave compensation and this may affect the transient noise.
  • Transient noise example.
  • In Ryan et al (2015) TS and complex data are not considered.
  • Complex and pulse compressed complex acoustic variables are largely handled in a manner similar to single frequency power, Sv or TS variables. When a new sample value is required, it is assumed that the complex phase for the sample is unchanged and the complex magnitude of the sample is calculated as for ComplexMagnitudeNew.
  • See also: The Data thresholding section on the Noise, background object and signal removal in Echoview page.
  • See also: The Effect of No data samples on the Wideband Frequency Response graph page.

Algorithm

Context Window for the transient noise comparison

Transient noise context window

Within the Context Window a smoothed copy of the data is used to identify noise.

  1. If a smoothed sample is transient noise then the corresponding original sample is replaced.
  2. If a smoothed sample is not transient noise then the original sample remains unchanged.

A smoothed sample is transient noise if:

It is between the Transient Noise Sample Removal Exclusion lines.

And

Its value is greater than the TVG-adjusted Transient Noise Sample Removal Exclude below threshold (dB at 1 m) value.

And

          Its Context Window is more than half full of valid* data values.

And

          Its value satisfies Transient_noise_comparison.

Where:

Exclusion lines = The lines specified under Exclusion on the Transient Noise Sample Removal page of the operator.
Vertical smoothing window =

The vertical smoothing window as specified under Smoothing on the Transient Noise Sample Removal page of the operator.

Exclude below threshold (dB at 1m) =

The value specified as Exclude below threshold (dB at 1m) under Exclusion on the Transient Noise Sample Removal page of the operator.

Context Window

=

The set of samples for the transient noise comparison.

The Context Window dimensions are specified on the Transient Noise Sample Removal page of the virtual variable.

m is the index for Vertical size (samples).

n is the index for Horizontal size (pings).

vij is the center sample of the Context Window.

The total number of samples in the Context window is the product of mn.

Invalid samples are: samples outside exclusion lines, No data samples and samples beyond the range of the ping. Note, you may want to use the Processed data operator to process samples within Bad data regions.

 

Number of valid samples = mn - number of invalid samples.

δ

=

Value for transient noise Threshold on the Transient Noise Sample Removal page of the virtual variable.

vij =

Sv in dB re 1m-1 of the center sample in the Context Window.

or

TS in dB re 1m2 of the center sample of the Context Window.

or

Power in dB of the center sample of the Context Window.

Lower percentile of samples in the context window =

Value for the Context Window Percentile. See the percentile equations below.

Context Window Percentile is specified on the Transient Noise Sample Removal page of the virtual variable.

Replaced =

Replacement method specified by Noise sample replacement value on the Transient Noise Sample Removal page of the virtual variable.

Available methods:

No data

vij = no data

Percentile

vij= Sample Percentile calculated using the all sample values, excluding the center sample of interest, within the Context Window.

Sample Percentile is specified on the Transient Noise Removal page of the virtual variable.

See also the percentile equations below.

Percentiles

Context Window percentile

The Context Window Percentile Lower percentile of samples in the context windowrepresents the lower percentile score of the sample values surrounding the center sample (vij) of the Context Window.

  1. Let vp be the value of the Context Window Pth-Percentile of an ascending order data set containing N elements with values. The N elements are the samples values within the Context Window, and exclude center sample value, and the element values are ordered as follows:

Percentile set

  1. Calculate:

Context Window lower percentile

Where P is specified by Context Window Percentile on the Transient Noise Sample Removal page of the virtual variable.

Replacement sample value percentile

The Sample Percentile vp is used to replace a center sample transient noise value with the specified Sample Percentile of the samples within the Context Window.

  1. Let vp be the value of the Sample Pth-Percentile of an ascending order data set containing N elements with values. The N elements are the samples values within the Context Window and the element values are ordered as follows:

Percentile set

  1. Calculate:

Transient noise Replacement Percentile

Where P is specified by Sample Percentile on the Transient Noise Sample Removal page of the virtual variable.

Percentile algorithm

General percentile

Where:

k'

=

Percentile rank where:

Percentile rank

P is the Pth-percentile.

N is the number of elements in the ascending order set of values.

k

=

floor function of k dash where Floor function k under a floor function, that gives the largest integer less than or equal to k'.

See also

About virtual variables
Background noise in Echoview
Background noise calculation on the Analysis page
Impulse Noise Removal operator
Attenuated Signal Removal operator
COM EvTransientNoiseRemoval
Using a SoundSpeedProfile: Notes