Using the Code operator

The Code operator is an Echoview virtual variable that is a Python application programming interface (API). The operator is specifically devised to transform echogram ping sample data. For example, according to a filter or algorithm of your design. The operator accepts other Echoview variables as input, performs computations on the associated ping samples in Python, and returns the output to Echoview.

The Code operator is only available under a licensed Advanced Operators module.

• Create a Code variable
• Code operator mechanics
• The Python source file
• The echoview package
• Wideband support
• Ping index
• Input/Output data types
• Importing packages and libraries
• Error handling
• No data handling
• Performance considerations
• Example Python source files

Security

Python programs may be written by anyone, and could contain malicious code or other security risks. Echoview recommends you review any files before executing them and only obtain them from trustworthy sources.

When opening an EV file that contains a Code operator or creating a Code virtual variable in a new EV file, a Security Warning window seeks permission to enable Python code for that EV file. Selecting No allows the display of and edits to the Code Variable properties dialog box. The Python source file may be edited, but is not executed.

A permission is valid while the EV file is open. To change the permission, close the EV file and reopen it.

The Security warning can be turned off under the Messages section of the Interface page of the Echoview Configuration dialog box.

Create a Code variable

1. Right-click on the Dataflow window and on the Shortcut menu select New, Variable.
2. On the Operator list select Code.
3. Enter a name for the New virtual variable and click OK. This will open the Variable properties dialog box for the Code operator.
4. On the Operands page use the Operand list to select the input operand(s). Use Add New Operand to add operands.
5. On the Code page, under Python source file, specify an existing Python source file (see Security).
   

   - OR -

   Click New to generate a new Python source file.
6. On the Code page, specify the Window size (pings) to access the adjacent pings to the matched ping (see Code operator mechanics and Ping index). This is an odd integer.
7. Click OK to execute the Python source file. This will close the Variable properties dialog box and create the Code variable on the Dataflow window.
8. Double-click the Code variable to display an echogram.

The Python source file

The Code operator allows you to generate a new Python source file. The default Echoview Python source file includes:

• comments to describe the code
• import statements for the
• echoview package (required)

  import echoview as ev

• NumPy package

  import numpy as np

• typing package

  from typing import List

• declaration of the Operator class (required)

  class Operator(ev.OperatorBase):

• declaration of the eval method (required)

  def eval(self, inputs: List[ev.OperandInput]):

  (see function annotation)
• Reassigning the input operand window (inputs[0]) to first_input

  first_input = inputs[0]

  and using the return statement (required) to communicate the ping sample data from Operand 1 to Echoview

  return first_input.measurement.data

  inputs[0] (and hence first_input) is an object of the OperandInput class and has a measurement attribute.
  first_input.measurement is an object of the Measurement class, which has a data attribute.

Notes:

• The default Echoview Python source file uses tabs to indent blocks of code. You may use tabs, or prefer to use spaces instead, but not mix both.

Code operator mechanics

• The Operator class needs to incorporate the OperatorBase class to interface with Echoview. (Refer to the syntax in the default Python source file.)
• The Code operator iterates through the pings of an echogram non-sequentially. The current input ping is called the matched ping and corresponds to the current output ping. The Window size (pings) setting is a built-in extension to give users access to the input echogram's following and preceding pings.
• Each operand that users select (via the Operands page) is an OperandInput object, or window. This window is a Python list whose elements are also Measurement objects. The matched ping is the element at the center of this list (see Ping index).
• By default the Code operator assigns the data type of Operand 1 to its output. Users can change this using the result_type method. If the conversion is not possible (for example the case of an angular position operand with Sv, to TS) the data type of Operand 1 is used. See also Input/Output data types.
• See also Error handling and Wideband support.

The echoview package

The echoview Python package comes installed with Echoview. The following classes in this package grant access to the pings from the input operands.

Classes in the echoview package

• OperatorBase
• OperandInput
• Measurement
• MeasurementType
• PingMode

• class echoview.OperatorBase

This is the parent class for the Operator class (required) in users' Python source files. The methods of OperatorBase are available to Operator through inheritance.

Attribute	Description
window_size	An odd-numbered integer from Window size (pings). Use the syntax self.window_size to obtain its value.
Methods	Description
eval(self, inputs)	This method reads in pings from all the input operands, executes your Python commands on the pings and communicates the resultant ping samples to Echoview. It is necessary to declare the eval method in your Python source file This method requires the return statement. The value is a NumPy array of the same shape as inputs. The inputs parameter is a list. Each element is an OperandInput object. inputs[0] corresponds to Operand 1. inputs[N] corresponds to Operand N-1. Since Echoview calls eval, use the syntax eval(self, inputs) when defining this method.
result_type(self, input_types)	The Code operator’s output defaults to the data type of Operand 1. You can deploy the result_type method to modify this. For example, to that of Operand N, or a data type from the MeasurementType class. The input_types parameter is a list. Each element is a MeasurementType object. input_types[0] corresponds to Operand 1. input_types[N] corresponds to Operand N-1. Since Echoview calls result_type, use the syntax result_type(self, input_types) when defining this method. Even if you do not declare it, result_type is automatically called in Echoview. See also Input/Output data types.

Note:

• The default Python source file uses function annotation to define the parameters for the eval method

inputs: List[echoview.OperandInput]

• You can refer to this parameter as inputs in the Python source file.

• class echoview.OperandInput

Represents the operands into the Code operator.

Attributes	Description
window_measurements	A list of Measurement instances from the operand that comprises the window.
window_index	The index of the middle element, also the matched ping in the window_measurements list.
measurement	The matched ping in the window_measurements list.

Note:

• OperandInput.window_measurements[OperandInput.window_index]

  is equivalent to

  OperandInput.measurement

• class echoview.Measurement

Each Measurement represents a ping.

Attributes	Description
beam_angle	The beam angle in degrees as an array for multibeam data. The beam_angle is at the center of the beam. Returns None for single beam data.
beam_width	The beam width in degrees as an array for multibeam data. Returns None for single beam data.
bearing_angle	See Bearing.
data	The ping sample data. Single beam data is represented by a one-dimensional NumPy array of real values. Multibeam data is represented by a two-dimensional NumPy array of real values. Axis 0 is the range axis and axis 1 is the beam axis. Notes: Single beam boolean data is stored as a one-dimensional NumPy array of boolean values. When the Code operator is used to generate single beam data from multibeam data, the assigned single beam geometry points in the direction of the transducer.
data_complex	Ping sample data (for acoustic wideband variables only): Complex power dB Complex Sv Complex TS Complex angular position Pulse compressed power dB Pulse compressed complex Sv Pulse compressed complex TS Pulse compressed complex angular position Single beam data is represented by a one-dimensional NumPy array of the real and imaginary components. A returned data array of type numpy.complex128 contains real and imaginary values comprised of (two 64-bit) double precision floating-point numbers. A returned data array of type numpy.complex64 contains real and imaginary values comprised of (two 32-bit) single precision floating-point numbers. See Wideband support.
datetime	The measurement timestamp represented as a Python datetime. Measurement.datetime uses datetime.datetime.utcfromtimestamp
gps_distance	The ping GPS distance in nautical miles. This may or may not be relative to the start of the echogram. See also vl_distance below.
gps_latitude	The latitude in degrees. See the Navigation section of the Details dialog box.
gps_longitude	The longitude in degrees. See the Navigation section of the Details dialog box.
heading	The heading in degrees. See the Navigation section of the Details dialog box.
index	The ping index or ping number relative to the operand passed to Python for evaluation. Both index and ping number start from zero. Refer to Ping index for more information.
matched_filter	A matched filter is represented by a one-dimensional NumPy array of the real and imaginary components of complex matched filter values (for acoustic wideband variables only). See Wideband support.
ping_mode	An object of the PingMode class.
speed	The speed. See the Navigation section of the Details dialog box.
start_depth	The start depth for each beam in the ping represented as a real value.
start_range	The start range for each beam in the ping represented as a real value.
stop_depth	The stop depth for each beam in the ping represented as a real value.
stop_range	The stop range for each beam in the ping represented as a real value.
tilt_angle	See Tilt.
type	An object of the MeasurementType class.
vl_distance	The vessel log distance in nautical miles. See also gps_distance above.
Method	Description
cal(name)	The Code operator may query a measurement/ping's calibration by using this to return the calibration value for the requested entry by the calibration name, for example Measurement.cal('PulseDuration'). Note: If no calibration value exists with the specified calibration name, Echoview generally uses a default value. In addition, where there is: no TransducerGain: Echoview returns TransducerGain if it is set. no TRFactor: the KaijoTrFactor is used if it is set otherwise TRFactor is determined using the Simrad EK60 power to Sv and TS equation. no AbsorptionCoeffient: when Francois Garrison values are available, the calculation for the center frequency is evaluated and used.

• class echoview.MeasurementType

The data type of the Measurement.

Attributes	Description
SINGLE_BEAM_ANGLE	Angular position
SINGLE_BEAM_BOOLEAN	Boolean
SINGLE_BEAM_WIDEBAND_ANGLE	Complex angular position
SINGLE_BEAM_WIDEBAND_POWER	Complex power dB
SINGLE_BEAM_WIDEBAND_SV	Complex Sv
SINGLE_BEAM_WIDEBAND_TS	Complex TS
SINGLE_BEAM_LINEAR	Linear
MULTIBEAM_ANGLE	Multibeam angular position
MULTIBEAM_BOOLEAN	Multibeam boolean
MULTIBEAM_MAGNITUDE	Multibeam magnitude
MULTIBEAM_PHASE	Multibeam phase
MULTIBEAM_SV	Multibeam Sv
MULTIBEAM_TS	Multibeam TS
MULTIBEAM_UNSPECIFIED_DB	Multibeam unspecified dB
SINGLE_BEAM_POWER	Power dB
SINGLE_BEAM_PULSE_COMPRESSED_ANGLE	Pulse compressed complex angular position
SINGLE_BEAM_PULSE_COMPRESSED_POWER	Pulse compressed complex power dB
SINGLE_BEAM_PULSE_COMPRESSED_SV	Pulse compressed complex Sv
SINGLE_BEAM_PULSE_COMPRESSED_TS	Pulse compressed complex TS
SINGLE_BEAM_SV	Sv
SINGLE_BEAM_TS	TS
SINGLE_BEAM_UNSPECIFIED_DB	Unspecified dB
UNDEFINED	Operand unavailable or variable upstream of the operand is unavailable.
Methods	Description
is_acoustic	Determines if self (i.e., the MeasurementType object) is an acoustic measurement. Returns a Boolean.
is_single_beam	Determines if self is a single beam measurement. Returns a Boolean.
is_multibeam	Determines if self is a multibeam measurement. Returns a Boolean.
is_db	Determines if self is an acoustic measurement in decibels. Returns a Boolean.
is_magnitude	Determines if self is a measurement in the linear domain. Returns a Boolean.
is_wideband_or_pulse_compressed	Determines if self is a wideband or pulse-compressed measurement. Returns a Boolean.

• class echoview.PingMode

The ping mode of the Measurement.

Refer to the About ping mode page for a description of these attributes.

Attributes
UNKNOWN
SINGLE_BEAM
MULTIBEAM
MULTIBEAM_HMODE
MULTIBEAM_VMODE
MULTIBEAM_SMODE

Wideband support

Echoview 12 onward allows processing the complex numbers from frequency modulated (FM) wideband data (see About transmitted pulses).

The complex values are available via Measurement.data_complex and Measurement.matched_filter. Define Operator.result_type within the Operator class to determine the correct echoview.MeasurementType:

      def result_type(self, input_types):
          return input_types[0]
    

Without this, the input operand ping data will be converted from FM to continuous wave. See also Wideband data types.

Complex value precision

Echoview uses complex numbers of type numpy.complex128, composed of two 64-bit precision floating-point numbers. However, you may use eval to return complex numbers of either type numpy.complex128 or numpy.complex64, depending on the precision required. For example

      return Measurement.data_complex.astype(np.complex64)

- OR -

      return Measurement.data_complex.astype(np.complex128)

Ping index

You can determine the matched ping's number from OperandInput.measurement.index (OperandInput.measurement is an object of the Measurement class).

Within the OperandInput window, this ping has index OperandInput.window_index.

For example, when using the default Python source file:

a) Ping 23 from Operand 1 in Echoview, using Window size (pings) 3.

inputs[0].measurement.index takes the value of the ping number, i.e., 23.

inputs[0].window_index takes the value 1. This is the middle item of the window comprising [Ping 22, Ping 23, Ping 24].

b) Ping 0 from Operand 1 in Echoview, using Window size (pings) 3.

inputs[0].measurement.index takes the value of the ping number, i.e., 0.

inputs[0].window_index takes the value 0. This is the first item of the window comprising [Ping 0, Ping 1].

Input/Output data types

The output acoustic variable data type from the Code operator is generally the same as the input acoustic variable data type of Operand 1.

You can change this by using Operator.result_type for example to that of operand 3

      def result_type(self, input_types):
          return input_types[2]
    

or some other data type using the attributes of the Measurement class.

      def result_type(self, input_types):
          return echoview.Measurement.SINGLE_BEAM_BOOLEAN
    

Important: Only the data type is changed, the data itself is not converted.

Wideband data types

The following wideband data types are output as corresponding single beam Sv, TS and power data types:

• Complex power dB
• Complex TS
• Pulse compressed complex power dB
• Pulse compressed complex Sv
• Pulse compressed complex TS

Input single beam operands with multibeam result output are not supported.

Input multibeam operands with single beam output is allowed. The output is similar to a Beam select operator. Note that the beam geometry is absent in the output data.

Importing packages and libraries

The Python NumPy and SciPy libraries are installed with Echoview along with other packages from the Python Standard Library (https://docs.python.org/3/library/site.html). You can immediately import these in your Python source file.

To import other packages you need to install them first.

Installing a package

Follow this procedure to install Python packages that are available from the Python Package Index (PyPI) using pip—the Python package installer.

1. Installing pip for Echoview (needed only once for your version of Echoview M.m)

   The following steps require Administrator privileges.

   1. Download and save https://bootstrap.pypa.io/get-pip.py into C:\Program Files\Echoview Software\Echoview M.m\Echoview\
   2. Open the Windows Command prompt as the administrator, and
      1. navigate to the Echoview Python interpreter location

         cd C:\Program Files\Echoview Software\Echoview M.m\Echoview\

      2. install pip for the Echoview Python interpreter

         python get-pip.py

         You may receive a WARNING from the installer that the scripts pip*.exe are not on PATH. Ignore this, as the packages you install will be for Echoview only and not any external Python environment.
2. Installing a Python package from PyPI

   With the Windows Command prompt

   1. navigate to the Echoview Python interpreter location

      cd C:\Program Files\Echoview Software\Echoview M.m\Echoview\

   2. indicate the destination to install the package

      SET PYTHONUSERBASE=%LOCALAPPDATA%\Echoview Software\Echoview64\M.m\

   3. install PACKAGE with the Echoview Python interpreter via pip

      python -m pip install --user PACKAGE

Reloading an imported package or library

If you update a package or library, you may need to

• close and reopen the EV file,
• close and restart Echoview, or
• use reload

for the change to take effect for the Code operator.

Error handling

The Code operator's echogram is displayed as black when:

• Echoview encounters an error with the Code operator
• there is an error in the Python source file
• the Python source file is not specified or cannot be found

In these instances, an error message either from Echoview or the Python, is sent to the Messages dialog box.

No data handling

The Code operator supports Echoview no data values in the following ways:

• a bitmap to indicate which values should be included. This is the approach taken by NumPy masked arrays.
• Not a Number (numpy.nan) values to indicate no data. (Some NumPy functions like numpy.nanmean will ignore NaN values.)

See also:

• Example Python source files
• Jay Alammar Visualizing machine learning one concept at a time (2019). A Visual Intro to NumPy and Data Representation. Available at online https://jalammar.github.io/visual-numpy/

Performance considerations

Follow these guidelines to improve the speed of your Code operator programs:

• Set Window size (pings) to the minimum that you need. This conserves the amount of data that will be loaded into memory.
• NumPy and SciPy (see Importing packages and libraries) contain highly optimized routines that can replace loops, conditional statements and mathematical operations.

Example Python source files

The following example Python source files serve to demonstrate some uses of the Code operator. These source files use the UTF-8 character encoding format.

You may need to configure the color scheme, grid, analysis and display settings and thresholds when viewing the Code variable echogram.

Example script	Suitable data	Details
Dusk and dawn bitmap	Single beam or multibeam	Generates a bitmap where all samples in pings from Operand 1 around dawn and dusk (±1 hour) are True.
Multifrequency categorization	Single beam	This is a multifrequency categorization technique (see Jech and Michaels, 2006) for echosounder data analysis. It implements an algorithm that classifies the Sv responses that register above a threshold, from each frequency.
Biomass density estimator	Single beam	Generates a mask, as a bitmap, that indicates if the biomass density is low enough to obtain reliable in situ TS values. This is accomplished by calculating the number of fish in the acoustic sampling volume (Nv, Sawada et al., 1993; see also Gauthier and Rose, 2001) and ratio of multiple echoes when measuring a single target (M) and then testing the resulting values against their respective threshold values (nv_threshold and m_threshold respectively).
Z score Multifrequency backscatter classification	Single beam	Calculates the normal deviate (Z-score) for walleye pollock (Theragra chalcogramma) as detailed in De Robertis et al. 2010 from three Sv inputs with the frequencies 38kHz, 120kHz, and 200kHz respectively.
Diagnosis		Demonstrates methods of diagnosing issues in your custom Code operator.
Multibeam fish detection filter	Multibeam data	Generates a bitmap that assists fish identification in multibeam echosounders (e.g., DIDSON or ARIS sonars).
Smoothing multibeam data using a convolution window	Multibeam data	Uses the Window Size property of the Code operator to smooth multibeam data in three dimensions. The smoothing is performed as a mean in the linear domain, using the Window size to define the number of samples in each dimension to include in the calculation.

See also

Code operator
About the Code operator