HTI data files

Echoview supports data files output by HTI Models 241/243/244 split beam digital echosounder systems. These echosounders can be configured in a variety of ways. See Sampling Plans and Echoview Variables below.

The supported HTI echosounders log the following data as text data files:

Raw echo (.raw)
Tracked echo (.ech)
Tracked fish (.fsh)
Physical echo (.bot)
Summary (.sum)
Integration output (.int)
Echosounder configuration (.cfg)
Transducer calibration (.cal)

Echoview supports (*.smp) files. This support extends to those files that contain raw sample data alone and those files that contain raw sample data as well as associated settings, GPS and calibration data. From Echoview 4.50 onwards, an HTI algorithm for 48kHz sample interpolation is also applied to *.smp data.

Echoview offers an HTI beam compensation algorithm for split beam target detection, from Echoview 4.60 onwards.

Warning: Prior to version 4.50 Echoview supported the HTI 12kHz sampling rate that is used in *.smp files. Using the Nyquist-Shannon sampling theorem, the 12kHz sampling rate supports a minimum pulse length of 4/12000=0.33 ms. Analysis of such short pulses requires resampling to at least 24 kHz to minimize errors in target detection and pulse height (TS) and length estimation. As resampling was not implemented in Echoview at that time Echoview Software recommended a minimum pulse length of 0.66 ms. Under the 12kHz regime, single target detection variables derived from *.smp files with shorter pulse lengths may be suitable for fish tracking and fish counting where it is acceptable to have errors in the pulse height (TS) and pulse length.

Echoview supports the following file types.

File type	Data extracted
.raw	Single target detections. Single target detection raw variables are derived from this data. Echoview supports version 1.0, 2.1 and 2.2 HTI raw echo (.RAW) files. See also: Overlaying Single Target Detections on Sv Data from .INT files
.int	Highly summarized Sv data (integrated over layers and strata). Sv data variables are derived from this data. Echoview supports version 0.0 and 1.1 HTI integration (.INT) files. See also: Overlaying Single Target Detections on Sv Data from .INT files
.bot	GPS data only. Echoview supports version 0.0, 0.2 and 1.0 HTI physical echo (.BOT) files. Notes: You must add .bot files into one fileset of an EV file and add the related data file(s) into other filesets in the same EV file. Bottom range/depth are not extracted and Echoview therefore cannot display a sounder-detected bottom line using data that may have been written to the .bot file. If you wish to display a bottom line on an echogram you must detect in Echoview using a line picking algorithm, see Line picking and bottom detection.
.ech	Fish track regions. Echoview supports version 0.0 and 2.2 HTI .ECH fileset. Note: You do not add these files to a fileset, you import them (after adding the matching .raw file to a fileset). See Importing in Echoview for more information.
.smp	Raw sample data. Sv, TS and angular position variables are derived from this data. Notes: Echoview can read calibration settings for this type of .smp file only when the following data files are in the same folder as the .smp file: .cfg - the configuration of the echosounder used when collecting the raw sample data. .cal - the calibration settings of the transducer(s) used when collecting the raw sample data. The Echoview team, together with HTI, have observed that some .smp files can contain short sections of data with incorrect timestamps. The sections are approximately 1 second in duration. Data from an affected file is displayed correctly (incorrectly time stamped data is ignored) and Echoview flags the time issue with a message in the Message dialog box. HTI .smp timestamp issues can arise from a rare mix of hardware and software interactions during data acquisition. Please contact HTI for further information. See: Adding *.smp files to an Echoview fileset for more information.
.smp (containing calibration and settings data)	Raw sample data. Sv, TS and angular position variables are derived from this data using the calibration values from the first header in the file. Notes: For version 1.05 SMP files, the Calibration settings on the Calibration page of the Variable Properties dialog box apply after the raw HTI variables are derived. Things such as onscreen analysis and TVT thresholds are affected by settings on the Calibration page. Version 1.05 (or older) SMP files may contain incorrect location information. Please contact HTI for more information. For version 1.06 SMP files, SoundSpeed is read from the file. See: Adding *.smp files to an Echoview fileset for more information.

Notes:

Some HTI data files contain a file format version number on the first line of the data file.
Post-calibration of the single target detections is not supported at this time.

Major and Minor 3 dB beam angles

HTI refers to split-beam angles as the up-down (ang_UD) and left-right (ang_LR) angles. Echoview translates these to "minor-axis" and "major-axis" angles respectively.

For elliptical beams:

The left-right angles refer to the major axis of the transducer beam.
The up-down angles refer to the minor axis of the transducer beam.

The MajorAxis3dbbeamAngle and the MinorAxis3dbBeamAngle is read from the data file.

Sampling Plans and Echoview Variables

A sampling plan is defined as a combination of sequences and sampling periods within those sequences.

A sequence is of fixed duration (e.g. 10 minutes) and there may be any number of sequences defined which will be executed cyclically. Each sequence contains up to 16 sampling periods and these are executed cyclically within the sequence, one ping at a time. Any sampling period is identified by its sampling index which takes the form SsPp where s is the sequence number and p the sampling period number, for example S1P1 is the first sampling period in the first sequence and S2P5 is the fifth sampling period in the second sequence.

Acquisition parameters are defined for each sampling period. For this reason Echoview assembles all of the data for each sampling period into a single variable which contains the sampling index in its name.

This allows for two methods of multiplexing (switching from one set of data acquisition parameters to another):

Slow multiplexing

If the sequences contain only one sampling period this is termed slow multiplexing. Each of the sequences has its own acquisition parameters defined and may collect data from different transducers, the same transducer or the same transducer with different settings. The pattern of sampling periods for a single sequence cycle will resemble:

S1P1
S2P1
S3P1
...
SnP1

Where:

n is the number of sequences

Fast multiplexing

If the sequences contain more than one sampling period this is termed fast multiplexing. Each sampling period has its own acquisition parameters and may collect data from different transducers, the same transducer or the same transducer with different settings. The pattern of sampling periods for a single sequence cycle will resemble:

S1P1
S1P2
S1P3
...
S1Pn1

Where:

n is the number of sequences
ni is the number of sampling periods in sequence i

Notes:

The duration of a sequence is typically greater than the duration of the all the sampling periods within that sequence. The echosounder will cycle through the sampling periods, one ping for each sampling period in turn, until the end of the sequence.
Any of the ni may be 1.

That is, any sequence may contain only one,
or even zero sampling periods.

In this way some sequences may be slow multiplexed
while others fast multiplexed and others idle (no data acquisition taking place).

S2P1
S2P2
S2P3
...
S2Pn2

S3P1
S3P2
S3P3
...
S3Pn3

...

SnP1
SnP2
SnP3
...
SnPnn

Note: It is possible that two different sampling periods, SaPb and ScPd for example, share the exact same data acquisition settings (the same transducer and settings). This may be used in practice to define a sampling plan using multiple transducers but favoring one. For example 2 pings on one transducer for every 1 ping on a second (this would require 3 sampling periods, 2 of which share the same acquisition parameters. In Echoview you will find one variable per sampling period (as defined by a unique sampling index SsPp). You can use the Merge pings operator to assemble a single variable from the sampling periods you know to have used the same data acquisition settings.

For further information about HTI echosounders and sampling plans contact HTI.

Mux Channels

One of the data acquisition settings available for a sampling period is a mux channel. This is the physical connection on the back of the echosounder where a transducer is connected. Echoview reports the mux channel for each variable in its name. There may be more than one variable using the same mux port (transducer). This is because more than one sampling period was defined using that same transducer and may be for one of two reasons:

Different settings: To use the same transducer with different data acquisition settings. For example you may ping with a short pulse length and long pulse length using the same transducer, on alternating pings. Any of the echosounder settings may be changed however, including echo selection criteria, strata definition, fish tracking parameters or FM slide CHIRP filtering options (see your Digital Echo Sounder Operator's Manual for further information).
Same settings: To use the same transducer with the same data acquisition settings in a sampling plan which pings on one transducer more often than another (see the note on sampling plans above).

Echoview does not consider the data acquisition settings when assembling pings into variables and provides one variable for each sampling period (as defined by a unique sampling index SsPp). In the case of Same settings above you will need to use the Merge pings operator to assemble these into a single variable. To do so you must know which sampling periods used the same data acquisition settings.

Determining ping times from HTI data

HTI data files are generally plain text files, and can be examined in any text editor. They typically do not record a specific time against each ping and Echoview must calculate from the available information a time for each ping. This is conducted as follows:

File format

Contents

Method of determining ping times

.raw

Single targets

Not every ping is recorded in these files, only those pings on which a single target was detected. The following information however is available for each sequence:

The start time
The total number of pings
The number of sampling periods in the sequence
The duration of each sampling period in the sequence
For each single target that was detected during the sequence:
the number of the ping within the sampling period on which it as detected.

From this information Echoview calculates for each sequence:

The number of pings in each sampling period (i.e. in each Echoview variable), from the total number of pings and the number of sampling periods.
The sampling period cycle time, being the sum of all the sampling period durations in the sequence
The start time for each sampling period, being the sequence start time plus the duration of all earlier sampling periods in the sequence.

The time of each of the pings in any one sampling period (Echoview variable) is then calculated as the sampling period start time plus the ping number times the cycle time. The Echoview variable will contain all pings in the sampling period, including those which did not have any single target detections recorded against them.

This assumes that:

the time recorded in the sequence header is the time of the first ping in the sequence
the pings in each sampling period (in a sequence) are numbered sequentially
the ping numbering sequence begins with ping 0

For example, if a sequence:

begins at 14:00:00
contains 100 pings
contains 3 sampling periods of durations:
- 1 second
- 2 seconds
- 1 second

then the ping times will be calculated as follows:

3 variables are created with, 34, 33 and 33 pings respectively.
The Sampling period cycle time is 4 seconds
The start time of the three sampling periods are 14:00:00, 14:00:01, 14:00:03 respectively.
Pings in each of the three variables will have times as follows:

Period 1

Period 2

Period 3

14:00:00

14:00:01

14:00:03

14:00:04

14:00:05

14:00:07

14:00:08

14:00:09

14:00:10

...

...

...

Note: the end time of each sequence is also recorded in these data files. Echoview does not use this information.

.ech

Fish tracks

.bot

GPS fixes

Each GPS fix recorded in these data files is attributed to a ping in a sampling period within a sequence. If the fix is already used by a ping, Echoview searches for the nearest valid fix. Where a fix cannot be found, the GPS fix is extrapolated. The following information is available for each GPS fix:

the number of the ping with which it is associated
a latitude
a longitude
a UTC time stamp
a local computer time

.int

Integrated Sv data

Individual pings are not recorded. They are averaged into sequences (intervals) and strata (layers). Echoview presents each sequence (interval) as a single ping which has a time calculated from the start and end times recorded for each sequence (in the header and footer of that sequence in the .int file). The time midway between the start and end time of the sequence is used.

.smp

Raw Sv data

The time of each ping is recorded explicitly in the *.smp file format.

Overlaying Single Target Detections on Sv Data from .INT files

Sv data (from .int files) is not generally available from HTI echosounders at the same resolution as the raw echoes (from .raw files). As a consequence, in order to compare these two forms of data with the Overlay operator it is first necessary to match ping times (using the Match Ping Times operator).

The Match Ping Times operator can be used to create a virtual Sv variable with a ping for every raw echo on a .raw single targets echogram. The single targets echogram may then be overlayed on the matching Sv variable.

Adding HTI sample files to an Echoview fileset

*.smp files that contain raw data (only)

To have calibrated Sv values, an HTI sample fileset in Echoview must contain:

one or more *.smp files (containing the raw sample data),
one and only one .cfg file (containing the echosounder configuration used when collecting the data) in the same folder as the *.smp file, and
one .cal file for each transducer referenced in the .cfg file (containing echosounder calibration data) in the same folder as the *.smp file.

The sample data in the *.smp file is converted to Sv, TS, minor-axis angle and major-axis angles as described in HTI Sample data to Sv,TS and angular position.

Note: A fileset may optionally not contain the .cfg or .cal file, in which case the resulting echogram will not be calibrated Sv data.

To add *.smp to a fileset (and read the calibration settings from .cfg and .cal files) you must:

On the Fileset window, click Add.
In the Add Data Files dialog box, in Files of type list, select HTI sample data.
Select the required files (press CTRL while clicking to select multiple files).
Click Open.

*.smp files that contain raw data plus calibration and settings data

All the values required to derive calibrated Sv values are contained with these *.smp files. The addition of .cfg or .cal files to the fileset is not required.