Data Structure

DOLfYN data objects are built on (subclasses of) pyDictH5 data objects, which are themselves subclasses of Python dictionaries. As such, these objects support all of the same basic functionality of dictionaries (e.g., indexing, iterating, etc.), with additional functionality that is designed to streamline the process of analyzing and working with data. Most notably, these data objects provide:

• a summarized view of the data structure when in interactive mode:

  >>> dat
  <ADV data object>
    . 1.05 hours (started: Jun 12, 2012 12:00)
    . INST-frame
    . (120530 pings @ 32Hz)
    *------------
    | mpltime                  : <time_array; (120530,); float64>
    | vel                      : <array; (3, 120530); float32>
    + config                   : + DATA GROUP
    + env                      : + DATA GROUP
    + orient                   : + DATA GROUP
    + props                    : + DATA GROUP
    + signal                   : + DATA GROUP
    + sys                      : + DATA GROUP
  

• attribute-style syntax (dat.vel is equivalent to dat['vel'])

• direct access of items within sub-groups (dat['env.temp'] works, and 'env.temp' in dat evaluates to True)

See the pyDictH5 README for additional details.

Data Description and Units

DOLfYN generally uses the *MKS* system, with most angles in degrees.

Table 1: The units of common variables found in DOLfYN data objects.

Name	Units	Group	Description/Notes
mpltime	matplotlib date format	—	The time data
vel	m/s	—	The velocity vector data
range	m	—	The distance from the ADP head
depth_m	m	—	The depth of the ADP
c_sound	m/s	env	The speed of sound used in velocity calculations.
press / pressure	dbar?	env	pressure measured by the instrument
temp	degrees Celsius	env	The temperature measured by the instrument.
accel	m/s2	orient	The vector acceleraton of the instrument
angrt	rad/s	orient	The angular rotation rate of the instrument
mag	???	orient	The magnetometer data.
velraw	m/s	—	The velocity without motion-correction.
velrot	m/s	orient	The rotational motion velocity (computed from angrt).
velacc	m/s	orient	The translational motion velocity (computed from accel).
acclow	m/s2	orient	The low-pass filtered acceleration signal.
heading	degrees	orient	The instrument heading* (clockwise from North)
pitch	degrees	orient	The instrument pitch*
roll	degrees	orient	The instrument roll*
orientmat	—	orient	The instrument’s orientation matrix in the earth reference frame
amp	???	signal	The acoustic signal return amplitude
corr	percent	signal	correlation between the send and receive pings.
ambig_vel	m/s	sys	The ambiguity velocity*
batt_V / batt	voltage	sys	The battery voltage
ensemble	—	sys	The ensemble counter
error	—	sys	The instrument error code*
status	—	sys	The instrument status code*
temp_clock	degrees C?	sys	The instruments clock temperature
temp_mag	degrees C?	sys	The instruments magnetometer temperature
temp_press	degrees C?	sys	The instruments pressure sensor temperature
tke_vec	m2/s2	—	Variance of velocity components (0: uu, 1: vv, 2: ww)
stress	m2/s2	—	The Reynolds stress array (0: uv, 1: uw, 2: vw)
sigma_Uh	m/s	—	Standard deviation of horizontal velocity magnitude
epsilon	m2/s3	—	Turbulence dissipation rate
Spec	<variables units>2/(rad/s)	—	A data group containing spectra of variables above.
omega	rad/s	Spec	Radial frequency.

Data Shortcuts (properties)

In addition to the data items listed above, DOLfYN data objects also contain shortcuts to tools and other variables that can be obtained from simple operations of its data items. These attributes aren’t listed in the view of the data shown above. Instead, to see the variables that are available as shortcuts for a particular data object, take a look at the dat.shortcuts property (new in DOLfYN 0.10.1).

Table 3: Notes on common shorcuts found in DOLfYN data objects.

Name	units	Description/Notes
shape	—	the shape of dat.u (see below)
n_time	—	the length of dat['mpltime']
u	m/s	dat['vel'][0]
v	m/s	dat['vel'][1]
w	m/s	dat['vel'][2]
U	m/s	the horizontal velocity as a complex quantity (dat.u + 1j * dat.v)
U_mag	m/s	the magnitude of the horizontal velocity
U_angle	radians	the direction of the horizontal velocity (counter-clockwise from East, X, or streamwise direction, depending on coordinate system)
subset	—	a tool for indexing a subset of the data object
tauij	m2/s2	the Reynold’s stress tensor
tke	m2/s2	the turbulent kinetic energy (half the sum of the data in ‘tke_vec’)
upup_	m2/s2	dat['tke_vec'][0]
vpvp_	m2/s2	dat['tke_vec'][1]
wpwp_	m2/s2	dat['tke_vec'][2]
upvp_	m2/s2	dat['stress'][0]
upwp_	m2/s2	dat['stress'][1]
vpwp_	m2/s2	dat['stress'][2]
Ecoh	m2/s2	Coherent turbulent kinetic energy (root-sum-square of Reynold’s stresses)
I	—	Ratio of sigma_Uh to horizontal velocity magnitude
Itke	—	Ratio of sqrt(2*tke) to horizontal velocity magnitude

Important Note: The items listed in Table 3 are not stored in the data object but are provided as attributes (shortcuts) to DOLfYN data objects.

User Meta-Data (dat.props)

The props data-group of DOLfYN data objects is a place for user-specified meta-data and DOLfYN-specific implementation data. The most common variables found here are described in Table 2.

Table 2: The entries in dat.props that are used in DOLfYN.

Name	Description/Notes
fs*	This is the sample rate of the instrument [Hz]
coord_sys*	The coordinate system of the data object. When a data object is rotated to a new coordinate system using the dat.rotate2() method, the value of dat.props['coord_sys'] is updated to reflect the final coordinate system. Valid values are: beam, inst, earth, and principal. For further details on these coordinate systems see the Rotations and Coordinate Systems section.
rotate_vars*	The variables in the data object that should be rotated when rotating the data object.
declination**	The magnetic declination where the measurements were made (in degrees that magnetic North is right of True North). Set this value using dat.set_declination(<value>).
declination_in_heading*	A boolean specifying whether the heading includes the declination. If this is True, then the earth coordinate system is True (i.e., v is velocity toward True North).
declination_in_orientmat*	A boolean specifying whether the orient.orientmat includes the declination. If this is True, then the earth coordinate system is True (i.e., v is velocity toward True North).
principal_heading	The heading of the +u direction for the ‘principal’ coordinate system [degrees clockwise from north].
has imu*	A boolean indicating whether the instrument has an inertial motion sensor/unit.
inst_make*	The manufacturer name.
inst_model*	The instrument model.
DutyCycle_NBurst*	The number of pings in a burst.
DutyCycle_NCycle*	The time – in number of pings – before the next burst starts. (this may be incorrect for some instrument types, please report issues)
inst2head_vec	The vector from the center of the ADV inst reference frame to the center of the head’s reference frame, in coordinates of the inst reference frame. It must be specified in order to perform motion correction. For ADPs this is always zero because the two coordinate systems are centered at the same place.
inst2head_rotmat**	The rotation matrix that rotates vectors from the instrument reference-frame to the head reference-frame. For ADPs this is always the identity matrix. Valid values are: 3-by-3 valid rotation matrices, or: 'eye', 'identity', or 1 all of which specify that it is the identity matrix. This is typically used for cable-head ADVs where the ADV head is not oriented the same as the ADV pressure case. It must be specified in order to perform motion correction.
body2head_vec	DEPRECATED as of v0.12. Please use inst2head_vec instead.
body2head_rotmat	DEPRECATED as of v0.12. Please use inst2head_rotmat instead.
motion accel_filtfreq Hz	The filter-frequency for the computing translational motion from the acceleration signal of an IMU. This is only used for motion correction. This high-pass filter is applied prior to integrating acceleration. This value is only used if when accel_filtfreq is not explicitly specified when motion-correcting
motion vel_filtfreq Hz	The filter-frequency for the computing translational motion from the acceleration signal of an IMU. This is only used for motion correction. This high-pass filter is applied after integrating acceleration. This value is only used if when vel_filtfreq is not explicitly specified when motion-correcting.
latlon†	The location of the measurements in decimal degrees. Latitude is positive North of the equator, longitude is positive west of the prime-meridian.

*: These entries are set and controlled by DOLfYN, and are not meant to be modified directly by the user. Attempts to set or change them directly (e.g., dat.props['fs'] = 10) will raise an error.

**: These entries are set and controlled via dat.set_<property name> methods. Attempts to set or change them directly (e.g., dat.props['declination'] = 20) will be deprecated in future versions of DOLfYN.

†: These entries are not used or set by DOLfYN, but they are useful measurement meta-data and are listed here to assist in standardizing the location and format of this information.

Specify meta-data in a JSON file

The values in dat.props can also be set in a json file, <data_filename>.userdata.json, containing a single json-object. For example, the contents of these files should look something like:

{"inst2head_rotmat": "identity",
 "inst2head_vec": [-1.0, 0.5, 0.2],
 "motion accel_filtfreq Hz": 0.03,
 "declination": 8.28,
 "latlon": [39.9402, -105.2283]
}

Prior to reading a binary data file my_data.VEC, you can create a my_data.userdata.json file. Then when you do dolfyn.read('my_data.VEC'), DOLfYN will read the contents of my_data.userdata.json and include that information in the dat.props attribute of the returned data object. This feature is provided so that meta-data can live alongside your binary data files.