libdivecomputer

• About
• Download
• Documentation
• Contribute
• Donate
• Contact

Transfer protocols

All the transfer protocols I have implemented so far (or for which I have the documentation) can be grouped into three categories.

1. Streaming

A dive computer in this category send all its data as one continuous data stream. You always need to download everything and there is no way to request individual dives.

• Uwatec (all models)
• Reefnet (all models)
• Suunto Solution and Eon

There are a number of improvements in the transfer protocol available:

Timestamp

Some dive computers have the ability to request only dives after a specific timestamp. This does reduces the transfer time considerably, because only the newer dives have to be transmitted.

• Uwatec Smart
• Uwatec Memomouse

Ending the transfer early

Some dive computers have the ability to end the transfer after each packet. This makes it possible to end the transfer as soon as an old dive is recognized, when the dive is after a specific timestamp, etc

• Reefnet Sensus Ultra

Realtime processing with multiple packets

Since you have to wait until the last byte is received to verify the checksum, you can't process the data before the complete data stream is received. However, some dive computers split the data stream into multiple packets with a checksum. This makes it possible to verify and process the data already during the transfer.

• Uwatec Memomouse
• Uwatec Smart (infrared)
• Reefnet Sensus Ultra

Transfers over an infrared connection don't need checksums at all, because the underlying infrared socket layer provides that functionality automatically.

2. Sequential access

A dive computer in this category supports retrieving individual dives. Dives are sent sequentially, so you can't request a specific dive. You always start downloading the first dive and keep requesting the next dive until no more dives are present. Since the dives are sent in reverse order (most recent dives are sent first), you can end the transfer as soon as an old dive is recognized. If you match on the date, you could easily implement the timestamp feature from the previous category.

• Suunto Vyper

This method is the most easy to use, because you don't have to deal with the internal memory layout (circular ringbuffers etc) to extract individual dives.

3. Random access

A dive computer in this category supports random access to the internal flash memory.

• Suunto Vyper
• Suunto Vyper2
• Suunto D9

This is the most flexible protocol of the three, and allows you to emulate the functionality of the two other categories. But it is also more complicated to use, because you become responsible for deciding which data to download and you also need to parse the internal memory layout yourself to extract individual dives.

API Proposal (version 1)

Downloading

I think that the possibility to read individual dives (category 2) is the most interesting for many applications. So the higher level api I had in mind is something along this pseudo c code:

device_t *device = <model>_device_open (parameters);
do {
   device_read_dive_next (device, buffer, size);
   ...
} while (!finished && !aborted && !error)
device_close (device);

I think this is a nice idea, because once the device specific backend is created, the downloading itself can be made device independent.

Parsing

Next, I would like to have some convenience functions for the decoding the most important data (datetime, depth, duration) and the profile data. Something like this pseudo c code:

dive_t *dive = <model>_dive_create (buffer, size, parameters);
dive_decode_datetime (dive);
dive_decode_maxdepth (dive);
dive_decode_duration (dive);
...
dive_destroy (dive);

Each function would extract the info from the binary dive. This shouldn't be very difficult, except maybe for the datetime and the profile.

Internal clock

Some dive computers (Uwatec and Reefnet) need the value of the internal device clock and the host clock to decode the timestamp in the dive. So they need to be available (and passed to the decoder backend). The device clock is provided in the handshake packet, but the host clock has to be obtained by the user.

Profile data

For the profile data, it is more difficult to provide a common api. How do you provide support for all the different formats in a flexible way? I could introduce a (large) sample struct that provides support for all know fields, but that has the problem that you need to know which model provides which field in the struct. I have been thinking of adopting the system that is used by the Uwatec Smart. They store a type indicator along with each sample value. In that case you only need to provide a list with all possible types (depth, time, pressure, temperature, events, ...) and a common storage format for each type (storage size and units). For example:

enum sample_type_t {
   SAMPLE_TYPE_TIME, /* integer, in seconds */
   SAMPLE_TYPE_DEPTH, /* double, in meters */
   SAMPLE_TYPE_PRESSURE, /* double, in meters */
   SAMPLE_TYPE_EVENT, /* integer, some standard flags */
   ...
};

And a dive_decode_sample_next function, that returns the next sample value (and its type) with each call. You call this function in a loop until it returns an EOF (or error) indicator. Or with a callback function:

typedef void (*sample_callback_t) (type, data, userdata);
dive_decode_profile (device, callback, userdata);

I think this could be a reasonable design for most applications. And if someone needs something more advanced, they can always obtain it from the binary data. What do you think about this idea? I'm open for other suggestions as well!

Problems and questions

Handshake and/or memory header?

Usually there is also some non-dive data present that does not fit very well into this scheme. For instance some dive computers send data during a handshake phase (Uwatec, Suunto and Reefnet), or have additional data in the header of the data stream (Uwatec). Others have a separate area in the internal memory for storing this kind of data (Suunto).

Some of this data is not very interesting or simply unknown and downloading it is only a waste of time. But in some cases this data (or a subset of it) is required for decoding the dive data (for instance the device subtype or the internal clock).Another problem is that the handshake phase is usually mandatory, whereas reading the extra memory area is often optional.

Adding a mandatory device_read_handshake function would be an option for models that have a well defined handshake. But what about the others (especially the Suuntos)? Does the handshake function download the entire memory header (which can be slow) or only what is strictly required? Or do we make it configurable? And what with models that have both a handshake and a memory header (Suunto D9 and Vyper2)?

Streaming protocol?

How to implement the proposed api for the dive computers in category 1? Download everything at once into a temporary buffer when requesting the first dive and start extract the dives when finished? With some types (where the stream is split into smaller packets) it would be possible to start extracting dives earlier (during the transfer itself). But since you should not abort the transfer before all data is received, this is maybe more suited for a callback approach?

typedef void (*dive_callback_t) (buffer, size, userdata);
device_read_dives (device, callback, userdata);

Progress indicator?

How to provide a progress indicator? With this api you can easily update a progressbar after each dive, but that is not very detailed. And with the models in category 1, the slow part is already over after receiving the first dive. Maybe introduce some callback function here? Another problem is that with some protocols (Suunto Vyper) the total amount of bytes that needs to be transferred is not known in advance.

Activation?

With some dive computers (Uwatec Aladin and Memomouse), the transfer is activated on the dive computer and not the PC. Thus, at some point the software needs to wait until the transfer is started. But this time is usually much longer than the communication timeouts.

API Proposal (version 2)

Downloading

For downloading individual dives, I suggest an iterator style api:

device_t *device = <model>_device_open (parameters)

device_handshake (device);

device_get_type (device);
device_get_capabilities (device);

device_set_progress (device, callback, userdata);

device_set_timestamp (device, timestamp);

device_read (device, address, data, size);
device_write (device, address, data, size);

device_prepare (device);
while (device_step (device)) {
   device_get_size (device);
   device_get_data (device);
}

device_reset (device);

device_close (device);

This design can be implemented for all three transfer protocols.

The handshake performs the (mandatory or optional) handshake sequence. If the underlying protocol does not have a handshake, this function does nothing.

A sequential protocol maps directly onto this design, and the prepare phase is simply a null operation. For a stream oriented protocol all data is transfered in the prepare phase and parsed during each step phase. For a random access protocol, the internal "housekeeping" data is retrieved during the prepare phase and with each step the next dive is downloaded.

Once data is successfully downloaded, the data (and its size) can be obtained with the get_size and get_data functions. This approach avoids the need to pass a buffer to the functions, which is nice because the necessary size is not always known in advance (e.g. dive data) and can be different for each type of device (e.g. handshake data). All memory management is handled internally in the library. The only downside is that the data remains only valid as long as no another function is called, so it has to be copied elsewhere to be able to use it later on.

Optionally, a reset function can be implemented to restore the device or internal state to the default, for instance to start a new download sequence.

Extra functionality, such as a random access read/write function or the timestamp feature, can be implemented when supported. If the device does not support it, the function can return an error code. Another possibility is to introduce a get_type and get_capabilities function to query the extra capabilities of the devices at runtime. Functionality that exist only for a specific device can still be accessed by means of a device specific function.

A progress report can be implemented by supplying a callback function, that is called during the prepare or step phase (depending on the protocol type). The callback function can provide the user with the amount of bytes already downloaded and the total amount that will be downloaded (if that is known in advance).

Parsing

Parsing could be done very similar:

parser_t *parser = <model>_parser_create (parameters);

parser_set_data (parser, data, size);

parser_datetime (parser);
parser_duration (parser);
parser_maxdepth (parser);

parser_sample_prepare (parser);
while (parser_sample_step (parser)) {
   switch (parser_sample_get_type (parser)) {
      case SAMPLE_TYPE_TIME:
         parser_sample_get_time (parser);
         break;
      case SAMPLE_TYPE_DEPTH:
         parser_sample_get_depth (parser);
         break;
      case SAMPLE_TYPE_EVENT:
         parser_sample_get_event (parser);
         break;
      default:
         break;
   }
}

parser_sample_reset (parser);

parser_destroy (parser);

This is very easy to use from the application side, but will be somewhat more difficult to implement in the library. Instead of one simple loop over all samples, the function has to store its current state, so it can resume parsing where it left after the previous call to the step function.

This approach is also great for backwards compatibility. If we add support for a new sample type, it is automatically ignore and does not break older applications. To update the application, you only need to add an extra case to the switch statement.

Protocol Overview for unified API

Feature matrix

Remarks

1. Performance is really slow, so the usage of this command is discouraged.
2. The data transfer is organized in pages, and the most recent ones are downloaded first.
3. No native support available, but easy to implement on top of the built-in commands.
4. The handshake command is required before each other command.
5. Data is read and written in pages of a fixed size and all addresses needs to be properly aligned to that page size.
6. The version command is required once at the start of a session, but can issued multiple times during that session.
7. The handshake and quit command can be omitted without any (known) side effects. Both commands abort the session when used in the middle of an active session.
8. The Uwatec Smart has some sort of pseudo handshake command, that can be omitted without any (known) side effects.
9. The Uwatec Smart supports multiple commands to obtain each piece of information about the device.

Explanations

When a device is opened, an active communication session is created, and one or more commands can be send to the device. Once the session is closed no further communication is possible. In some cases a special handshake sequence is required before other communication is possible. There are two different types of handshakes:

• A session handshake is performed at the start and/or end of the session.
• A command handshake is performed before every other command.

The version sequence returns information about the connected device, such as the serial number, the current internal clock, etc.

The main functionality of the device depends on the underlying protocol category. A random access protocol provides a read and/or write function. A stream oriented protocol provides a function to download all data at once. And a sequential protocol provides a function to download a single dive with each call.

Open questions

Suunto Vyper2/D9

• Is the Suunto vyper2/D9 version command mandatory, or can it be omitted?

Uwatec Aladin and Memomouse

• How is the transfer initiated with the Uwatec Aladin and Memomouse? Can we initiate a second transfer in the same session? How does the Memomouse know when to send its handshake? It only returns a fixed ID string that is not useful to the end user, so we can easily hide it in another function.

Uwatec Smart

• The Uwatec Smart protocol seems to have a mixture of handshake (two phases) and version (device time, serial and model number) commands. Are they mandatory, or can they be omitted? Is there a fixed sequence or can they be used at any time?

Oceanic Atom 2

• What is the purpose of the handshake and quit commands? The Oceanlog software accepts anything that is sent by the device. And omitting the commands seems to have no effect at all.
• Are there commands you can send when the usb cable is connected, but without a dive computer attached? Maybe some commands are answered by the interface and some by the dive computer?
• What happens if you send random data (or a slightly corrupted message, say a read with B3 instead of B1)? Maybe the device does answer everything it receives?
• Is the write command really composed of two parts that are both answered separately?
• Is the unknown command a keep alive command? What is the value of the autodisconnect timeout?

Parsing issues

Date/time representations

There are two ways device store timestamps. This relates to the internal clock of the device.

Absolute time (UTC)

Devices in this category have an internal clock that increase monotonically, with a fixed reference point. This reference point is often unkown (e.g. time of manufacturing) and therefore, the clock needs to be calibrated. This is done by obtaining a reference timestamp on the host PC and the device at exactly the same time during the transfer. The calibration formula is thus very simple:

datetime = clock_host - (clock_device - timestamp)

Since the host clock is typically expressed in UTC (e.g. the time() function of the C library time() returns the number of seconds since the unix epoch in UTC.), the resulting datetime is also in UTC. To be able to parse a stored timestamp, you'll have to provide those two reference values to the parser:

void parser_set_clock_calibration (parser_t *abstract, time_t clock_host, time_t clock_device);

Local time (watch style)

Devices in this category have an internal clock that functions similar to a regular watch. Time is stored as localtime and the clock can be adjusted freely. Thus time can move backwards and forwards, and the lack of a fixed reference point makes comparing timestamp tricky.

Typically, the timestamp is stored as a broken down time structure (e.g. similar to a struct tm in the C library). And to convert this value to calendar time, you'll need additional information on how to perform this conversion. For instance consider it as localtime (and use a mktime() like function for the conversion), or as utc time (and use a timegm() like function for the conversion), or even in a specific timezone:

void parser_set_clock_localtime (parser_t *abstract);
void parser_set_clock_utc (parser_t *abstract, long offset);

To provide maximum flexibility, the library would not only return the converted datetime value, but also the timezone (utc offset) that was used to perform the conversion. That would allow an application to easily switch between a localtime representation (for display) and a utc representation (for correct time math, sorting, etc). Applications that are only interested in localtime, could simply ignore this utc offset.

Depth calibrations

Reefnet devices store depth as absolute pressure values. To perform a conversion to regular depths, you need to specify (or assume) values for the atmosperic pressure and water density (e.g. seawater versus fresh water).

Model code

Some devices need knowledge about the specific model to parse the data.

Others?

TODO

Feature matrix

Copyright © libdivecomputer