Roadmap

This page will document a number of planned features, but do not expect a timeline here.

Dive metadata

Datetime parsing

For parsing the datetime, some backends need information that has to be obtained during the transfer. For instance uwatec and reefnet devices need the host clock, and oceanic devices need access to the previous/next dive(s) to be able go guess the correct decade. Since all this information is available during the transfer, it makes sense to parse the datetime in the download code. So this is another piece of information that needs to passed for each dive (more on that in the next item).

For the datetime format, I had something in mind like the C struct tm, but extended with a timezone offset. The timezone offset would be obtained from the host system, unless the device has native timezone support. If you're not interested in the timezone support, you can just ignore its presence.

struct datetime_t {
    // Localtime part.
    int year;
    int month;
    int day;
    int hour;
    int minutes;
    int seconds;
    // Timezone part.
    int timezone;
};

But depending on the clock type, this timezone field has a different impact:

For a reefnet/uwatec style clock, the timezone is important even if you are not interested in it. Due to the necessary clock calibration, the resulting timestamp is in always utc. To obtain the localtime part, you always need to apply a timezone offset. By default this will be taken from the host system (because that is about the only reasonable option we have). But if the the timezone at the dive site was different you need to apply a timezone correction to get a correct localtime part. Thus the localtime part changes, but the utc timestamp remains unchanged. Even if you are only interested in the localtime part, this timezone issue has an impact. But since the timezone field is there, you can use it do the correction and discard the timezone fields after the correction (once the localtime part is correct).

For a suunto/oceanic style clock, the localtime part is directly available. In this case the correct timezone is only necessary to be able calculate the utc timestamp. So when taking the timezone information from the host system, we are simply assigning a value to the timezone field. This timezone assignment is very different from a timezone correction, because now the utc timestamp changes, while the localtime part remains unchanged. Thus if you are only interested in the localtime part, this has no impact at all.

A real timezone adjustement can still make sense for a suunto/oceanic style clock, when the user forgot to change the device clock when diving in a different timezone. In this case, the localtime part needs adjustment. But this is a different use case compared to the reefnet/uwatec correction. In fact it is more similar to a user manually editing the datetime, because the device clock was wrong (for a suunto/oceanic), or the host clock was wrong (for a reefnet/uwatec).

Thus any automatic timezone adjustment, in the sense that an application can (optionally) provide a timezone widget to indicate one or more dives took place in a different timezone than the one on the host system, needs to take into account the clock type. It's like the user is overriding the default timezone (taken from the host) and explicitly providing the correct timezone offset. And this is something for which I don't have a nice solution yet. For instance, using a single global timezone for all downloaded dives does not work when the download contains dives from different timezone areas. A function that performs the correction on the datetime_t value alone does not work because the device clock type is lost (unless you always want a reefnet style adjustment, even for suunto style clocks). The only solution that I could come up with is a device_entry_set_timezone() function, that changes the timezone according to the device clock type. The adjusted datetime value can then be retrieved with the normal device_entry_get_datetime() function. (Read the next item first for more info on this "entry".)

Additional dive data

Related to the previous item, is the fact that some parsers (e.g. Uwatec Smart/Galileo) need additional information such as the model code to be able to parse the dive data. So this is yet another piece of information that needs to be communicated to the application. Note that the model code is already passed in the device info event. But events were designed to be an optional feature in the sense that everything should still work fine when you do not enable them. So that means the device info should be available as well.

Passing all this additional data by means of extra parameters on the callback function, becomes ugly and does not allow future enhancements (or even backend specific extensions). Introducing a new data structure, to hold everything related to a single dive together, becomes necessary. Since I prefer not to expose internal implementation details to the application, I think this is a good candidate for a new opaque data structure, with a number of accessor functions:

int
data_cb (device_entry_t *entry, void *userdata)
{
    device_entry_get_data (entry, &data, &size);
    device_entry_get_datetime (entry, &dt);
    device_entry_get_fingerprint (entry, &fp_data, &fp_size);
    device_entry_get_devid (entry, &id);
}

This device_entry_t pointer would be owned by the device handle. Thus there will be no need to free this pointer and it will only be valid inside the callback function. This is exactly the same as with the binary blob we have now.

But as you probably know, some backends have a very strict timing (e.g sensusultra). About the only thing you can (and should) do is copy the data and store it elsewhere for later processing. However, storing only the opaque pointer is not sufficient. It's only a lightweight handle, and it contents is likely to become invalid when the callback function returns. Thus the only option is copying each piece of data manually, which adds extra complexity, especially when we want to have the possibility of backend specific extension. (Note that copying an opaque pointer is not an option since you don't have access to its internals.) There are two more elegant solutions that I think are worth considering:

Introduce a new device_entry_copy() and device_entry_free() function, that can make a deep copy of the opaque handle. Since this copy is now disconnected from the device handle and owned by the application, it can easily be stored for later processing, while preserving its contents. Of course the application also becomes responsible for freeing this copy.

Instead of an explicit copy function, we could also introduce a new ownership flag that you can pass to the foreach function to request an entry that is owned by the application. In this case the copying is done internally, but for the rest it is identical to the previous one.

device_foreach (device, ownership, callback, userdata);

Note that this new device_entry_t could be easily re-used in the transition to an iterator style api.

Iterator style api

Replace the current callback api with an iterator style api.

Downloading

device_t *device;
device_entry_t *entry;

xx_device_open (&device, ...);

device_set_option (device, ...);

device_set_events (device, events, callback, userdata);

device_set_fingerprint (device, data, size);

[device_entry_init (device);]

while (device_entry_next (device, &entry) == SUCCESS) {
    device_entry_get_data (entry, &data, &size);
    device_entry_get_datetime (entry, &dt);
    device_entry_get_fingerprint (entry, &fp_data, &fp_size);
    device_entry_get_devid (entry, &id);
}

device_entry_reset (device);

device_close (device);

Parsing

parser_t *parser;
parser_entry_t *entry;

xx_parser_create (&parser, ...);

parser_set_option (parser, ...);

parser_set_data (parser, data, size);

[parser_entry_init (parser);]

while (parser_entry_next (parser, &entry) == SUCCESS) {
    switch (parser_entry_get_type (entry)) {
    case PARSER_ENTRY_TYPE_TIME:
        parser_entry_get_time (entry, &time);
        break;
    case PARSER_ENTRY_TYPE_DEPTH:
        parser_entry_get_depth (entry, &depth);
        break;
    case PARSER_ENTRY_TYPE_PRESSURE:
        parser_entry_get_pressure (entry, &pressure);
        break;
    }
}

parser_entry_reset (parser);

parser_destroy (parser);

Some open questions

Cancellation support

Support cancelling a running operation from a different thread.

Using a thread-safe cancel object?

cancel_t *cancel_create ();

device_set_cancel (device, cancel);

cancel_cancel (cancel); /* Executed in a different thread. */

cancel_destroy (cancel);

Or using a callback function, and leave the thread-safe access to the cancel variable to the application?

int cancel = 0;

int callback (void *userdata)
{
   return cancel;
}

device_set_cancel (device, callback, userdata);

cancel = 1; /* Executed in a different thread. */

cancel_destroy (cancel);

Image backends

Support reading from memory dumps by introducing image backends.

Filename encodings

Summary: Require UTF-8 encoded filenames on Windows.

Recent windows versions (NT and later) are using unicode strings internally, while the older Win9x based are using ansi strings. Because of that, each windows api functions that accepts a string has two variants: a wide character (unicode) and a ansi version. The generic function is implemented as a macro that evaluates to one of the two variants, depending on whether the "UNICODE" macro is defined or not. And there are also two character types: the normal 8 bit char and a 16 bit wchar_t type.

So you are probably wondering what this has to do with out library? Well, currently our api only accepts the 8 bit char type, and thus the ansi variant is always used. Unfortunately, this means we can only handle filenames that can be represented in the current ansi codepage. For the moment this is not a problem, because we only have to deal with serial port names, and those are simple ascii strings. But this could become a problem when we want to add support for reading memory dumps. If we want to fix this, we could:

Unix systems don't have this problem because they don't require a specific character encoding for filenames. As long as there is no directory separator or null character in the filename, the kernel will accept it. Only when you want to display that filename, the charset becomes important, but not for doing I/O.