Roadmap

[Jef Driesen]

Hi,

I'm planning a major cleanup of the public api before making a first 
official release. Because some of those changes require breaking 
backwards compatibility, I'm going to take this opportunity to introduce 
many changes that have been on my todo list for a long while, but have 
been postponed forever to avoid breaking backwards compatibility as much 
as possible.

The remainder of this (long) email is an overview of the planned 
changes. There are a couple of questions as well.

1. Decide on a version scheme

The current plan is to use a typical major.minor.micro scheme. The 
infrastructure to support this scheme is already in place for a long 
time, but it's just not being used.

The main question is then how to increment these numbers? Either based 
on features or binary compatibility? I have a personal preference for 
the last method. It might also be a good idea to be able to distinguish 
between release and development versions. For example using a "-dev" 
suffix.

Worth reading: http://apr.apache.org/versioning.html

2. Separate public and private headers

All public header files are moved to the include/libdivecomputer 
subdirectory. The main benefit is that the example applications can use 
the recommended include style (e.g. #include 
<libdivecomputer/header.h>), regardless of whether they are build inside 
the build tree, or standalone. And as a bonus, it becomes very clear 
which header is considered public or private.

3. Improved error codes

The device and parser error codes will be merged to a single set. At 
the same time a few extra error codes are introduced:

* DC_STATUS_SUCCESS     /* Success */
* DC_STATUS_UNSUPPORTED /* Unsupported operation */
* DC_STATUS_INVALIDARGS /* Invalid arguments */
* DC_STATUS_NOMEMORY    /* Out of memory */
* DC_STATUS_NODEVICE    /* Device not found */
* DC_STATUS_NOACCESS    /* Access denied */
* DC_STATUS_IO          /* Input/output error */
* DC_STATUS_TIMEOUT     /* Timeout */
* DC_STATUS_PROTOCOL    /* Protocol error */
* DC_STATUS_DATAFORMAT  /* Data format error */
* DC_STATUS_CANCELLED   /* Operation cancelled */

These are roughly identical to the current error codes, with a few 
modifications. The old TYPE_MISMATCH code is changed into the more 
general INVALIDARGS. The old ERROR is changed into the new DATAFORMAT or 
INVALIDARGS, depending on the context. The NODEVICE and NOACCESS codes 
have been added to return more fine grained error information when 
opening a connection to the low-level hardware (e.g. serial/irda/usb). 
Note that these two new error codes are not intended to report the 
status of the communication with the actual dive computer!

Are there any error codes missing?

4. Namespacing the public api

All public API (functions, structs, enums, etc) should be prefixed with 
"dc_" (which is short for libdivecomputer of course) to reduce the 
chances of conflicts with other libraries.

In recent additions this prefix is already used (e.g. dc_datetime_t and 
dc_buffer_t), but in older code it needs to be added. Note that this 
will break backwards compatibility everywhere!

5. Improved naming consistency

In a few places the naming of the typedef's is inconsistent. For 
example there is a callback named device_event_callback_t and 
dive_callback_t (missing "device" in its name), an enum named 
device_type_t and device_event_t (missing "type" in its name).

I intend to introduce a more consistent naming scheme:

dc_event_type_t;      /* An enum listing all possible types. */
dc_event_progress_t;  /* A struct corresponding to one specific type. 
*/
dc_event_devinfo_t;
dc_event_clock_t;
dc_event_callback_t;  /* A callback function. */

Related to this issue is the question whether the callback function 
should return the object pointer for which it is called or not (e.g. the 
this or self pointer)? Currently the event callback has this pointer 
(because you need this pointer for the fingerprint feature), while all 
others don't. I want to make this consistent by either adding it to all 
callbacks or removing it. Technically, you can always pass the object 
pointer by means of the userdata parameter, but it can be convenient to 
have it for free (at the expense of needing an extra parameter in the 
callback function signature).

Does anyone have a particular preference regarding adding/removing this 
object pointer?

6. Cleanup the parser api

The parser api needs some major improvement.

The biggest issue here is that the parser_sample_value_t union is 
passed by value in the sample callback. This means we can't introduce 
new sample types in the future without breaking backwards compatibility 
(because extending the union may change its size). This can be avoided 
by breaking the union into individual structs and pass them by reference 
with a pointer. This is similar to the current event callback, so it 
will improve consistency too.

The downside is the application will have to cast the pointer to the 
correct data type manually. It might be possible to keep the union (but 
still passed as a pointer of course), to eliminate the need for a manual 
cast, but this might not be entirely portable. I'll need to investigate 
this further.

Another concern are the sample events. Currently we have a large list 
with all possible event types supported by the many dive computers. But 
this starts to become a big mess because the data associated with each 
type varies greatly between different dive computers, so it's nearly 
impossible to map this into a single data structure. Therefore, I 
propose the remove the event sample type, and use the vendor type for 
this purpose. Then we can just document the format and pass the complete 
data to the application.

Note that in a later stage, we can always add back the more general 
purpose events (e.g. gas switches, etc), but then with an appropriate 
data structure for each event and not the single catch-all data 
structure we have now. This will need further discussion.

7. Re-order structures to eliminate padding

A few structures have unnecessary padding bytes inserted by the 
compiler due to alignment requirements (e.g. device_clock_t). These 
useless padding bytes can easily eliminated by re-ordering the fields in 
the structure. This breaks backwards compatibility, but since some of 
the other changes will break compatibility anyway, it's a good 
opportunity to get rid of the padding too.

8. Drop deprecated functionality

There are a few functions that have a better alternative, and can be 
removed.

For example the *_set_timestamp() functions in the reefnet and uwatec 
backends can be replaced with the more general device_set_fingerprint() 
function. You can still use an integer timestamp with the correct 
serialization to the binary fingerprint.

Another candidate for removal are the *_set_maxretries() functions. I 
think it's better to just handle this completely internally. In the end, 
the backend should know best how many attempts should be made to retry a 
failed command.

The device_version() function is also obsolete. All backends that have 
a mandatory version/handshake sequence during the communication perform 
this already internally. Only when you need the data returned by these 
commands for some reason, you can call the function explicitly. But 
although this functions looks device independent, it is not because you 
need to know the buffer length in advance. And then we could equally 
well turn it into a backend specific function. In cases such as the 
sensusultra handshake, where you wouldn't be able to call the function 
at any time, there is already a backend specific function to retrieve 
the cached handshake packet. We could even go one step further and 
introduce a special vendor event to pass the raw version/handshake 
packet to the application.

Would this cause trouble for anyone?

9. Improved tools

I plan to create a new "dctool" commandline application that is more 
flexible than the current universal application. the idea is to have a 
tool supports a number of actions:

dctool [<options>] action [<args>]

Planned actions are dump, dives, parse, extract, read and write. The 
major difference with the universal application is that the new tool 
will only perform a single action, but the output can be piped to the 
next action. This will require defining a common data format to pass the 
data through the pipe.

Note that this isn't a high priority item because it doesn't have any 
impact on the public api, and hence can be completed later on.

10. Stable numbers for the backend types (and merging device and 
parser)

To maintain backwards compatibility, the existing enum values shouldn't 
change when new backends are added. To achieve this, and at the same 
time keep backends from the same manufacturer grouped together, I 
propose to introduce two numbers to identify each backend: one for the 
vendor, and one for the device itself. This is similar to the USB 
VID/PID numbers. Both the VID and PID numbers can then be encoded into a 
single 16 or 32-bit number to make up the enum value.

Additionally, I would like to merge both the device_type_t and 
parser_type_t enums into a single enum. Right now every device backend 
has a corresponding parser backend (with a few exception where a parser 
can be shared by several device backends), and there is no good reason 
to keep just a single enum. I'm still looking for a good name for the 
new enum.

11. Return dc_buffer_t pointer instead of integer

I plan to make all dc_buffer_*() functions return the dc_buffer_t 
pointer to indicate success or failure, instead of an integer. These 
functions can only fail due to out-of-memory errors, so returning NULL 
makes more sense, and additionally it allows to chain calls.

12. Units: hydrostatic vs density

A few backends (e.g reefnet, cobalt, ostc) store the depth values (d) 
as pressure values (P). This makes a lot of sense considered the fact 
that a depth sensor is actually a pressure sensor. But it also means a 
conversion is necessary, and that requires knowledge of the atmospheric 
pressure (Patm) and the water density (rho):

P = rho * g * d + Patm

or if you prefer:

d = (P - Patm) / rho * g

Since those values are not always provided by the device (atmospheric 
pressure can be measured, but water density can't and is thus typically 
a user configurable setting or simply unknown), the calibration 
constants have to be provided by the user.

Note that this conversion has no influence on the decompression 
calculations, because those are done with the pressure values. The 
conversion to depths is strictly for displaying to humans, who prefer to 
see depths :-)

The use of the atmospheric pressure is straightforward, but for the 
water density their are many possibilities (density, salinity ratio, 
etc). In the current implementation a "hydrostatic" value is used, which 
is nothing more than the multiplication of the density with the standard 
gravity (and thus is equivalent to the pressure of 1m of seawater). The 
mean reason why this was chosen is simplicity in the implementation, and 
the nice correspondence with the simplified formula used by divers in 
the field:

PRESSURE(bar) = DEPTH(m) / 10 + 1
PRESSURE(atm) = DEPTH(ft) / 33 + 1

by defining the parameters (see units.h for the constants) as:

atmospheric = BAR
hydrostatic = MSW

However I'm not convinced the use of this "hydrostatic" parameter is 
very intuitive, compared to using the more well-known density value. 
Using the density directly would basically hide the gravity constant 
from the public api. Do you have any preference towards using 
hydrostatic vs density?

Note that you can still obtain the same results as with the simplified 
formula, by defining the parameters as:

atmospheric = BAR
density     = MSW / GRAVITY ~= 1019.7 kg/m^3

The math for imperial units is similar, resulting in these parameters:

atmospheric = ATM
hydrostatic = FSW / FEET
density     = (FSW / FEET) / GRAVITY ~= 1027.2 kg/m^3

13. Documentation

I should take some time to write proper documentation (e.g. doxygen or 
something similar). But since my time is pretty limited, this has the 
lowest priority.

That's it for now. Feedback is welcome as usual.

Jef