device_set_fingerprint

[Jef Driesen]

On 01/28/2012 09:33 PM, Steve Boyd wrote:
> I am starting to write my own program to download my dive computer using
> the libdivecomputer DLL.
>
> I am a bit confused about exactly what the device_set_fingerprint
> function does.  The documentation suggests that it has something to do
> with limiting downloads to just new dives, but it is unclear about
> exactly how it accomplishes that.
>
> Any light you can shed on this is appreciated.

The explanation is as follows.

In the dive callback you receive for every dive, both the dive data and a 
fingerprint. This fingerprint is some piece of data that uniquely identifies 
that particular dive. Usually it's the date/time of the dive, but that's no 
guarantee. You shouldn't care about the actual contents and treat it as a opaque 
piece of data.

Now when a download finished successfully, you should store the fingerprint of 
the most recent dive somewhere. Keep in mind that dives are downloaded in 
reverse order, so the most recent dive is always the first one you get. Since a 
user may have more than one dive computer, you should store a separate 
fingerprint for each device. That's where you'll need the serial number from the 
DEVINFO event, so you can store a fingerprint for every serial number (and 
preferable combined with the backend type too, because serial numbers are not 
guaranteed to be unique across different backends.).

During the next download, when you get the DEVINFO event, you have to lookup the 
corresponding fingerprint and pass it to the device_set_fingerprint function. 
Then the backend code will internally do the right thing, and download only the 
dives that are more recent than the dive associated with that fingerprint.


You can find an example implementation in the universal application. When you 
use the -c option to point to an (existing) directory, it will store/load the 
fingerprints as small files there. There is also a -f option to supply a hex 
encoded fingerprint directly.


There is however one additional complication due to downloading in reverse 
order, and that's how to handle partial downloads where you downloaded some 
dives successfully, but not all. Imagine there are three dives A, B and C to 
download, with A being the oldest and C the newest. Let's assume you download C 
successfully, but an error occurs during downloading dive B:

C -> success
B -> error
A

Now, imagine you would ignore the error, and update the fingerprint store with 
that fingerprint of dive C (as you would do in the case of success). If you try 
to download again, the fingerprint of dive C will be recognized as already 
downloaded, and abort the downloading. Thus you are unable to download dive B 
and A, which is bad. However if you would not update the fingerprint store, 
you'll download all three dives again. Thus you end up with a duplicate of dive 
C, but at least you can still get B and A. This is obviously the least bad 
option. A possible solution would be to maintain an ignore list, where you store 
the fingerprints of the dives that did download successfully, and then compare 
every downloaded dive against this list. If all dives downloaded without errors 
this ignore list can be cleared, so you don't have to match against your entire 
logbook.


You can always implement your own "download only new dives" feature manually in 
your application, but I advise against doing that for a number of reasons:

1. The fingerprint support can take advantage of device specific knowledge, 
resulting in the most efficient experience. For example there are backends where 
this feature is incorporated into the download protocol. By doing this manually 
you can't take advantage of that, and you'll always perform a worst case 
download scenario where you download everything and then ignore the dives you 
already have. Even if it's not part of the protocol, there are backends where we 
have a more efficient implementation.

2. When you rely on the date/time field to make the decision whether you already 
have a dive or not (which is probably the most likely way to implement this), 
you may run into a number of issues. For example many dive computers allow you 
to set the clock backwards (e.g. clock reset due to battery replacement, 
timezone adjustments, etc). Some devices require the host clock for calculating 
the dive time, and a small change in the host clock may lead to slightly 
different dive times (e.g. clock drifts, downloading on a different system, 
etc). Again with the fingerprints we take advantage of device specific knowledge 
and match directly against the device timestamp, avoiding all these problems.

If you have more questions, just let me know!

Jef