On 22-08-14 16:21, Dirk Hohndel wrote:

On Fri, Aug 22, 2014 at 03:53:05PM +0200, Jef Driesen wrote:

...

On 2014-08-22 13:29, Calle Gunnarsson wrote:

...

And implemented this for Cressi Leonardo, see the attached patches.

Calle's patch uses a double for the temperature, which I think is fine. I remember that a while ago there was some discussion about defining the exact meaning of such temperature field. Keep in mind that not all dive computer store the same kind of temperature value(s). Some may record the minimum or maximum temperature, the temperature at maximum depth, etc. Others may even record several values. I think trying to support all these variants will quickly end up in a mess, so I'm fine to provide just a single value without attaching any particular meaning. Of course the interpretation will always be fixed within a particular backend, but temperature values from different backends may have a slightly different interpretation.

I will repeat my earlier comment that that's disappointing. There is no way an application can now how these temperatures are defined. Isn't that what libdivecomputer tries to do, to take away the need for the application to worry about which dive computer was used?

I'd love if we could add a flag with at least three value:

• TEMP_IS_LOWEST_TEMP
• TEMP_IS_AT_DEEPEST_POINT
• TEMP_IS_OTHER

I know from diving with multiple dive computers that at least some single temperature Suunto dive computers use the "at deepest point" logic, while some my old Mares appears to log the lowest temperature.

I certainly don't mind a solution where the exact interpretation is better defined then in my simple proposal. That's exactly why I brought up this discussion again. I would like to get some feedback from the application developers on what they need for their application. And then we can work out something from there.

One of the main reasons I'm not really in love with the flags you propose, is that it doesn't really solve the problem that some dive computers store multiple temperature values. Which one should libdivecomputer pick then? What if your application really wants the minimum temperature, but libdivecomputer decided to pick the maximum temperature? If you would just ignore the flag, then why do you need it in the first place? If you discard the temperature if it doesn't have the right flag, then your users will likely complain about the missing temperature. So yes, the flags does give you some extra info, but I doubt it will be very useful in practice.

Anyway, let's approach the problem from a different side. I spend the evening collecting some info about which type of temperature(s) dive computers record by looking at the data displayed by the manufacturers application. Not everything has been reverse engineered, but this should provide a good insight of what is likely stored in the data. This is what I found:

Suunto DM3: maxdepth, begin, end Suunto DM4: maxdepth

For the vyper family, I can confirm the temperature at maxdepth, begin and end of the dive is stored in the data. For the newer vyper2 and d9 I don't know. But these models store a full temperature profile, so the header value is less interesting. Suunto DM4 just shows less information than the older DM3.

Mares DiveOrganizer: min, max Mares Iris/Drak: min

The DiveOrganizer app supports the iconhd family devices, while the Iris and Drak app supports the older nemo and puck family. For the iconhad family, both the min and max values are stored in the data. And if my info is right, the older models only store the min temp.

Cressi PCLogbook: unknown Zeagle DiveLogbook: unknown

These Seiko based applications just show a temperature value, and don't give any indication of the type. I also don't know how and where it's stored in the data. Most likely somewhere in somewhere in the header, because there is no full temperature profile.

Cressi Pc Interface: min

This is the application for the leonardo family. And also the one for which Calle provided a patch.

Scubapro Logtrak: min, max Uwatec Smartrak: min, max Uwatec Datatrak: min, air?

For the smart and meridian family the applications show min and max temperature. This matches with the reverse engineered documentation. These models also have a full temperature profile. For the older aladin/memomouse family, the application shows min and air temperature, but after downloading dives, the air temperature isn't filled in, so it might be just a field for manual entry by the user. The docs indicate there is a temperature at the start of the dive if the dive has profile data (which is not always the case).

Oceanic Oceanlog: min

Application shows min temperature. No idea whether it's stored in the data or not. I suspect it is, but with oceanic you never know. With the exception of a few models, they also have a full temperature profile.

Shearwater: none Atomics Cobalt: begin OSTC2/3: min

These are the ones for which we have official documentation. What I indicate above are the values stored in the header, but they all have a full temperature profile too (note that for the ostc the sample temperature can be disabled by the user).

Wlog: min, air Diving Log: air, water Subsurface: air, water Diveboard: surface, bottom Jdivelog: surface, water

These are some of the existing general purpose applications. They all seem to support a surface and water temperature. (Air and bottom are essentially the same thing, just a different name.) I assume this is a good indication of what most user expect to see in a logbook application independent of the dive computer manufacturer.

To summarize, we have 5 different variants:

* minimum * maximum * at begin * at end * at maxdepth

The first two need very little further explanation. The minimum temperature is definitely the most common variant. I assume one of these values is also what most people will record as the water temperature in the general purpose logbook applications? Although I haven't come across a general purpose application that supports both minimum and maximum water temperature, I can see the value in supporting these two subtypes.

The temperature at begin and end of the dive are a lot less common (Suunto, Cobalt and maybe the old Uwatec Aladins). I think they are also among the least useful variants. The temperature sensors in most dive computers are very slow and need a significant amount of time (in the order of several minutes) to adapt to changing temperatures. So the temperature at the start of the dive will not be the water temperature, but the air temperature. I guess the reason why people want to log this, is because it gives an indication of the weather conditions and maybe also the difference with the real water temperature? But what's the purpose of the end temperature?

The temperature at maximum depth is a Suunto only thing. In practice this will be roughly equivalent to the minimum temperature, because due to physics laws, the coldest temperature is usually found at the greatest depth. I know there are exceptions, but for 99% of the dive sites this is probably correct. So either not supporting this one, or mapping it to the lowest temperature, is not unreasonable I think.

That brings us down to 3 different temperatures worth supporting: surface temperature, minimum and maximum water temperature.

This corresponds nicely with the values that the existing general purpose applications are using. Except that we distinguish between minimum and maximum water temperature (if available). In case only one water temperature is available, we could even return the same value for min and max. For devices with a full temperature profile, min and max can easily be calculated from there if absent in the header (note that I don't intend to implement that part for now, because the header value is most useful for devices which don't have a full temperature profile, so this is lower priority.)

For the api, maybe we can stick to the DC_FIELD_TEMPERATURE with a single double value, but use the flag parameter of the dc_parser_get_field function to specify the subtype:

typedef enum dc_temperature_t { DC_TEMPERATURE_SURFACE, DC_TEMPERATURE_MINIMUM, DC_TEMPERATURE_MAXIMUM, } dc_temperature_t;

In the application you would call like this:

dc_parser_get_field (parser, DC_FIELD_TEMPERATURE, DC_TEMPERATURE_XXX, &value);

What do you think?

...

For the tank pressure, we certainly need support for multiple tanks, so my proposal is to add DC_FIELD_TANKPRESSURE along with a DC_FIELD_TANKPRESSURE_COUNT. That's consistent with what we already have for the gasmixes. For the corresponding data structure, I propose this:

typedef struct dc_tankpressure_t { double begin; double end; } dc_tankpressure_t;

That would be consistent with the data that is delivered in the samples (e.g. a pressure value with a tank id). The begin/end fields would simply correspond to the value of the first/last sample. So no attempt to add tank size (which not many dive computers support) or anything else.

Why not? You support at least one DC that does provide the data (Cobalt) and it would be easy enough to include the three values that dive computers may record:

typedef struct dc_tankdata_t { double begin; double end; double air_volume; // this is usually cuft double wet_volume; // this is in liter double working_pressure; // in bar like behin and end };

Even having just one backend that today fills these values makes things more useful for applications - and as more dive computers provide this you have no need for yet another API.

Today Subsurface extracts this information from raw data - we can of course continue to do so but I'd much prefer if this was in the official API.

I'll leave this part for another time, because I'm running out of time now.

Jef

On Monday, August 25, 2014, Sven Knoch info@divinglog.de wrote:

I think this would be a good solution. Is there a way to find out if temperature data is not available? Because returning a zero could mean zero degree temperature or that no data is available. Importing a 0°C if it was actually 30°C will cause lots of support requests.

This is a god idea. Another piece of software that use libdivecomputer, MacDive, still doesn't know the difference between 0°C and a missing temperature. Definitely my #1 annoyance with MacDive, but I don't really know if libdivecomputer is to blame here.

Cheers, Henrik

On Sun, Aug 24, 2014 at 11:02 PM, Jef Driesen jef@libdivecomputer.org wrote: [...]

The temperature sensors in most dive computers are very slow and need a significant amount of time (in the order of several minutes) to adapt to changing temperatures.

Does anyone have different experience? Any datasheet of pressure sensor used in a dive computer, which would prove above wrong?

Also, any publication in the context of temperature, which would claim that anything beside cold dive (minimum temperature) matters?

[...]

Regards,

w

On 25-08-14 00:02, Jef Driesen wrote:

For the api, maybe we can stick to the DC_FIELD_TEMPERATURE with a single double value, but use the flag parameter of the dc_parser_get_field function to specify the subtype:

typedef enum dc_temperature_t { DC_TEMPERATURE_SURFACE, DC_TEMPERATURE_MINIMUM, DC_TEMPERATURE_MAXIMUM, } dc_temperature_t;

In the application you would call like this:

dc_parser_get_field (parser, DC_FIELD_TEMPERATURE, DC_TEMPERATURE_XXX, &value);

What do you think?

Answering my own question, I think it will be easier (to implement on my side) to just define three temperature fields:

DC_FIELD_TEMPERATURE_SURFACE DC_FIELD_TEMPERATURE_MINIMUM DC_FIELD_TEMPERATURE_MAXIMUM

instead of subtypes. On the application side this shouldn't make any difference.

Jef

On 04-09-14 16:30, Dirk Hohndel wrote:

On Thu, Sep 04, 2014 at 07:21:19AM +0200, Jef Driesen wrote:

...

On 25-08-14 00:02, Jef Driesen wrote:

...

For the api, maybe we can stick to the DC_FIELD_TEMPERATURE with a single double value, but use the flag parameter of the dc_parser_get_field function to specify the subtype:

typedef enum dc_temperature_t { DC_TEMPERATURE_SURFACE, DC_TEMPERATURE_MINIMUM, DC_TEMPERATURE_MAXIMUM, } dc_temperature_t;

In the application you would call like this:

dc_parser_get_field (parser, DC_FIELD_TEMPERATURE, DC_TEMPERATURE_XXX, &value);

What do you think?

Answering my own question, I think it will be easier (to implement on my side) to just define three temperature fields:

DC_FIELD_TEMPERATURE_SURFACE DC_FIELD_TEMPERATURE_MINIMUM DC_FIELD_TEMPERATURE_MAXIMUM

instead of subtypes. On the application side this shouldn't make any difference.

The difference is, I think, that THIS change requires the app to be aware of this and read all the values. The alternative change would an older app to just work (and simply not know which type of temperature this is - just like before).

From a Subsurface case I am happy with either solution.

The DC_FIELD_TEMPERATURE field doesn't exist yet, so there is no backwards compatibility problem here. An application that doesn't care about the type of temperature, and which just wants a temperature, will also have to pass some subtype in the other solution. I would just pass 0, then it would get the temperature corresponding to the subtype we happen to list first in the enum (SURFACE for example). That wouldn't be any different than passing DC_FIELD_TEMPERATURE_SURFACE without subtypes. Or am I missing something?

Jef

On Mon, Aug 25, 2014 at 04:58:52PM +0200, Jef Driesen wrote:

...

Even having just one backend that today fills these values makes things more useful for applications - and as more dive computers provide this you have no need for yet another API.

Today Subsurface extracts this information from raw data - we can of course continue to do so but I'd much prefer if this was in the official API.

My criteria for including new features has always been that it has to be reasonable generic AND supported by at least a few dive computers. And that last criteria does not apply here. If we ignore the Uemis, which is rather special because it has a complete built-in logbook of its own, the Cobalt is basically the only dive computer which gives us the tank size.

You can argue that we need this anyway, because future dive computer will provide this information too, but I'm not really convinced that will happen. The reason behind that statement is very simple. A dive computer doesn't need this kind of information to do its job. For gas consumption estimations (e.g. How long before I'll run out of gas?), your dive computer doesn't need to know the tank size. The existing air integrated dive computer are the perfect proof of that. So why would a manufacturer bother adding this? It's a bit like adding a knob that isn't hooked up to anything.

About the only use-case that's left for the tank volume is the gas consumption calculation in the logbook application (e.g. How much gas did I consume on my dive). If the dive computer supplies the tank size, then you no longer have to enter this manually, but that's about it.

And the dive computer can show you your SAC rate during the dive. The Uemis does that.

However, unless you are always diving with the same size of tank, I think it's far more convenient to enter this information in the logbook application than the dive computer UI. I have a Cobalt and use it in combination with different tanks (6L 300bar, 10L 300bar, 12L 200bar, 15L 200bar), but I no longer bother adjusting the tank size in the Cobalt. It's too annoying to change it every time, and usually I just forget to do that anyway. I think it's far more user friendly to setup the tanks in the application, and assign them to your dives there. If you always dive with the same tank, then you can easily setup a default tank that automatically gets assigned to new dives, and then the user experience is exactly the same. Even better, because it also works for devices which don't provide the tank size!

And then I haven't even talked about the different systems to specify the tank volume (e.g. air vs wet volume). There is a reason why libdivecomputer always uses metric units. Mixing different units is only confusing and asking for trouble. I wouldn't mind if I can leave that mess to the applications :-) Luckily there are only a few small countries that hold on to their crazy imperial units :-) (me runs away now)

Humor aside, I continue to be puzzled by this argument. There are a couple of divecomputers that provide these data and there is a clear use case. Whether you use it on your dive computer frankly doesn't matter (and just for the record, I tend to always adjust the tank size on my Uemis). What matters is that there are data that could reasonably be provided by libdivecomputer at zero incremental cost for all the dive computers that don't have it, though for the past two years whenever the conversation comes up you explain why it's not needed.

I agree, gasoline cars work and most cars have a gasoline engine. There are only a few electric car makers out there. Yet as the owner of two such cars I think it would be useful if gas stations and hotels provided a way to charge cars. Oh, and they use different charging ports. One that's a bit awkward but very popular, the other that's better and used by the rest of the world. Smart gas stations and hotels will support both...

But I digress. It's your project. I continue to be too lazy to fork it so I will continue to just work around the parts where you don't see that you continue to go the wrong direction.

/D

On Tue, Sep 02, 2014 at 10:04:01AM +0200, Jef Driesen wrote:

[Sorry for the slow response, but I've been busy with non libdivecomputer related things.]

How dare you have a real life! :-) :-)

I actually took the last few days off myself and did a computer free trip into the mountains.

Remember that adding features ad-hoc, based on what a certain manufacturer (e.g. suunto, which happens to be the first supported devices) provides is pretty much how the current libdivecomputer "design" evolved into what we have now. Linus and you already complained several times about the current libdivecomputer api. Don't get me wrong, I absolutely agree that many of those concerns are indeed valid and should be addressed. But for the time being, I just don't want to repeat those mistakes again. So yes, I'm a lot more careful when it comes to adding new features now.

Which I think is a wise choice. Don't get me wrong, I am not advocating to add any random thing. I'd like at least a couple of dive computers from different vendors to provide the information. Which makes this a tricky one as the Uemis with its ass-backwards download approach isn't supported by libdivecomputer at all. And I'd like at least a vague "OK, this could be useful" check. The Uemis includes rather illdefined values for "cold_fact", "work_fact", and "bubble_fact" in every sample. I wouldn't suggest supporting something like that in libdivecomputer because even if the Uemis was supported... what the heck would you do with these?

Also from an application point of view, I think it's more interesting to have a rather limited set of features that is very well supported by the majority of the backends, then having a large number of features that are only supported by a few backends. That's basically why I have been spending most of my time and energy on improving the existing features, and not adding anything new.

I think we get at least as many "the vendor application supports this, why don't you" email as you do. And then we often go back and look and notice that the vendor application is actually making data up that isn't included in the download (I think you just helped us in one case where the vendor application shows a pressure graph but looking into the sample data it is clear that there isn't even enough space there to hide the pressure data...) - so it's just interpolated data. Same goes for deco information. Sadly very few vendors actually include that in the downloadable data - yet their applications show it by simply "re-doing the math".

But on the flip side - if a computer (or better, a couple of computers) do provide something that I can imagine being really useful (heart rate comes to mind, compass heading, ndl / deco status), then I'll argue to include it even if only a few DCs support it.

But okay, maybe I went into the defensive corner too fast. So let's start all over again, and assume we want to include the tank size too. Can you explain me how you would use the dc_tankdata_t structure in the following scenarios:

• No tank size available (e.g. just begin/end pressure): How do we indicate

that the tank size isn't available?

I see where you are going. The libdivecomputer design pattern would be to return an error when that piece of information isn't availble, yet here we have some data available and others not available.

We could fall back to saying "all values 0" means no data available. If we have neither compressed nor wet volume, we obviously don't know how big the tank is (i.e., while 0°C is a valid temperature, a volume of 0ml is NOT a valid tank volume).

• Metric vs imperial tanks (e.g. water vs air capacity): How do we indicate

the difference? For imperial tanks, we'll certainly need the working pressure. Metric tank also have a working pressure, and although it's not needed for calculating the amount of gas, it may be available too. Do we convert imperial units to metric units to be consistent with the rest of the libdivecomputer api, or do we make an exception here?

With all due respect to my host country, the American way of defining cylinder capacity is beyond braindead. Yes, it seems to make intuitive sense when you tell someone "this is an 80cuft cylinder" (assuming you have wrapped your mind around the non-metric measurements), but of course this only makes sense if you actually know the correct working pressure - and as I frequently find out on dive boats it is not clear at all to a lot of divers that an HP119 and LP95 contain the same amount of gas at the same pressure...

So I would suggest that we have a member for the wet volume in mL. And a different member for the volume at working pressure - also in mL. And a third member for the working pressure - in mbar. (oops, I am assuming the Subsurface style "we prefer integers" approach - you can obviously use doubles and liter and bar - I like integer math better, but I'm fine either way).

Technically one of the three values should of course be redundant - but we have also learned the hard way that it actually isn't. Because the "marketing size" of many cylinders is incorrect. And apparently almost no one takes the compressibility factor into account (at 300bar a tank contains only about 270 times as much air as its wet volume). So if you have all three, give them. If you have only wet volume (or only gas volume and working pressure) set the others to 0.

And then we can work out something from there.

Thanks Jef. Much appreciated.

Sven, Nick - any input on what would be useful for your apps?

/D

On 03-09-14 17:04, Jef Driesen wrote:

On 2014-09-02 22:53, Dirk Hohndel wrote:

...

So I would suggest that we have a member for the wet volume in mL. And a different member for the volume at working pressure - also in mL. And a third member for the working pressure - in mbar. (oops, I am assuming the Subsurface style "we prefer integers" approach

you can obviously use doubles and liter and bar - I like integer math better, but I'm fine either way).

Technically one of the three values should of course be redundant - but we have also learned the hard way that it actually isn't. Because the "marketing size" of many cylinders is incorrect. And apparently almost no one takes the compressibility factor into account (at 300bar a tank contains only about 270 times as much air as its wet volume). So if you have all three, give them. If you have only wet volume (or only gas volume and working pressure) set the others to 0.

Won't it be confusing if all three values are allowed to contain valid data? How do you know if it's a metric or imperial tank? I assume most applications already have logic to deal with this difference anyway. Are there any devices that give us the wet volume for imperial tanks? As far as I know the cobalt doesn't. As you already mention, this conversion may need to take into account the compressibility factor (e.g. ideal vs vanderwaals gas law). I think that's something libdivecomputer should leave to the application.

Wouldn't it be simpler to have either a metric size (wet volume, and optionally working pressure) or imperial size (air volume and working pressure), but never both at the same time. And then a third field containing the type? Something like this:

struct dc_tank_t { unsigned int type; /* metric or imperial */ double volume; /* either wet or air volume depending on the type, but always in liter */ double workpressure; /* in bar */ double beginpressure; double endpressure; };

I would love to get some feedback on this!

Although the discussion on whether libdivecomputer should calculate missing header fields from the samples or not, is an interesting one, it's not a priority at the moment. Initially libdivecomputer will only implement the new tank and temperature fields for devices where such info is available in the header, especially those that do not even have sample data. But I really would like to have this included in the next v0.5 release!

Jef

On 09-09-14 21:27, Dirk Hohndel wrote:

On Tue, Sep 09, 2014 at 01:04:29PM +0200, Jef Driesen wrote:

...

...

Wouldn't it be simpler to have either a metric size (wet volume, and optionally working pressure) or imperial size (air volume and working pressure), but never both at the same time. And then a third field containing the type? Something like this:

struct dc_tank_t { unsigned int type; /* metric or imperial */ double volume; /* either wet or air volume depending on the type, but always in liter */ double workpressure; /* in bar */ double beginpressure; double endpressure; };

How is that simpler than having all three values and figuring it out based on that?

There is certainly a bit of personal preference involved, but I find it more intuitive and descriptive to read something like this:

typedef enum dc_tanksize_t { DC_TANKSIZE_NONE, DC_TANKSIZE_METRIC, DC_TANKSIZE_IMPERIAL, } dc_tanksize_t;

if (tank.type == DC_TANKSIZE_METRIC) { /* Wet volume */ } else if (tank.type == DC_TANKSIZE_IMPERIAL) { /* Air volume */ } else { /* No tank size available */ }

compared to checking the contents of the fields for the magic value zero. It also avoids the problem of the floating point comparisons (see below). And as a bonus, it also very clear that it's either metric or imperial, but never both. Allowing both is only confusing if you ask me.

...

I would love to get some feedback on this!

Although the discussion on whether libdivecomputer should calculate missing header fields from the samples or not, is an interesting one, it's not a priority at the moment. Initially libdivecomputer will only implement the new tank and temperature fields for devices where such info is available in the header, especially those that do not even have sample data. But I really would like to have this included in the next v0.5 release!

I would LOVE if you would switch these new fields to be integers. And I think it would be perfectly fine to have wet_volume, air_volume and working pressure and simply test against 0 to see which one is set. And if for some odd device both are set, have the app figure out what it wants to do.

I have nothing against integers, but everywhere else in the libdivecomputer api we already use floating point values. So at the moment I have a preference towards maintaining consistency and sticking to floating points. The other reason is that storing the wet volume (in liters) in an integer doesn't provide enough precision. The cobalt for example stores the data with a precision of 1/10 liter. We could resort to using milliliters instead, but that would be even more alien to the current libdivecomputer api.

If we decide to switch to integers eventually, then it should be done everywhere, and not just for these new fields. But that's definitely out of scope for the next release.

Jef

On Sep 10, 2014 7:00 PM, "Janice McLaughlin" janice@moremobilesoftware.com wrote:

First, my dc_tankinfo_t looks like this: typedef struct dc_tankinfo_t { float helium; float oxygen; float nitrogen;

..

Ate these actually "float" from the dive computer?

Because we've seen bad problems with floating point values when dive log software used it. Things like saving XML files that have crazy stuff with exponents etc. Or values saved as 24.4999996 instead of 24.5 and things like that. I think I saw this particularly with jdivelog, but maybe I misremember. It's one of the reasons that subsurface trends to try to keep things as integers (ie integer mbar instead of a floating point bar)

Also, nitrogen seems very redundant.

Linus

On 2014-09-11 04:00, Janice McLaughlin wrote:

I've been reluctant to venture into this discussion because I don't have a strong opinion about this aspect directly. But this is because I've already implemented my own strategy because it was missing from libdivecomputer. There would have to be a compelling reason to switch from what I have now, that is working and shipping, to something new and untested. Getting the same information in a different way is not that compelling. Perhaps other application developers are in the same boat. However a later discussion about a different subject is related to this and I think tied in.

First, my dc_tankinfo_t looks like this: typedef struct dc_tankinfo_t { float helium; float oxygen; float nitrogen; float startpressure; float endpressure; float tanksize; float workingpressure; float minPPO2; float maxPPO2; int type; int sensor; } dc_tankinfo_t;

This is simply a direct result of what the UWatec Trimix firmware delivers for each of 10 tanks. So I grab this information from the DC and fill out the structure. Right now I'm only using this for UWatec Trimix firmware and no other manufacturers though as it hasn't been needed. But that may change soon. (BTW - UWatec does NOT give tank size or working pressure however so I don't use those elements and I don't use the type or sensor yet either; the UWatec also has an additional field per tank that is still TBD for me though) (BTW - I could skip Imperial tanksize units altogether too as calling it "braindead" is being too kind)

My point here though is that it also includes the gas mix for each of the tanks. So my only comment is that if you're going to add a new API for tanks, I would make it as expansive as possible and just have a way to indicate "no value" for the fields that are not applicable. That would accommodate the largest range of dive computers.

One of the main problems here, is that most dive computers don't have any knowledge about the tanks, only the gas mixes. That's why libdivecomputer does not even try to match gas mixes with tanks. In most cases, there is simply not enough information to do this correctly. And we're not talking about some odd corner cases, but very common scenarios!

Currently, libdivecomputer can report the list of available gas mixes, and also the gas switch events in the sample data. This is a very simple model that can easily be supported by all dive computers. (I'm aware that the current api for the gas switches has a serious design flaw. It doesn't deal well with multiple identical gas mixes. But that's another story, so let's ignore that for a moment.)

For the air integrated models, libdivecomputer provides the tank pressure in the sample data. Multiple tank pressure sensors are fully supported, because each pressure is tagged with the tank id (which is just a zero based index). Because the tank pressure is fully decoupled from the gas mixes, this model also works well for all dive computers. Non-air integrated models just don't report any tank pressure.

If you want to integrate the gas mixes into the tank info, then you'll run into problems with devices that do not track tank info, but only the gas mixes. And those are still the majority of the devices. Even air integrated models typically only track those tanks for which they have a pressure sensor. The only case where you can do this correctly, are those devices that model gas switches as tank switches. Because there you have a direct association between the tank and its gas mix.

On Sep 9, 2014, at 12:00 PM, Jef Driesen jef@libdivecomputer.org wrote:

...

[To give a bit more context on why I picked the gas mixes as an example. It has been discussed several times already that the current libdivecomputer api for gas switches is far from ideal. A much better design would be to provide the id of the current gas instead of the O2/He percentages directly. That would for example solve the problem that right now you don't really know which mix you switched to, if you happen to have configured two identical gas mixes. But that means libdivecomputer has to collect all gas mixes up front, to be able to assign an id!]

So this is why I think that what you do for tanks now might have a greater impact in the future. Right now libdivecomputer will report gas switches but not tank switches. Based on your explanation of how this came about, this make sense.

That's not entirely correct. There are no explicit tank switch events, but in many (all?) cases the necessary info is already there. As far as I know, all supported devices support at most one active tank sensor. Once you switch to a certain tank sensor, all others are suspended and will no longer record a tank pressure. The result is that in the sample data, you'll only see one tank pressure, corresponding to the active sensor. That means that if you receive a tank pressure sample with a tank id that is different from the previous sample, then that means there has been a tank switch.

In theory it's possible that a device records tank pressure from multiple sensors at once. The libdivecomputer api supports this, and some data formats are in theory also capable of doing this, but I've never seen this in practice.

Data from a dive computer with multiple pressure sensors is difficult to find. If anyone has such a setup, I would love to get some real data from it!

However, I would like to see the addition of tank switches to accommodate the dive computers that use this concept. And - you can model gas switches AS tank switches if you want to but you can't model tank switches as gas switches.

So, you mention that ostc and nitekq use the gas switch model. And so does Shearwater, although you can get the list of gases up front. But Atomic, Liquivision, Pelagic, and UWatec all have a tank switch model. I'm not sure about Suunto as I don't have good switching data, but it *seems* like it's tank based but could be gas based. Same for the newer Mares. But right now, if a diver switches from one tank to another with the same mix in it, it may not get recorded. In any case, I think there is a good argument for also having tank switch capability in libdivecomputer for more accurate information from the DC.

Anyway, I suggest that libdivecomputer could keep the existing DC_FIELD_GASMIX_COUNT and DC_FIELD_GASMIX API, and could also add a DC_FIELD_TANK_COUNT and DC_FIELD_TANKINFO interface, and then that would nicely give id's and a way to get SAMPLE_EVENT_TANKCHANGE events in the sample stream for those DC's that support it.

The problem of not being able to detect a switch to another tank with the same gasmix is due to the fact that the gas change events carry the O2/He percentages instead of a gasmix index. This is a design flaw that I intent to fix after v0.5. It's not directly related to not having tank switches.

When I read your email for the first time, I thought introducing tank change events would indeed be a good idea. But the more I think about it, the more I doubt whether that's the right solution.

For devices with a tank based switch model, and those are the ones where this info would be useful, the tank switches and gas switches will always coincide. So you kind of get this information already. You just don't know whether you have such device or not, because libdivecomputer doesn't tell you that. So maybe what we really need here is a way to tell the application that there is a one to one mapping between the tanks and the gas mixes? One property of these devices with a tank based switch model, is that the number of tanks will always be equal to the number of gas mixes. Maybe that's something the application can use for that purpose?

Also, assuming we introduce tank switch events in the samples, then the application still has to work out which gas mix is associated with a particular tank. But even while this association is fixed for the entire dive, you'll have to this by manually processing the tank and gas switches in the samples. That's a bit odd if you ask me. It also means you'll only be able to do this for those tanks and gas mixes that have been used on the dive. Maybe it's a better idea to include the id of the gas mix in our tank structure?

struct dc_tank_t { unsigned int gasmix; ... };

When we don't have enough information to provide this info, we set the field to some predefined value to indicate "unknown". Unfortunately we can't use zero here. The gasmix index is zero based, so zero would be a valid value. But something like 0xFFFFFFFF should do:

#define DC_GASMIX_UNKNOWN 0xFFFFFFFF

The key point here is that we keep the gas mixes outside the tank structure. We need that in order to support devices that don't track tanks. But for devices where there is a known one to one mapping between tanks and gas mixes, we can also provide this info to the application.

(I'm just thinking aloud here, feel free to join!)

I have a USB analyzer that I needed for my driver work. If someone can deliver one of these units into my hands, I'd be happy to take a look and take some traces. And we could go from there.

For capturing USB communication there are also software solutions. That's basically what I have been doing all those years for serial communication. It's certainly a bit more practical than having to send an (expensive) dive computer. But if the owner doesn't mind doing that, that's certainly also an option.

Jef

On 2014-09-14 04:03, Janice McLaughlin wrote:

My thought was not to replace the existing Gas Switch model at all, but just to add the tank model for those dive computers that it makes sense for. So perhaps a ostc and Shearwater return 1 or 4(8) for DC_FIELD_GASMIX_COUNT and return 0 for DC_FIELD_TANK_COUNT. And they would continue to put gas switch events, (with id's probably to fix the problem you reference) in the sample stream. But an eg: Atomic would return 0 for DC_FIELD_GASMIX_COUNT and would return 3 for DC_FIELD_TANK_COUNT. And the corresponding dc_tank_t would have the tank capacity (where this all started) and starting and ending pressure in it, just like the DC itself does. All of the DC's that support a tank model seem to have all of that information available about each tank, including gas mix.

That's not entirely correct. There are also air-integrated dive computers, that support gas switches, and also record tank pressure, but which do not follow the tank switch model.

Take for example the Suunto Cobra 2 (just to name one). It's an air integrated model (hose), which supplies tank data in the sense that it records tank pressure, but without any info about the tank size. Being a hose based model, it can of course only support a single tank sensor. It also support multiple gas mixes, and the gas switches are recorded using the gas switch model.

So in this case, you can't really replace DC_FIELD_GASMIX with DC_FIELD_TANK, because there are more gas mixes than tanks. Of course those extra gas mixes are also stored in a tank, but we just don't have any info about that. We don't even know which gas mix is in the tank connected to the tank sensor. Most likely it will be the main tank, and the other gas mixes will be some deco mixes in stages. But we can't guarantee that. So the DC_FIELD_TANK is still needed for the tank pressure, but there should be no gasmix inside.

That's the main reason why I proposed to keep the DC_FIELD_GASMIX as the main (and only) interface for the gas mixes. The DC_FIELD_TANK would be an optional interface for devices which also provide some info about the tank (tank pressure, tank size, etc). And if the exact gas mix is known (e.g. for tank switch based devices, or if the device provides this info in some other way), you would also get its id there. Thus there is no need to also include the actual O2/He percentages in the dc_tank_t structure.

With my proposal, the cobalt parser would return the same value for both DC_FIELD_GASMIX_COUNT and DC_FIELD_TANK_COUNT (note that the newer cobalt 2 supports more than 3 gas mixes). The gasmix field in the dc_tank_t struct would contain the id of the corresponding dc_gasmix_t structure.

...

In theory it's possible that a device records tank pressure from multiple sensors at once. The libdivecomputer api supports this, and some data formats are in theory also capable of doing this, but I've never seen this in practice.

Data from a dive computer with multiple pressure sensors is difficult to find. If anyone has such a setup, I would love to get some real data from it!

FYI The Liquivision Lynx allows you to monitor the air supply of up to 10 other divers (in addition to your own) and give's pressure samples for each throughout the dive. Now, why you would want to "log" someone else's pressure samples is a good question :-) But I'm assuming that you can use some of this capacity to get pressure data from more than one of your own tanks.

With wireless sensors it's certainly possible to use multiple sensors with a single dive computer. I assume several libdivecomputer supported models (from Uwatec, Mares, Uwatec, etc) already support that. I just don't have data where that feature is being used.

Some Oceanics are also capable of monitoring someone else' tank pressure. Although that might be a useful feature during the dive, I also don't really see the use-case from a logging point of view.

...

The problem of not being able to detect a switch to another tank with the same gasmix is due to the fact that the gas change events carry the O2/He percentages instead of a gasmix index. This is a design flaw that I intent to fix after v0.5. It's not directly related to not having tank switches.

Seems like a good idea to instead put the gas mix id in the event.

Indeed. Unfortunately we can't fix this right now, without breaking all existing applications. So this will have to wait until after the v0.5 release :-(

...

When I read your email for the first time, I thought introducing tank change events would indeed be a good idea. But the more I think about it, the more I doubt whether that's the right solution.

For devices with a tank based switch model, and those are the ones where this info would be useful, the tank switches and gas switches will always coincide. So you kind of get this information already. You just don't know whether you have such device or not, because libdivecomputer doesn't tell you that. So maybe what we really need here is a way to tell the application that there is a one to one mapping between the tanks and the gas mixes? One property of these devices with a tank based switch model, is that the number of tanks will always be equal to the number of gas mixes. Maybe that's something the application can use for that purpose?

Also, assuming we introduce tank switch events in the samples, then the application still has to work out which gas mix is associated with a particular tank. But even while this association is fixed for the entire dive, you'll have to this by manually processing the tank and gas switches in the samples. That's a bit odd if you ask me. It also means you'll only be able to do this for those tanks and gas mixes that have been used on the dive. Maybe it's a better idea to include the id of the gas mix in our tank structure?

struct dc_tank_t { unsigned int gasmix; ... };

But to address your point of decoupling the gas mix from the tank information .... this would not be *my* preference. That would mean that the sample stream would return both a gas change AND a tank change event every each time there was a tank change. And, as you suggest, you'd have to put the gas mix id inside the tank information ..... and that just seems confusing. I'd rather see the gas mix inside the tank info and only 1 event in the stream. Which may or may not mean a gas change as well depending on what's in the tank.

No, I think you misunderstood me. The idea is that in the samples, there will be no tank changes, only gas changes. For a device using the tank switch model, the gasmix field provides a fixed one to one relationship between the dc_tank_t structure and the corresponding dc_gamix_t structure. So you don't really need a tank switch.

For devices using a true tank switch model like the cobalt one, the number of dc_tank_t and dc_gasmix_t structures will always be identical. Thus if the id's is a simple zero based index, then the id in the gas change event can be directly mapped to the tank structure.

...

When we don't have enough information to provide this info, we set the field to some predefined value to indicate "unknown". Unfortunately we can't use zero here. The gasmix index is zero based, so zero would be a valid value. But something like 0xFFFFFFFF should do:

#define DC_GASMIX_UNKNOWN 0xFFFFFFFF

The key point here is that we keep the gas mixes outside the tank structure. We need that in order to support devices that don't track tanks. But for devices where there is a known one to one mapping between tanks and gas mixes, we can also provide this info to the application.

(I'm just thinking aloud here, feel free to join!)

Again, I would not change anything for devices that "don't track tanks" and continue to do what you are doing. And for DC's that have a one to one mapping, then put the gas mix inside the dc_tank_t.

How do you deal with the "hybrid" case that I explained above (e.g. Suunto Cobra 2)?

I think having the gasmix in either the DC_FIELD_GASMIX or the DC_FIELD_TANK interface, depending on how the device stores it internally, will be confusing. It basically forces an application that is interested in the gas mixes, to always check the DC_FIELD_TANK interface too. That's counterintuitive is you ask me.

This would mean that none of the applications would need to change much. I think that each application that is using libdivecomputer will also have their own model for how they communicate this information to the end user. I think if libdivecomputer gives more "hints" about the DC itself and how it works, then it gives the application more flexibility on how to give the information to the end user and map it to their own data model.

...

...

I have a USB analyzer that I needed for my driver work. If someone can deliver one of these units into my hands, I'd be happy to take a look and take some traces. And we could go from there.

For capturing USB communication there are also software solutions. That's basically what I have been doing all those years for serial communication. It's certainly a bit more practical than having to send an (expensive) dive computer. But if the owner doesn't mind doing that, that's certainly also an option.

You've done this enough that perhaps you could write a new backend for a new device simply based on some USB traces. I just meant that *I* could not do this :-) There was talk of trying to get more people to contribute to the project and so if I were to contribute a new backend to libdivecomputer, I could reverse engineer it but I would also need one in my hands to test with.

Ah ok. I misunderstood. I thought you suggested to capture the communication and then send the files to someone else for analysis. In that case, the person doing the reverse engineering still has no physical access, and thus it makes no difference how the data was captured. But if you do everything yourself (capture, analyze, implement and test), then that's another story or course.

I've always relied on the device owner to capture the data and run the tests for me, so I guess I'm already used to that. I think the only device I have reverse engineered with physical access is the Diverite Nitek Q.

Jef

On 2014-09-22 04:04, Janice McLaughlin wrote:

On Sep 16, 2014, at 11:26 AM, Jef Driesen jef@libdivecomputer.org wrote:

...

On 2014-09-14 04:03, Janice McLaughlin wrote:

...

But to address your point of decoupling the gas mix from the tank information .... this would not be *my* preference. That would mean that the sample stream would return both a gas change AND a tank change event every each time there was a tank change. And, as you suggest, you'd have to put the gas mix id inside the tank information ..... and that just seems confusing. I'd rather see the gas mix inside the tank info and only 1 event in the stream. Which may or may not mean a gas change as well depending on what's in the tank.

No, I think you misunderstood me. The idea is that in the samples, there will be no tank changes, only gas changes. For a device using the tank switch model, the gasmix field provides a fixed one to one relationship between the dc_tank_t structure and the corresponding dc_gamix_t structure. So you don't really need a tank switch.

For devices using a true tank switch model like the cobalt one, the number of dc_tank_t and dc_gasmix_t structures will always be identical. Thus if the id's is a simple zero based index, then the id in the gas change event can be directly mapped to the tank structure.

I see. And the pressure samples will also use the same id (tank) in the future then too?

Yes, that's the idea.

...

...

Again, I would not change anything for devices that "don't track tanks" and continue to do what you are doing. And for DC's that have a one to one mapping, then put the gas mix inside the dc_tank_t.

How do you deal with the "hybrid" case that I explained above (e.g. Suunto Cobra 2)?

I don't think the Suunto Cobra 2 is a hybrid - it seems like the base configuration of the majority of dive computers. One supply of breathing gas that the user and the DC have to agree upon in advance. In this way, it's no different from an Atomic. And while diving, the Cobra2 will put information about gases in the sample data and the Atomic will put information about tanks in it's sample data. But in this simplest case which is the majority of what everything out there is, it just becomes semantics as to whether it's a gas switch or a tank switch at the beginning and there is no point in getting into semantics. But for the more advanced DC's, then I think we should have a scheme that expands to support them. Keeping the gas and tank info separate is "fine", not my preference, but I can live with it as long as all the id's match up.

Hybrid was maybe a bad term to refer to the Cobra2. I just wanted to point out that it supports multiple gas mixes and tank info, but no information to link the two together.

Anyway, let me illustrate what kind of info you would get for the three main type of devices, by means of a few examples:

1. Non-air integrated model (OSTC)

Gas mixes: 0: Air 1: Nx32 2: Nx36 3: ... 4: ...

Tanks: None

In the samples, a gaschange event will reference the gas mix using the corresponding id. (Right now the event contains the O2/He percentages directly, but let's assume we have fixed that already.) Since this is a non-air integrated model, there are no tanks reported, and also no pressure samples.

2. Air integrated model, gas switch model (Cobra2)

Gas mixes: 0: Air 1: Nx32 2: Nx36

Tanks: 0: gasmix=UNKNOWN, volume=xxx, pressure=yyy, ...

For the gas mixes and gas change events in the samples nothing changed at all. Because we now have an air integrated model, there will be one or more tanks available. The tank pressure samples do not only contain the tank pressure, but also the tank id. That's already the case today in order to support multiple tank sensors per sample, so again nothing really changes here.

However, due to the gas switch model, we don't have any info about which gas mix is associated with the tanks, so we simply leave the gasmix field blank (UNKNOWN). It's up to the application (or the end-user) to fill in the missing info. Libdivecomputer won't try to guess it.

2. Air integrated model, tank switch model (Cobalt)

Gas mixes: 0: Air 1: Nx32 2: Nx36

Tanks: 0: gasmix=0, volume=xxx, pressure=yyy, ... 1; gasmix=1, volume=xxx, pressure=yyy, ... 2: gasmix=2, volume=xxx, pressure=yyy, ...

This is a very similar case to previous one, except that due to the tank switch model, there is now a clear one-to-one correspondence between tanks and their gas mixes. So we can now link each tank with its gas mix. The gasmix and tank id's are in fact identical because the number of gas mixes and tanks are by definition identical. Thus, although a gas change event doesn't directly carry any info about the tank switch, you can easily figure that out by looking at the gas mix id in the tank structures.

I believe this results in a consistent api, that can be implemented for all types of devices, no matter how they store their tank or gas mix info internally.

My proposal is now as follows:

typedef enum dc_field_type_t { ... DC_FIELD_TANK_COUNT, DC_FIELD_TANK, } dc_field_type_t;

#define DC_GASMIX_UNKNOWN 0xFFFFFFFF

typedef enum dc_tankvolume_t { DC_TANKVOLUME_NONE, DC_TANKVOLUME_METRIC, DC_TANKVOLUME_IMPERIAL, } dc_tankvolume_t;

typedef struct dc_tank_t { unsigned int gasmix; /* Index of the gas mix, or DC_GASMIX_UNKNOWN */ dc_tankvolume_t type; double volume; /* Either wet or air volume depending on the type, but always in liter */ double workpressure; /* Pressure in bar. */ double beginpressure; double endpressure; } dc_tank_t;

PS: To fix the gaschange event issue right now, we can consider adding a new SAMPLE_EVENT_GASCHANGE3 variant that does not contain the O2 and He percentages, but the gasmix index. Adding the new event doesn't break backwards compatibility, and thus we can slowly start porting the backends to this new model. Of course the downside is that applications that are not updated to recognize this new event will no longer get gaschange events from backends that already moved to the new event.

Jef

On Thu, Sep 04, 2014 at 09:22:08AM +1200, Nick Shore wrote:

Ultimately I think I would prefer a consistent api. That said, currently I have custom parsers as I extract a little extra info in most cases. The Cobalt tanks are a good example of that. So while I say a consistent api, I'm doing something completely different in practice. In a perfect world, the application wouldn't need to handle anything manufacturer/model specific. If you could query whether information was available for a certain brand or model, then that would help.

Certainly I get a lot of people asking why the manufacturer software shows X, and where is that in MacDive? I've seen the "fake" pressure graphs you're talking about, and that trips users up. This particular case I'd say is pointless at best, and misleading at worst. I think that information is best not displayed. I don't think that libdc should be trying to fake this information at all if it is ending up with data that is in no way accurate.

I do agree with you Dirk that if enough devices support a particular feature then perhaps it's worth adding. As long as there's a convenient way for an application to say "does this model include heart rate information?" then I think it can be included in a way that applications can handle it nicely. I guess I straddle the middle ground, as I do have custom parsers so that I can extract some info that isn't currently handled by libdc, but would ultimately like to not have to maintain them.

I trust that this has not helped in any way :)

I think it helped very much.

Here's what I learned from your response:

- MacDive, just like Subsurface, ended up implementing custom parsers for information that isn't extracted by libdivecomputer. - MacDive, just like Subsurface, would prefer to get such data from libdivecomputer, as long as it is easy to tell if the data is available or not - so no made up, interpolated or otherwise fabricated data. Either an indication "the data aren't available", or actual data from the dive computer.

/D

On 04/09/2014, at 17:14, Jef Driesen jef@libdivecomputer.org wrote:

Almost right. The original reason is a bit different. Macdive is a few years older than subsurface, and started around the same time as libdivecomputer itself. At that time the libdivecomputer parser was pretty much non-existing or at least even more basic than it is now (e.g. sample data only). So writing a custom parser was basically the only option. Nowadays the libdivecomputer has much improved (although their is certainly room for improvement!) and while Nick and I discussed switching to the libdivecomputer parsers more than once, that just didn't happen yet. It works, and there have been always some more urgent things on the todo list :-)

Jef is right there, it's primarily that there are other more urgent things to do. There aren't too many special cases left these days. Off the top of my head, one is the surface interval - but this is an interesting one. In the cases it's in the data, I read it. But in the cases it's not, I generate it afterwards. Which is fine since I know the previous dives. libdc might have a hard time generating it if you're only downloading the latest couple of dives and had no further context to tell if a dive was part of a series.

Another one was the name of the diver that is available on some devices. I believe Jef and I have argued over that one in the past :)

While we're talking about accessing values, dare I ask for the serial number to be a char* again..

We're mostly in sync, but sometimes we have minor differences (or different bugs if you prefer :-)) here or there.

...

• MacDive, just like Subsurface, would prefer to get such data from libdivecomputer, as long as it is easy to tell if the data is available or not - so no made up, interpolated or otherwise fabricated data. Either an indication "the data aren't available", or actual data from the dive computer.

It's hard to disagree with that!

Yep.

-nick

On 04-09-14 16:23, Dirk Hohndel wrote:

On Thu, Sep 04, 2014 at 05:34:29PM +1200, Nick Shore wrote:

...

Jef is right there, it's primarily that there are other more urgent things to do. There aren't too many special cases left these days. Off the top of my head, one is the surface interval - but this is an interesting one. In the cases it's in the data, I read it. But in the cases it's not, I generate it afterwards. Which is fine since I know the previous dives. libdc might have a hard time generating it if you're only downloading the latest couple of dives and had no further context to tell if a dive was part of a series.

And again - I wouldn't WANT libdivecomputer to calculate it. I want libdivecomputer to give me JUST the things that come from the dive computer. And nothing more. If it needs to be calculated, I can calculate it just as well.

And no, I don't mean this in a dogmatic way. Of course libdivecomputer should do unit conversions and other normalizations. What I mean are things like fake tank pressures, surface intervalls that weren't reported by the dive computer, deco that wasn't reported by the dive computer, etc.

I think that is exactly what libdivecomputer does today. It just is NOT reporting some values that the dive computer might provide... and I think we should add some of those :-)

Libdivecomputer will never try to calculate fake tank pressure, or any other sample data. That would indeed be a very stupid idea.

But what about header data, like begin/end pressure, temperature, maxdepth, divetime, etc? There are several devices where such information isn't available from the header, but where it's (indirectly) in the sample data. In that case libdivecomputer can calculate the missing info from the sample data. I already do that in a few places for divetime and maxdepth.

From my point of view doing this provides a more consistent user-exerience for the application developer. If libdivecomputer already takes care of falling back to using the sample data, then the application can just get the value. The fact that one device stores it explicitly and another doesn't, would become an implementation detail.

Note that I'm writing this with the future in mind. I'm not going to implement such a fallback immediately. The priority will always be to implement it first for those devices that actually store it in the header (and maybe don't even have the sample data).

...

Another one was the name of the diver that is available on some devices. I believe Jef and I have argued over that one in the past :)

Yes, Uemis gives you that. Plus the suit that the user used. Plus the GPS coordinates and name of the dive site. The amount of weight that the diver carried. And a dozen others. There clearly is a point of diminishing return here. I will usually err on "I'd rather have it returned by libdivecomputer", but on some of those I certainly would agree with Jef. E.g. diver name... for 99% of the cases what would the consumer (the dive log software) do with that information? Tell the diver "these aren't your dives"?

...

While we're talking about accessing values, dare I ask for the serial number to be a char* again.

:-)

Of course you may ask ;-)

Jef

On 05-09-14 18:46, Linus Torvalds wrote:

On Sep 5, 2014 9:27 AM, "Jef Driesen" jef@libdivecomputer.org wrote:

...

But what about header data, like begin/end pressure, temperature, maxdepth, divetime, etc? There are several devices where such information isn't available from the header, but where it's (indirectly) in the sample data. In that case libdivecomputer can calculate the missing info from the sample data. I already do that in a few places for divetime and maxdepth.

I can mind of see the point of doing max depth and time, just to always have those fields in the header data for a dive.

But cylinder pressures etc? Bad idea. They aren't always there anyway, claiming them just because you also have sample data is just wrong, stupid, and print to error (ie the cylinder connected to the pressure sensor may not be the first one, you don't know, stop gues

How can you not have begin/end tank pressure, when the samples contain tank pressure values? You are right that not every tank may have a pressure sensor connected, but why should that be a problem? You'll only get pressure information from tanks with a pressure sensor. Usually your dive computer isn't even aware of those other tanks. In that case you typically configure the gas mix, and the dive computer doesn't even care about the tanks at all. That's actually the reason why libdivecomputer will not try to associate gas mixes to tanks, because in most cases it doesn't have enough information. But here we are only talking about pressure sensors, not gas mixes.

In your example, where the pressure sensor is not connected to the first tank, then indeed libdivecomputer won't have that info. As far as libdivecomputer is concerned there would only be a single tank. Or more, if there are multiple sensors. But in that case libdivecomputer will know which pressure value belongs to which sensor. Because if that's not true, then the data format would be really broken. So the concept of the first (or second, etc) tank doesn't really apply here.

Assuming (just theoretically) that libdivecomputer would calculate begin/end pressures for devices which don't store those values in the dive header, then we would just go through the samples. For each tank sensor find the first and last sample with pressure data for that sensor and return those as the begin/end pressures. How can that go wrong? Am I missing something here?

So don't add implied data. Even max depth and time is debatable (max time based on samples should take surface time into account, do you? Probably not), anything else is just really wrong.

Most dive computers store the dive time in the header, so this isn't really an issue. But for those that don't, the way libdivecomputer calculates it, depends on the dive computer. Some exclude surface time, others don't. I try to match with the manufacturers application, because that's usually also what is shown by the dive computer. Most of the time they can easily omit the divetime (to save a few bytes of storage I assume) because they just show number of samples multiplied with the sample interval. So in a sense the divetime is there, just not stored directly as a number of seconds.

Anyway, if you want the dive time to be consistent, independent of the dive computer, then you should ignore the value and always calculate the time yourself. Because in most cases, we simply don't know the exact details of how the dive computer calculates the dive time (e.g. does it exclude shallow samples, what's the threshold, etc). If you ask me, there isn't really a correct answer either.

I think most people expect to see the same value as reported by their dive computer, so that's what libdivecomputer tries to provide.

If you can do it from sample data, then the real application that probably can do better and more (like correlate "multiple* she computers etc). Your calculated value at best might be just wrong, at worst they mess up a better calculation.

Give what the dive computer gives. Nothing more. Nothing less.

Given my explanation above, I think what libdivecomputer does today is pretty much the way you want it.

Jef

On 06-09-14 20:46, Linus Torvalds wrote:

On Fri, Sep 5, 2014 at 2:58 PM, Jef Driesen jefdriesen@telenet.be wrote:

...

...

I can mind of see the point of doing max depth and time, just to always have those fields in the header data for a dive.

But cylinder pressures etc? Bad idea. They aren't always there anyway, claiming them just because you also have sample data is just wrong, stupid, and print to error (ie the cylinder connected to the pressure sensor may not be the first one, you don't know, stop gues

How can you not have begin/end tank pressure, when the samples contain tank pressure values?

You may not *have* sample pressures, because maybe you don't have a transmitter.

Well, if there is no sample pressure, then of course libdivecomputer will never make up such data. That's just stupid.

For the tank pressure (and other fields) there are 4 different cases:

1. No header, no samples: This is probably a non-air integrated model. We won't try to provide any pressure data whatsoever.

2. Header, but no samples: This is typically an older or lower-end air-integrated model, which doesn't have enough memory to store a full pressure profile. We provide just the begin/end pressure from the header. We won't try to interpolate sample data.

3. Samples and header: This is a device that gives us everything. Nice and easy.

4. Samples, but no header: This is basically a special case of the previous one, except that the device tries to save some memory by not storing redundant information. The begin/end pressure can easily be obtained from the sample data. This is the category where this discussion is all about.

The point I'm making is that the user of libdivecomputer cannot possibly _depend_ on having beginning/ending cylinder pressure data.

So there is no point in trying to make such data up. It doesn't buy the library user anything, and you almost certainly make things *worse*, not better.

In other words: you add no actual information, you add nothing that is valuable, but you *do* add confusion.

If the pressure data is in the samples, then subsurface (or any other user) can - and will - look at the sample data. If you start reporting beginning and ending pressures separately that you just take from the sample data anyway, the *only* thing you do is to make for a more complex download with more room for errors.

Why the f*ck would you want to do that? It's stupid. It's wrong. It doesn't _help_. It only hurts.

Now, if the dive computer *separately* reports data like that, in addition to the data in the samples, then by all means, pass that kind of information through. But don't make it up based on other data.

An example of that is "maxdepth", where many dive computers (all?) report maxdepth as a separate thing from the depth data in the profiles. And it doesn't always match - the dive computer may save sample data at some particular frequency that is almost certainly separate from the actual internal sampling frequency. So in this case, maxdepth may actually have a *separate* value from the maximum depth found in the samples. THEN it is real data.

But if it's just something you figured out from the samples, you SHOULD NOT LIE to the application and tell it that the dive computer reported it.

Get it? You'd be *lying* about the dive computer. You'd be making shit up. And you'd almost certainly be *worse* at it, than we are. You only look at one dive computer at a time, subsurface will not. You don't have any way to match pressure sensors to particular cylinders, but subsurface might (ok, we don't, but at least we track it and there's a chance we might allow user editing etc). The moment you say "let me make things up based on other things", you're basically just making it harder to use the library sanely, because at that point we can no longer trust the data you give us to be what the dive computer reports. See?

I'd much rather *not* get begin/end pressure data, and know that that means "the dice computer does not save that separately" than get made-up pressure data that I then don't know if it's the dive computer that has such things, or just Jef that is making shit up again. See the difference between the two cases?

(And this is not just about pressure data. This is about temperature, about depth, about anything. Give us what the dive computer reports, not some "sanitized view" that means that we don't know what is dive computer and what is some random library version.

I think the main reason why we have a different opinion here, is because we see the role of libdivecomputer a bit different. You prefer a "minimal" library that just provides access to the data that is explicitly stored by the dive computer. So libdivecomputer only abstracts away where (byte offset) and how (little/big endian, metric/imperial, etc) the data is stored and that's it. That used to be my opinion too, but nowadays I think that ideally (and we're talking about the long term goal here) it should go one step further: libdivecomputer gives you an abstract representation of a dive, and the application can just get the info, no matter where or how it is stored (explicitly in the header or implicitly in the samples).

From my point of view, the main benefit is that the application side becomes more simple. Instead of requiring all applications to implement their own fallback code, it would all be centralized in libdivecomputer. Then libdivecomputer could guarantee things like if the device has a pressure sensor, then you can just get the begin/end pressure. If there is a full pressure profile, then great, you can get that too, but that's optional. (Again, this is for the long term goals.)

You consider this as making things up or lying, I don't. For things like begin/end pressure, maximum depth there is no real calculation involved. It's basically collecting some trivial "statistics" on the sample data. How would an application be able to calculate this different or better? The input data is exactly the same! Yes, you are right that if a user has multiple dive computers, then the application may have a bit more info. But in that case, you'll have to merge the data somehow anyway. The maxdepth (or other header data) from two devices will rarely be exactly the same, and that's true regardless of whether you take maxdepth from the samples or dive header.

I'll try to illustrate my point with a complete different piece of date: the gas mixes. Most dive computers store the available gas mixes in the header. But there are several exceptions to this rule. The ostc has an extra manual gas that can be changed during the dive. The nitekq has no gas mixes in the header at all, only gas switches in the sample data. What should libdivecomputer return when subsurface (or another application) calls dc_parser_get_field with the DC_FIELD_GASMIX? Just the gasmixes in the header? For the ostc or nitekq that means you would miss some or all the gas mixes. Or should we parse the sample data and also pick up the manual gas (ostc) or process the gas switches (nitekq)? Do you also consider this making things up or lying?

If you ask me, libdivecomputer should do the latter, because this is just some internal detail of how the device records the gas mix data. The end user or the application shouldn't need to be aware of such things. I just don't understand why this would be any different than collecting begin/end pressure or maxdepth from the sample data. Where do you draw the line?

[To give a bit more context on why I picked the gas mixes as an example. It has been discussed several times already that the current libdivecomputer api for gas switches is far from ideal. A much better design would be to provide the id of the current gas instead of the O2/He percentages directly. That would for example solve the problem that right now you don't really know which mix you switched to, if you happen to have configured two identical gas mixes. But that means libdivecomputer has to collect all gas mixes up front, to be able to assign an id!]

I'm also not convinced that the application can always calculate missing data better than libdivecomputer can, as you claim. Libdivecomputer can easily apply device specific knowledge, which is much harder (or at least annoying) to do on the application side. Take for example the sensusultra. It records no divetime (or maxdepth, etc) in the header, only depth and temperature samples. But it also has a user configurable depth threshold and endcount setting to determine when a dive is considered over. (I assume all dive computers use a similar logic, but with the settings hardcoded.) The consequence is that the tail of the dive always consists of endcount samples with a depth shallower than the threshold. Most users will likely prefer this tail to be excluded from the divetime. Libdivecomputer can easily take these settings into account (although today it doesn't do that yet). Do you really want such device specific knowledge on the application side?

Note that I agree that for certain fields, like divetime, there are indeed multiple ways to calculate it. But as I already explained in my previous email, there is not really a correct answer, and if the goal is to have a consistent value across different devices then you should probably always ignore the value from the header and calculate it from the samples.

Jef