254 seems to be the maximum value of Unsigned Byte datatype in (at least) OM 9.10
I've been using OM 9.10 on Windows 10, and I noticed some weird results of Channel Math functions when involving Unsigned Byte channels of a database.
I am aware that the unsigned byte range is 0 to 255. However, in a database of Oasis Montaj it seems to be 0 to 254. I used several ways to test it, just getting similar outcome.
A simple example of the issue: typing 255 as a value for a UByte channel record, one gets 254 instead. (my wrong results were initially generated by Channel Math, and it was quite time consuming to get to the actual cause)
Was there an actual reason behind this behavior or just an aspect not being noticed because of not many affected yet?
Dumitru
Comments
-
Hello Dumitru;
Indeed there is a valid reason. Apologies if it is not being spelled out in the documentation. We will update the documentation.
By Geosoft convention the extreme value of a data type represents the DUMMY value for that data type. If the type is signed then the minimum limit of the data type represents the DUMMY value, if it is unsigned the maximum value of the data range represent the DUMMY value. As a result these extreme values are treated differently by math functions.
You will find the definitions of the Oasis montaj data ranges from line 273 - 373 of the attached header file. This file is included in the GX-developer download.
While on the topic of channel math, may I ask if you have used the offset functions, they expand the capability of channel math quite a bit.
0 -
Thank you Elizabeth for this clarification and quick response. It makes perfect sense, and now I can address my workflows accordingly.
I missed the connection between the dummy values outlined in the grid format help page (or in the GX developer docs) and the database values. I was actually playing with colors (to control individually R,G,B components), and in that context the math operations hid well the UByte dummy case - e.g. the result was not Dummy, now clearly explainable as a combination of factors involving 'if's, internal type conversions etc.
I guess a help page just for Dummy values might be very handy. But yes, I can see that this sort of details might not be easy to make the user well aware of, ahead of exposing it in a real case.
Regarding the offset functions you mentioned, I have been using them frequently, and to me they are extremely useful indeed. I am not sure if there was any addition to them after 9.10, but one that I would have loved dearly was to be able to use a variable as argument, something like C0 = C1.offset(C2) or C0 = C1.offset(@TempIdxVal). The same stands for .at(). This is a separate topic, but I use this opportunity to suggest it, if not already present in the latest OM version. It should be an easy implementation since C1 and C2 channels for the current line would be anyway in memory and fully accessible, and, for @TempIdxVal case, the value already calculated. That small addition would bring significant benefit to database math operations.
0 -
Dumitrulon,
I am glad you find Om' s DUMMY convention logical. We are already looking at Om's help content to imbed the info where it is in context.
I am also glad that you find Om's math tools so useful, indeed you can do so much with them. Your thought on supporting a variable index in offset operations has been in our backlog, and we have investigated it in the past. The point is that it is more involved than it may appear. The math engine dll has a specific and complex preset structure; the structure nodes are populated with the user entries then fed into the dll. To allow it to take in variable indices requires a change to this structure which requires a deeper dive than a mere replacement of a constant with a variable.
0