29 Replies Latest reply on Feb 17, 2017 3:17 PM by Paul_Booth

    No. of decimal places in returned value

    bhiggin2

      I am building elements in the AF Explorer and assigning a PI point to the attribute, the question I have is why a whole number returned rather than the actual PI value e.g. 2.04 and why are only postive values returned.

       

      AF.JPG

       

       

       

      Barry

        • Re: No. of decimal places in returned value

          Your value type is set to Integer, so you would only expect a rounded whole number.

            • Re: No. of decimal places in returned value

              That's correct, you should be using a Floating point number type (single or double) rather than and Integer number (Int16, 32 or 64).

               

              In case you are not very familiar with AF and you got stopped by the fact these fields are read-only, note that you will probably have to change the data type directly in the AF Template that this AF Element is based off on. Go to Library in the PI System Explorer and look for Templates > Element Templates.

               

              Hope this helps!

                • Re: No. of decimal places in returned value

                  I am going to hijack Barry's thread...

                   

                  On the flip side, is there any reason why formatting a value via the PI Point DR was left out?  I would have like to have seen either the DisplayDigits Point Attribute have an effect or a Format Number dropdown added to the configuration dialog.  At the moment you can only format a number in an Attribute by having a second Attribute (Formula using RoundFrac) for a PI Point Attribute.

                    • Re: No. of decimal places in returned value
                      bhiggin2

                      Rhys @ RJK Solutions

                      I am going to hijack Barry's thread...

                       

                      On the flip side, is there any reason why formatting a value via the PI Point DR was left out?  I would have like to have seen either the DisplayDigits Point Attribute have an effect or a Format Number dropdown added to the configuration dialog.  At the moment you can only format a number in an Attribute by having a second Attribute (Formula using RoundFrac) for a PI Point Attribute.

                       

                       

                      I also would like to see this, it seems like a feature that's crying out to be included in AF 2.3.

                        • Re: No. of decimal places in returned value

                          Rhys @ RJK Solutions

                          is there any reason why formatting a value via the PI Point DR was left out?
                          Yes: this is a server-side setting whereas the number of decimals one wants to display is more of a client-side thing. Just like the PI Points' DisplayDigits attribute is not really used anymore, in favor of display settings in PI ProcessBook, Microsoft Excel, or any other client application.

                           

                          But I'm sure the AF product manager and the AF development team would happy to reconsider it if you bring a compelling use case we didn't think of...

                            • Re: No. of decimal places in returned value

                              Particular case I have is the precision of the data stored in PI tags is required for calculations but for display purposes it has no added value to show the full precision when using a PI Point DR.  A client is only interested in seeing 3 decimal places for volumes in m3, no decimal places for kg etc

                                • Re: No. of decimal places in returned value
                                  formerpigeek

                                  So what do you think  would be best for you Rhys: associating a specific number of decimal places with UOMs? Or supporting another AF attribute qualifier (number of decimal places). And last thing, should that be used for visualization only or for calculations as well?

                                    • Re: No. of decimal places in returned value
                                      bhiggin2

                                      Laurent Garrigues

                                      So what do you think  would be best for you Rhys: associating a specific number of decimal places with UOMs? Or supporting another AF attribute qualifier (number of decimal places). And last thing, should that be used for visualization only or for calculations as well?
                                      I would prefer the AF attribute qualifier,  the number of decimal places assoicated with a UOM could vary from customer to customer.

                                       

                                      Thanks

                                       

                                      Barry

                                      • Re: No. of decimal places in returned value
                                        Rick Davin

                                        I don't want to see decimal places associated with UOMs.  I'd like it to be an AF attribute qualifier and not just limited to PI Points.  That way I can use it on any attribute especially those with Formula DR or custom DR's even if there is a null UOM.

                                         

                                         

                                        • Re: No. of decimal places in returned value
                                          Rick Davin

                                          Laurent Garrigues

                                          And last thing, should that be used for visualization only or for calculations as well?
                                          That's a tricky call.  I'm inclined to say for visualization.  I could see pros and cons for either, but I am leaning toward visualization. 

                                          • Re: No. of decimal places in returned value

                                            I am with the other comments above, it should be applied as an Attribute qualifier rather than within UOMs and should be purely cosmetic.  Rather than just number of decimal places I would say it should be a format string that can be specified just like you can with a Value symbol in ProcessBook.

                                             

                                            3 AF related projects I have worked on have had formatting of values in the specs, which is fine from a custom application as you can build the logic but 2 projects   wanted to use AFExplorer/PISystemExplorer.  It was maybe coincidence that the reason behind the formatting of values was due to the UOM for the Attributes but I don't think formatting within UOM would satisfy all industry standards..?

                                              • Re: No. of decimal places in returned value
                                                cmanhard

                                                I have some concerns in providing this functionality in AF. I’ll enumerate them below and then perhaps others (OSI and vCampus members), can weigh in on this:

                                                1. Precision is mostly a function of the client.  Our clients already do support formatting (Excel, PB, WebParts) in various ways that extend beyond what is specified in PI.
                                                2. In this vCampus thread, there is a change in the request from what PI provides: “number of decimal places”, to “format string”.  Making this change in AF may will make it more difficult for our clients to make the handling of AFAttributes vs. PIPoints a consistent experience – something that we are still working on.
                                                3. Along these same lines, we could now have a conflict in AFAttributes attached to PIPoints with regard to formatting.  Can AF be different than PI for the same point?
                                                4. Because the number of AFAttribute’s can be quite large, we only very cautiously add new features to it.  Consider that a typical AMI metering application could have billions of attributes defined.  Adding an additional 2, 4, (or more for strings) bytes per attribute has significant ramifications on disk storage, memory use, bandwidth , etc.
                                                5. In PI System Explorer, we have avoided, even for PI Tags, limiting the precision one sees.  This is because PI System Explorer is also used for editing/configuring these values – and when there is a difference between what is displayed and what is edited, it leads to confusion (either the application needs to suddenly adjust to the higher precision at the moment the user starts modifying data – which can have odd effects when initiating via mouse clicks, or, the precision is just lost on edit .)

                                                Thoughts?

                                                  • Re: No. of decimal places in returned value

                                                    Hi Chris...

                                                     

                                                    In response to your points above:

                                                     

                                                    1.  I see PI System Explorer as a client tool so if you are looking for consistent approaches across the OSI tool set, then PI system Explorer should have followed suit - i.e. ProcessBook, WebParts both support formatting of values retrieved from PI.  PI System Explorer does to an extent but only for the Formula DR by way of the Round & RoundFrac functions.

                                                     

                                                    2.  For me, it should be a format string rather than decimal places to accomodate better formatting.  I see this as an attribute change rather than a specific Data Reference change.

                                                     

                                                    3.  ProcessBook can already be different for a point than the PI server.  If you have an operator looking at a PB display with a temperature rounded to 1 d.p. then a field engineer taking the temperature reading & entering via PI-ManualLogger with full precision, then the operator wouldn't see a small change in value.  If the same display had a PI Calc DataSet, then the calc uses the full precision of the value as opposed to the formatted value displayed.

                                                     

                                                    4.  I guess you guys have a much better view on this so I won't comment but I agree with your concerns.

                                                     

                                                    5.  When you edit a value via PI System Explorer, you could switch from "formatted" value to "full precision" value.  If the user cancels or accepts the edited value, it switches back to being formatted - the extended value view would clearly show full precision and formatted values.  Programmatically via AFSDK the default is to always return the full precision value so it is up to the code author to apply the format string (if it is set). e.g. MyAttribute.GetValue(null,null).Value.ToString(MyAttribute.FormatString)

                                                     

                                                    At least this topic is on your radar

                                                     

                                                     

                                                      • Re: No. of decimal places in returned value
                                                        aabrodskiy

                                                        Hi All,

                                                         

                                                        Should we bring this topic back again? I find it of a great value and would like to advocate for the same functionality being included into a future release if possible.

                                                         

                                                        Our use cases and demands are pretty much the same as Rhys and Rick have given in their replies above. We have a lot of custom clients using AF Data via OLEDB/Web services, and the only way to keep consistency in displaying values is to store configuration of the precision of the values in AF Attributes...

                                                          • Re: No. of decimal places in returned value
                                                            MichaelvdV@Atos

                                                            I think this is an interesting (and almost philosofical) discussion. Being able to limit the precision on attribute values seems a good addition, but it also has some impacts, and maybe limits some transparancy.

                                                             

                                                            What's your take on Chris his 'concerns' and Rhys his reply?

                                                              • Re: No. of decimal places in returned value
                                                                Rick Davin

                                                                I'm going to borrow from USA's electioneering stance and say that I firmly straddle the fence on this one.  The performance issues raised by Chris weigh very heavily on me.  Like Rhys, I consider PSE to be a client tool in addition to its developer's configuration tool.

                                                                 

                                                                If we listed out all the things I want added or changed in AF, formatting of the numbers ranks well outside my hot list.  I also already have some control over the formatting by use of the roundfrac function inside formulas, and for many of the data references that I've written I will use .NET's Math.Round(number, digits) method when neccessary.

                                                                 

                                                                 

                                                                  • Re: No. of decimal places in returned value

                                                                    What we lack right now are extended properties of Attributes, and one of the 'built in' extended properties should be display digits.  I am talking about the type of thing we see with Tuning Parameters in the PI Server.  Typically you won't need to set the extended property 'display digits' for an Attribute, it is assumed if the extended property is not set then AF will stick to the default behaviour.  So this would not affect all your Attributes, only those you explicitly set (and I'm certainly not talking about limiting this to the PI Point DR!).

                                                                     

                                                                    Extended Properties of Attributes should be able to be set by Templates and also modified/overridden in derived Element Attributes.  In the same way we should also be able to toggle visibility of derived Attributes of Templates. 

                                                                     

                                                                    I know we (I consider the community as part of OSIsoft's family ) should tread carefully when suggesting such changes that add to data storage of AF Elements/Attributes, but AF needs to scalable and able to handle such changes in the era of "big data".  I am sure billions of Attributes within a single AF Database is not the norm for the current exposure of AF...or is it?

                                                                      • Re: No. of decimal places in returned value
                                                                        pcombellick

                                                                        Certainly "billions of Attributes" is not the norm, but "billions of Attributes" is an important target.  Could display digits be a property on an AttributeTemplate but not necessarily on an Attribute (or Attribute Value)?

                                                                          • Re: No. of decimal places in returned value
                                                                            Rick Davin

                                                                            While I compliment Rhys for a cool concept of extended properties for PSE, I'm not too gung ho about setting this display format property for just PSE.  If I must also independently declare the display format for Excel, ProcessBook, or other client tools, then the efforts put in PSE aren't enough for me.  I'd want it to be the default display format across all client tools, which may be a significant performance drag.

                                                                             

                                                                            My experiences have been that the numbers I really need to focus on formatting are those coming from Formulas or custom data references.  PI Points and static values without a data reference aren't too big of a concern.  With various inputs and possible UOM conversions in formulas or DR's, I sometimes get a long, unsightly string of decimal digits.  Yet, it's within my own control today to use roundfrac in Formulas or Math.Round in my .NET code to adjust those numbers to a desired precision and clean up how they look.  To do that judicously in a formula or DR is a smaller performance hit than doing it for ALL attributes in my database, and the cleaner looking number in PSE also looks cleaner in Excel and ProcessBook.

                                                                             

                                                                             

                                                                              • Re: No. of decimal places in returned value

                                                                                @Rick, glad you like the concept.  A slight clarification:  I am talking about extended properties of an AF Attribute object and not extended properties of PSE.  Much like we currently have extended properties of AF Element objects.  Also, I am suggesting these properties are only interpreted by AF SDK is they are set, if not then there is no overhead (or addition to the SQL database) apart from some additional logic in the AF SDK.  If you apply those extended properties to an AF Attribute Template then of course it will affect a lot more of your AFDatabase.

                                                                                 

                                                                                @Paul: (From my perspective) I would want to set a property on an AF Attribute Template, be able to override that property on an AF Attribute derived from the template, or even remove/disable the property from the derived AF Attribute.  In a similar manner, I would want to be able to set properties on an AF Attribute that is not derived from an AF Template.  You could re-use a lot of configuration logic by using Enumeration Sets for example.  

                                                                                  • Re: No. of decimal places in returned value
                                                                                    Rick Davin

                                                                                    @Rhys … I understood fully what you meant.  What I see is a novel PSE-only solution that has no bearing on any other client tool for visualization of data.  The emphasis of our visuals is still ProcessBook, Excel, and some custom SharePoint; outside of our application designers, few users will ever touch PSE.  So I would reap little benefit from configuring a theoretical DisplayFormat extended property of an attribute in PSE.  By rounding to a significant number of digits where warranted, I can influence the appearance of that number in all my client visuals right here, right now.

                                                                                     

                                                                                     

                                                                                     

                                                                                    Let’s consider a very trivial example.  Consider an attribute of “Raw Temperature” with a UOM of °F and a static value of 32°F.  We all know that 32°F = 0°C on paper, but we know it’s not so clean or trivial due to how floating point numbers are represented in our computers.  In addition to the “Raw Temperature”, I also have a trivial UOM conversion formula for attribute “Without Formatting”.

                                                                                     

                                                                                     

                                                                                     

                                                                                    AFAttribute: Raw Temperature

                                                                                     

                                                                                     

                                                                                     

                                                                                    6406.Raw-Temperature-32F.png

                                                                                     

                                                                                     

                                                                                     

                                                                                    AFAttribute: Without Formatting

                                                                                     

                                                                                     

                                                                                     

                                                                                     

                                                                                     

                                                                                     3021.Without-Formatting.png

                                                                                     

                                                                                    The formula as text is simply: A=Raw Temperature;UOM=°C;;UOM=°C.  Note that instead of a clean and simple 0°C, I get a far more uglier and slightly confusing -7.105427357601E-15 °C thanks to the UOM conversion of floating point numbers.  This produces what I would call an undesirable ‘yuck factor’ no matter what client visual is being used.

                                                                                     

                                                                                     

                                                                                     

                                                                                    AFAttribute: With Formatting

                                                                                     

                                                                                     

                                                                                     

                                                                                     

                                                                                     

                                                                                     2500.With-Formatting.png

                                                                                     

                                                                                    The formula is simply: A=Raw Temperature;UOM=°C;[roundfrac(A,3)];UOM=°C.  Now, I get a much cleaner and nicer 0°C.  Again, I should now get this in any client visual.  The effort I expended in selecting this rounding is now leveraged across any client avoided possible yuck factors.

                                                                                     

                                                                                     

                                                                                     

                                                                                    Now let’s talk performance. 

                                                                                     

                                                                                     

                                                                                     

                                                                                    I changed from a static value for “Raw Temperature”, which was chosen to illustrate a very trivial case, to being a PI Point DR were I have 85,625 events across some time range.  I’m testing on my laptop which is running both PI and AF servers.  I fetched data for each attribute 3 times and produced an average per attribute.  Note that each invocation produced similar results fairly close to each respective average, so I do not believe caching skews my results.  AF Server version 2.3.0.4048.  PI Server version 3.4.385.59.

                                                                                     

                                                                                     

                                                                                     

                                                                                     

                                                                                     

                                                                                    { A 'Mea Culpa' tag is needed.  I discovered I did invoke caching when using a PI Point DR.  My test code would check for PI Point DR and then make a call to the RawPIPoint object before eventually calling GetValues.  This act was not reflected in the timings.  It is corrected below.  I'd treat many of the numbers below as being incomplete.  For a better test, see thread http://vcampus.osisoft.com/discussion_hall/development_with_osisoft_sdks/f/28/t/2383.aspx . }

                                                                                     

                                                                                     

                                                                                     

                                                                                    For the “Raw Temperature” attribute (PI Point DR), the average fetch time was 289 1059 milliseconds.  This is around 296 82 AFValues per millisecond.  This should be considered a base line for subsequent time comparisons below.

                                                                                     

                                                                                     

                                                                                     

                                                                                    The “Without Formatting” attribute (Formula DR), the average fetch time was 1646 ms.  That is around 52 AFValues per ms.  That’s an eye-opening performance drop for something that does a trivial UOM conversion.

                                                                                     

                                                                                     

                                                                                     

                                                                                    The “With Formatting” attribute (Formula DR), the average fetch time was 2084 ms.  That is around 41 AFValues per ms.  That’s for the same trivial UOM conversion that includes a roundfrac call.

                                                                                     

                                                                                     

                                                                                     

                                                                                    One could argue that the numbers suggest I should not use roundfrac.  Keep in mind this is a trivial example and I have far more complicated formulas and DR’s.  I’d rather selectively use the call to roundfrac where warranted, instead of having theoretical DisplayFormat property on each and every attribute in my database.  Even if you have this property set to null or nothing for the majority of attributes, AF would still have to check to see if the property is set.  This check alone will add some time to a data fetch.  In the law of big numbers, the more attributes requiring even a trivial check will eventually add up.  I’m hoping to reduce that number by judiciously using roundfrac on fewer attributes.

                                                                                     

                                                                                     

                                                                                     

                                                                                    I fully admit my viewpoint is skewed given few of my users will actually touch PSE.  But I can better control those select instances today (and I have), without OSI adding  developer time for a feature that has the potential to degrade performance.

                                                                                      • Re: No. of decimal places in returned value
                                                                                        aabrodskiy

                                                                                        Rick,

                                                                                         

                                                                                        I understand your points, but even though it might impact performance, the extended properties of attributes are still required. We still need to store somewhere the formatting of each attribute, and other additional properties. As you know, some attributes, e.g. Density, always require 3 decimal places, other may need 3 places, etc.. Store this information somwhere else would be just inconsistent approach, so having them as properties of the AF Attributes would be neat. But we cannot allow a luxury of having AF Formulas with roundfrac for each and every attribute we need to display. Also performance degradation, as you already pointed, and other funny things when working with AF Formulas.

                                                                                         

                                                                                        I concur with Rhys that we would need extended properties on the Attribute level, that would come from AttributeTemplate, but can be overriden in elements. One of the pre-defined properties could be DisplayFormat or DisplayDigits, etc... Other should be just free-form configurable, e.g. I need to specify for some attributes: Thresholds, DataSource, different flags, etc.. And this should be accessible in the clients/AFSDK/OLEDB Enterprise...

                                                                                          • Re: No. of decimal places in returned value
                                                                                            Rick Davin

                                                                                            @Alex, I don't agree that the extended properties of attributes are still required universally.  Maybe for you and your customers, but not for me or mine.  If OSI were to ever add such a feature, I would hope they would leave the GetValue and GetValues methods alone, and instead add something like a GetFormattedValue and GetFormattedValues methods.  I'd prefer to see a zero-impact to existing methods for any proposed feature that has the potential to degrade performance.

                                                                                             

                                                                                             

                                                                                        • Re: No. of decimal places in returned value
                                                                                          pcombellick

                                                                                          @Rhys,

                                                                                           

                                                                                          I can envision adding a property for "displayformat" to an AFAttributeTemplate, but adding support to override in the AFAttribute creates performance issues for extremely large systems.  I have been performance tuning a system with over 250 million attribute values.  Adding a 10 char UNICODE format string to each attribute, adds 5GB to the database.  Though disk space is essentially free, disk IO(except with SSD), network bandwidth, and client RAM are not.  Of course, no one would override 100% of the displayformats, but if it is supported, somebody will do it.

                                                                                           

                                                                                          Paul

                                                                  • Re: No. of decimal places in returned value
                                                                    buse

                                                                    I know this is an old post, but what about formatting the AF Values inside an AF Notification message delivered through the Email delivery channel?

                                                                    3-13-2015 11-17-45 AM.png

                                                                    This is a little bit too accurate

                                                                     

                                                                    Thanks,

                                                                    Ionut

                                                                      • Re: No. of decimal places in returned value
                                                                        dng

                                                                        Hi Ionut,

                                                                         

                                                                        There are indeed existing enhancement requests for configuring the display format for AF values:

                                                                        • 5947 - Request for "DisplayDigits" type property on an AFAttribute, or at least a way to pass through PI's (general)
                                                                        • 44053 - Let user format the way formatted values are displayed (specific to notification messages)

                                                                         

                                                                        As workarounds, you can either change the “value type” of the attribute to single (instead of double). This will decrease the number of decimal places (but also the precision). Or create a separate attribute, and use roundfrac() to round to the number of fractional digits specified.

                                                                          • Re: No. of decimal places in returned value
                                                                            Paul_Booth

                                                                            This is still an issue of course, despite what developers in a non industry environment think. It is VERY useful to have a centralised (i.e. PI) guide for a value precision. Of course, it can be overriden by any client, but it should still be available. If I use a PI point directly in ProcessBook, I can configure the precision via the display digits and it will dynamically alter the value as required (not everything is fully defined at devolpment time!). If I construct the same thing but via AF, I can fix the precision only at build time. This is an incompatibility that shouldn't have been put in: probably it was a bit difficult to do, and then easier to make up a rationale as to why it was missed out. From the dates on the thread it looks like OSI isn't going to fix this problem, and, yes, it is a problem. Not every system is fully defined and cast in stone when displays are created.

                                                                          • Re: No. of decimal places in returned value
                                                                            Mike Zboray

                                                                            There is one work around, if you create a global parameter called DoubleFormat that will be used to format doubles in all notification messages.

                                                                             

                                                                            See David Moler's answer for complete details.

                                                                    • Re: No. of decimal places in returned value
                                                                      bhiggin2

                                                                      Thanks Steve and Rhys, I'll sort this out when I'm back in the office in the morning. I am getting to grips with AF and I will be back with more questions .