4 Replies Latest reply on Mar 1, 2018 4:54 PM by tramachandran

    AFDatapipe memory concern

    natdavidson

      I have some code that is subscribing some AF attributes that span many different elements. I started with about 8k attributes, and the code performed how i expected, but the memory was quite high, >1Gig of memory. I modified the code to include some deeper elements, and that increased the number to 13k attributes, and now i'm really running into some memory issues. When I debug my code, I see the memory is being consumed to subscribe the attributes to the AFDatapipe object, with 13k attributes, its going above 2gigs of memory committed. Is there a better way to go about this, and is this normal behavior? The attributes are results from analytics in AF.

        • Re: AFDatapipe memory concern
          tramachandran

          What is the version of AF SDK library & PI AF/Notification and would it be possible for you to share your code?

          The numbers you have for the AF Datapipe seem to be "large" but this qualification is subjective depending on various other factors.

           

          You can refer to PI AF SDK Best Practices - Summary of Existing Resources which may help you improve upon your code.

          https://techsupport.osisoft.com/Troubleshooting/KB/KB01216/

          1 of 1 people found this helpful
          • Re: AFDatapipe memory concern
            Rick Davin

            Hi Nate,

             

            As Thyag mentioned, it would be great to know your version numbers.  Also, is your app 32 or 64 bit?  I've been annoyed with Visual Studio since 2015 when they seem to favor 32 bit builds by default.

             

            If you could share the relevant code, it would be great. By all means, obfuscate it where you need to.

             

            General stuff ...

             

            Since you are dealing with attributes in a data pipe, this most likely means a full load on the elements was done.  A full load will load info about any reference type properties such as Elements, Attributes, and Analyses.  Are all the attributes in the pipe PI points?  A formula or table lookup attribute may have a dependency on one or more other attributes, so those would be loaded too.  And if an attribute has a dependency on another attribute on another element, then that other element is fully loaded too.

            1 of 1 people found this helpful
            • Re: AFDatapipe memory concern
              natdavidson

              AFSDK Version: 2.8.0.7444

              PI AF Server Version: 2.8.5.7759

               

              Not sure how much of my code you would want to see, a lot if it is just establishing the pi specific stuff, the template setup, then doing stuff with the results of anything in the Datapipe, but this is the part where i see the memory go crazy and it also takes the longest.

              AFNamedCollection<AFAttribute> colAttributes;
              AFDataPipe myAFDP = new AFDataPipe();
              
              
              colAttributes = AFAttribute.FindElementAttributes(
                                  myDB,
                                  null,
                                  null,
                                  null,
                                  ElementTemplate,
                                  AFElementType.Any,
                                  "Element Type",
                                  null,
                                  TypeCode.Empty,
                                  true,
                                  AFSortField.Name,
                                  AFSortOrder.Ascending,
                                  20000);
              myAFDP.AddSignups(colAttributes);
              

               

               

              PISquareHelpDatapipes.png

               

              The above screen shot it is to give you an idea on what kind of attribute this is, it's an analytic, with 4 sub attributes that are also analytics, which also have other sub attributes. So the image on the left shows a basic composition of the attribute i'm putting in the data pipe, and the right shows how complex it can get. The resulting attribute that is at the top, gets either a a 0 or a 1, and this is what we are using to perform some other actions on, but the top attribute is what i'm searching for in my code, and then adding all of these to a data pipe.

                • Re: AFDatapipe memory concern
                  tramachandran

                  As Rick Davin mentioned we would like to know the application bitness.  I would prefer a little more detailed snippet of your code (it can be redacted and you can message that to us through PI Square if you don't want to publicly share it). What I would like to see is how the client handles the events. Normally event retrieval would be called from a background thread to handle the processing of the changes. This also allows the client to control how often to check for events.

                  Are you implementing an IObserver in your client side code?

                  From the documentation:

                  There are two ways to get the data change events: a) direct GetUpdateEvents method to get events; b) through IObserver of AFDataPipeEvent. The IObserver pattern provides significant performance improvement over direct GetUpdateEvents method for high throughput scenario. Application will have to implement the IObserver and register the IObserver with the data pipe via the Subscribe(IObserver<AFDataPipeEvent> ) method.

                   

                  An AF SDK trace would also be helpful (filter by process ID)