13 Replies Latest reply on Dec 19, 2012 7:55 PM by jamliu

    AFSDK Connection Pooling

    aldorexbraam

      Hi I was wondering what the options are to use connection pooling on AFSDK connections. 

       

      It looks as if that common feature, in almost all oledb providers, is not available. 

       

      Currently we have implemented our own 'pool manager' (based in the concurrent number of threads, our webserver is configured to handle), to bypass this ....

       

      but it does not feel right. 

       

      Perhaps we overlooked something....

       

       

        • Re: AFSDK Connection Pooling
          MvanderVeeken

          Good question.

           

          The communications to the AF Server are done via WCF (web services), and I believe net.tcp bindings are used.

           

          OLEDB is a very different protocol. Usually the basic .NET framework WCF client proxies make use of connection pooling. I am not at all sure how the implementation from an AFSDK standpoint is. 

           

          Maybe one of the OSIsoft AF Developers that are active here (Chris Manhard for instance) could comment on this.

            • Re: AFSDK Connection Pooling
              David Hearn

              The AF Connection model is slightly different than OLEDB.   Under the hood, AF does maintain a connection pool for communicating to the AF Server, and will create and use multiple connections so that multiple threads can concurrently access the server (beginning in AF 2.3 Client).  However, unlike OLEDB, there is a substantial cost in establishing an initial connection for each AF SDK instance, as there is significant client side caching of objects that must occur.  Because of this, we ask that you do not connect and disconnect (Open/Close) with every web-service call as would be done in an OLEDB connection as disconnecting will destroy the AF SDK cache.  Under the hood, AF SDK will allow connections to the AF Server drop after periods of inactivity, and will automatically re-establish them when activity is resumed.  

               

              AF maintains separate caches of objects based on user identity, and additionally based on a flag passed to the new PISystems(bool forceNewInstance) method.  Individual users and AFSDK instances within the same application will have their own AF SDK cache.

                • Re: AFSDK Connection Pooling
                  aldorexbraam

                  Ok great....thanks for the answer.....that is clear....so what would be your advised approach if we wanted to make a highly scaleable webservice that potentially serves a high number of users....it seems that our current approach of managing our own 'pool' is a valid one after all.

                    • Re: AFSDK Connection Pooling
                      MvanderVeeken

                      Aldo, could you elaborate on why you need more performance? Do you have run into scalabillity issues? Any numbers you can share with us?

                        • Re: AFSDK Connection Pooling
                          David Hearn

                          So I do not think it makes sense to pool AF connections to the AFServer. But it may make sense to use a single account to read the objects from the AFServer so that AFSDK only has a single user cache if you are concerned about memory usage. The issue here is you will need to use an account that has read permission on all objects and then you must perform the security check against the current user on each object before returning.

                            • Re: AFSDK Connection Pooling
                              aldorexbraam

                              @michael: We are doing load testing on our customers production monitoring application. initial user load is pretty low (10 users) and poses no issue but we want to 'design for performance' to enable scaling up in users.

                               

                              @David : thanks, the issue you point out is exactly the issue that is at hand....if you want to use AF's security mechanism in a proper/usefull way you need to connect with the clients identity. For simple cases you can get away with using for example a least privileged service account, but this basically kills the usage of AF security.  

                               

                              But if you want to use AF how it is meant to be (a federated information model ) IMO you still need a pool per user identity.

                               

                              I am now working with a customer that wants to use AF as a integration platform for all kinds of monitoring apps with potentially hundreds of concurrent users and enterprise scale security controls so hence I am sort of trying to find the system boundaries.

                                • Re: AFSDK Connection Pooling
                                  MvanderVeeken

                                  Aldo Braam

                                  @michael: We are doing load testing on our customers production monitoring application. initial user load is pretty low (10 users) and poses no issue but we want to 'design for performance' to enable scaling up in users.

                                   

                                  PI AF is designed for pretty heavy use cases, and performance is increasing with each release. In danger of stating the obvious -- there is a very good chance that performance will meet your specifications for the heavy use cases. Optimization could bring added complexity and risks. "Premature optimization is the root of all evil'. (Love when I get the chance to use that quote, it's awesome )

                                   

                                  Aldo Braam

                                  I am now working with a customer that wants to use AF as a integration platform for all kinds of monitoring apps with potentially hundreds of concurrent users and enterprise scale security controls so hence I am sort of trying to find the system boundaries.

                                   

                                  That's where PI AF will be very suitable. If you are at liberty to share more about your quest in search of the system boundaries, I think a lot of people here would be interested!

                          • Re: AFSDK Connection Pooling
                            jamliu

                            Hi David,

                             

                              I am in the case that needs to use the ForceNewInstance option in my SharePoint webpart to access the AF objects to make sure the each user will connect to AF with separated cache for security management concerns in web multi-threading environment. However, I am getting the "Cannot connect to server 'AFServerName'" error when external user 1st time opens the page in the browser right after the code deployment. And then the AFObject.FindObject("elementPath") will only return "null" later when different element paths were passed in postback. But internal user doesn't have this issue. (External users access via ISA2006.)

                             

                            Any thoughts? Thank you.

                              • Re: AFSDK Connection Pooling
                                jamliu

                                To follow up my post above, I am not sure how, but this "FindObject() returned Null"  is gone after a code redeployment, and I am facing the original issue I was trying to resolve now. We intermittently received "Cannot connect to server AFServerName" error with 10054 socket error code when the same account is logged in to the page from multiple machines.

                                 

                                Does anyone have any idea? Thank you.

                                 

                                Click here for my original post.

                                  • Re: AFSDK Connection Pooling
                                    David Hearn

                                    You really do not want to use the 'ForceNewInstance' from a webpart. This will consume a large amount of memory and multiple connections in the client for each call. You should either impersonate the user and get a new PISystems object in each call which will re-use that user's cache and connections or use a single user account to connect to AF and then do your own security checks using the AFSecurity.CheckSecurity method.

                                      • Re: AFSDK Connection Pooling
                                        jamliu

                                        Hi David,

                                         

                                          Thank you for the feedback. I tried both with/without the "ForceNewInstance" option, but couldn't get rid of my original problem -- received intermittent "Cannot connect to server 'AFServerName'" when the same account was logged in from multiple machines. (Please see my other post here.)

                                         

                                          I thought the "ForceNewInstance" is the fix for my original problem, but it turned out not, and created an additional problem mentioned above.

                                         

                                          Please let me know if you have any suggestions. Thank you.

                                         

                                          

                                         

                                         

                                          • Re: AFSDK Connection Pooling
                                            David Hearn

                                            The issue you are seeing with connections to the AFServer is being looked into by TechSupport.

                                              • Re: AFSDK Connection Pooling
                                                jamliu

                                                Update:

                                                 

                                                  By following the suggestion from David and OSISoft Tech support, I got the intermittent "cannot connect to server" issue resolved and observed better webpart performance. The key is to only new PISystems object once when the page is first time loaded without using "ForceNewInstance" option, but not in the following postbacks. And the suggested SecurityCheck() works like charm! Thanks David.

                                                 

                                                  If anyone is interested in details. please see post here.