16 Replies Latest reply on Nov 28, 2018 7:08 PM by David Hearn

    AF database has been disconnected

    Robatjazi

      Hey guys,

       

      I have an issue with AF database. This is the scenario:

      - There is one AF Server with multiple database.

      - There is a WEB API which connect to AF database to read  and write to it.

      - There is 5 services, 3 of them using the API and 2 of them connect to AF database

      - We have 3 different users here, one is used for Web API, one is used for 3 service (which use API), the last user is used for 2 service that directly connect to AF database.

       

      In one of the services we get following error time to time:
      System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Database 'XXX' has been disconnected. This is caused by using an object from a database after calling PISystem.Disconnect.

       

      The thing is that it does not happen all the time. this service is running every 1 minutes and every hour or so we get this error.

      Based on stack trace, it happens when service trying to read attributes from AFElement.

       

      Anyone has any suggestion?

       

      Thank you!

        • Re: AF database has been disconnected
          gregor

          Hello Majid,

           

          I assume the 2 services connecting against AF Database (not using PI Web API) are using AF SDK.

           

          Please see AF AFSDK Connect problem with multithreading, Database 'somedatabase' has been disconnected or Adding AF Elements using AF SDK

          Because you mention to use certain credentials, I assume you are seeing the issue link # 2 and 3 are about.

          • Re: AF database has been disconnected
            VasanthakumarR

            Hi Majid,

             

            This seems to be a security issue. Can you please let us know what service account is configured iin the service which is getting the error. check whether that account has enough permissions to access the AF database and attributes. Please refer the following link for more details regarding AF security,KB00841 - An overview of PI AF security

             

            Regards,

            Vasanth.

            • Re: AF database has been disconnected
              Lal_Babu_Shaik

              Hi Majid

               

              Could you please let us know AF client version you are using? We had similar disconnection issues and upgraded to AF client 2018 SP1 on server running the services which fixed retry problem when connection was lost. In our case backup agent terminating connections and  AF client fails to connect again but after upgrade it fixed.

               

              Thanks,

              Lal

                • Re: AF database has been disconnected
                  Robatjazi

                  Hi Lal,

                   

                  We use 2017 R2 (2.9.5) as PI AF Client

                  Also we use the same AF SDK version.

                   

                  Should I upgrade both or just AF Client?

                   

                  Regards,

                  Majid

                  • Re: AF database has been disconnected
                    Robatjazi

                    Hi Lal,

                     

                    I upgraded AF Client to 2018 (2.10) and it did not help. Still getting disconnect error every one or two hours.

                     

                    Regards,

                    Majid

                      • Re: AF database has been disconnected
                        gregor

                        Hi Majid,

                         

                        How many occurences of PISystem.Disconnect can you find in the source code?

                        Can you share the relevant methods and also let us know if they are used single- or multi-threaded?

                          • Re: AF database has been disconnected
                            Robatjazi

                            Hi Gregor,

                             

                            None. as it is recommended. I do not disconnect at all. I just dispose the PISystem object whenever I am done using the AF objects.

                            Which method you are asking for?

                              • Re: AF database has been disconnected
                                gregor

                                Majid Robatjazi wrote:

                                 

                                I just dispose the PISystem object whenever I am done using the AF objects.

                                 

                                Well and this will pretty likely be the reason for your problem. The PISystem object becomes instantiated only once per application and user account. Unless you create the object for multiple concurrent users, there will be a single object. If you dispose it, it will eventually become garbage collected. When this happens and your application tries to use the object, it runs into the problem corresponding the error you are getting.

                                 

                                If you plan using the PISystem object, why do you want to dispose and re-instantiate it? No, please don't do that. By the way, the same applies for the PIServer object.

                                  • Re: AF database has been disconnected
                                    Robatjazi

                                    Well I dispose my PISystem object in our pool and not anything in AF Server/Client. I do not want to have a persistent open connection object to PI. Whenever I need to communicate with AF I just create an instance of my AF Repository  which stablish a connection to AF (Iassume because I do not disconnect the created connection before, it is cached in AF side and is will be used). Like as following:

                                             

                                    using (IAFRepository repo = AFRepoFactory.CreateRepository())
                                    {
                                             //Doing some Af related stuff ...
                                    }
                                    

                                     

                                    class AFRepositoryFactory
                                    {
                                               public IAFRepository CreateRepository()
                                               {
                                                    PISystem pisystem = new PISystems()[ConfigManager.GetConnectionStringObject("af-connection").DataSource];
                                                    pisystem.Connect();
                                    
                                                    return new AFRepository(pisystem);
                                               }
                                    }
                                    

                                     

                                    class AFRepository
                                    {
                                                protected virtual void Dispose(bool disposing)
                                                 {
                                                   if(IsDisposed)
                                                   {
                                                      return;
                                                   }
                                    
                                                   if(disposing)
                                                   {         
                                                      if(_database.IsDirty)
                                                      {
                                                         _logger.Warn("Rolling back unexplicitly terminated PIAF transaction");
                                                         _database.UndoCheckOut(true);
                                                      }
                                                      _pisystem.Dispose();
                                                      IsDisposed = true;
                                                   }
                                                }
                                    }
                                    

                                     

                                    This is simplified version of what we have thou.

                                      • Re: AF database has been disconnected
                                        gregor

                                        Hello Majid,

                                         

                                        Not sure why you are sealing PISystem in your custom class AFRepositoryFactory. 2nd snippet, line 05 is your way to chose the PISystem to use and I assume this also works outside the factory. The new statement is for the PISystems object which is the Known Servers Table.

                                         

                                        Line 06 is not necessary at all. The PISystem object is capable of handling the connection by itself including reconnections due to network issues. It also prevents against creating multiple instances for the same destination (AF Server) within a single application and the same user context. Assume your application would create PISystem objects pointing to the same AF Server but using 4 different credentials, it indeed would create 4 independent PISystem objects, all with its own client side cache and by considering account specific permissions. It will not create more than those 4 instances. With a single account, the same PISystem instance will be used as long as it is found in memory.

                                         

                                        Your problem is not due to your custom class but how you use it which you show with the 1st snippet. The IAFRepository object is only valid inside the using clause. As soon as your code exits the using, the object is potentially garbage collected. Now assume that AF SDK decided to not create a new PISystem object but to reuse the one it found in memory and at the same time the garbage collector decided to purge this same PISystem object. This is where you run into the invalid operation exception. You avoid this situation by creating a global static PISystem object and by keeping it alive as long as your service needs it. So if you like to be nice and cleanup, one good reason would be when the service becomes stopped.

                                         

                                        Majid Robatjazi wrote:

                                         

                                        I do not want to have a persistent open connection object to PI.

                                         

                                        Why not? Can you please explain why this is a concern?

                                         

                                        Edit: By the way, I've applied Syntax highlighting to your code snippets. Syntax highlighting is available in the Advanced Editor through the ">>" icon. 

                                        2 of 2 people found this helpful
                                          • Re: AF database has been disconnected
                                            Robatjazi

                                            Hi Gregor,

                                             

                                            There are more into those classes and methods, I just put those line which are related to discussion. AFRepositoryFactory is part of "repository factory pattern".

                                            Basically I just create a repository whenever I need it and close is, which dispose the repository object and PISystem as well. I think what you are saying make sense regarding the PISystem dispose. I removed the PISystem dispose from AFRepository dispose and it seems that the issue is gone.

                                             

                                             

                                            I did not know that pisystem.Connect() is not necessary! So why this method exist as a public method then!