9 Replies Latest reply on Mar 16, 2010 10:53 PM by ken

    Getting a tag list from a PI server takes quite long time.

    formerpigeek

      We have a program that has trend displays to show the data from a PI server (similar to the Tend Display in PI ProcessBook. The problem we had is that the trend configuration form in our program takes about 60 - 75 seconds to load the tag list. And after selecting a tag, it takes about 30 seconds to show the data in the trend. The time of retrieving the tag list (total 40,000 tags in the server) is quite long comparing to ProcessBook (if I create a trend display using the same PI tag in PI ProcessBook). The program uses PI-SDK to open a connection to PI server and retrieves the tag list and point data. The following is the code in the program:

      PISDK.PISDKClass piSDK;
      PISDK.Server piServer;

      piSDK = new PISDK.PISDKClass();
      piServer = piSDK.Servers[“servername”];

      //Connect using a trusted connection
      piServer.Open(String.Empty);

      Regex regExp = newRegex(“tag filter string from the configure form”);

      PISDK.PointList piPoints;
      string filter = "pointsource = 'O'";

      piPoints = piServer.GetPoints(filter, null);

      DataSet dataDS = newDataSet("BrowseTag");
      DataTable dataDT = newDataTable("BrowseTag");
      dataDT.Columns.Add("tagName");

      // Build the return dataset
      foreach (PISDK.PIPoint piPoint in piPoints)
      {
          // Only return points match the regexp filter
          if (regExp.IsMatch(piPoint.Name) == true)
          {
              DataRow dataDR = dataDT.NewRow();

              dataDR["tagName"] = piPoint.Name;
              dataDT.Rows.Add(dataDR);
          
      }
      dataDS.Tables.Add(dataDT);

      Then just return the dataDS to the function caller. The function caller will display the tag list in the trend config form.

       

      I like to know what caused the delay of returning the tag list. What is the reasonable time to filter a group of tags from a PI server (we have about 40,000 tags with pointsource='O'). What can we do to improve the performance? This 60 seconds delay in returning a tag list is too long. 

       

      Your help will be very much appreciated.

       

      Tina

        • Re: Getting a tag list from a PI server takes quite long time.
          mheere

          Hi Tina,

           

          Are you sure that it's the call to get the point from the PI server that's taking all the time?  It looks to me like you are creating a PIPoint object with every itteration through the returned Pointlist collection.  This could actually take quite some time to do, and you'd see the processor and memory usage of your program spike as well.

           

          If it turns out that it's the FOR..EACH loop that's eating all the time, try decalring the PIPoint object before the loop definition, so that it reuses the object definition rather than making a new one each time.

           

          I'd also be curious about the regular expression match.  Can you not also pass the tagname criteria to the GetPoints call, or is the filter more complicated than simple wildcard matching?

           

          If it really is the call to PI, then hopefully someone else will have a good answer.  Bunch of smart folks here on vCampus

           

          Matt

            • Re: Getting a tag list from a PI server takes quite long time.
              formerpigeek

              Hi Matt,

               

              Thanks a lot for your respond. You are right, both getting the tags from the PI server and building the dataset for the tag names in the foreach loop take time. I do see the processor and memory usage spike. But I could not declare the PIPoint object before the loop definition, the type and identifier are both required in a foreach statement. Yes, the tag name filter in our trend configuration form is a simple wildcard text matching. What I did is pass the filter string to the GetPoint call to let the PI server handle the search. That reduces the process time a lot.

               

              Thank& you!

               

              Tina

            • Re: Getting a tag list from a PI server takes quite long time.
              andreas

              Hi Tina,

               

              getting the 40k tags will take some time. According to my tests here (a 130k PI Server, retrieving 130k tags and using a regular expression reducing that to ~50k) most of the time is spent in GetPoints (~25s).

               

              I can not see anything that you can improve beside reducing the amounts of tag returned from the server by adding more restrictions to the filter (that are now in the regular expression).

               

              What bothers me more here is your statement that it requires 30s to retrieve the data? Can you show the code for this as well?

               

              Regards,

                • Re: Getting a tag list from a PI server takes quite long time.

                  One thing we don't see in your code is whether you are making use of attributes other than .Name; for example, are you making checks on other attributes that you omitted in this post for the sake of consiseness? When calling GetPoints, the PIPoint objects are created for the PI SDK object cache and a only a default set of PIPoint attributes are returned by the original call. Any subsequent use of additional attributes will return to the PI Server for more PointAttributes. Can you confirm this? A better call would be IGetPoints2.GetPoints2, to which you can specify specific attributes to be returned by the original call - thus reducing the number of roundtrips with the PI Server.

                   

                  Something else you might want to change is the fact that the call is made synchronously. Maximum throughput could be obtained by using the asynchronous mechanism and beginning to use the objects as they are made available. If asynchronous data calls were made, then the concurrency of the PI SDK and PI Server could be leveraged to get to the final result more quickly.

                   

                  Those are the likely differences with PI ProcessBook...

                  • Re: Getting a tag list from a PI server takes quite long time.
                    formerpigeek

                    Hi Andreas,

                     

                    I did what you suggested – add more restrictions to the filter to reduce the amounts of tag returned from the server. The results are very good. I created 25K PI tags in our testing system. With the tag name filter string passed to GetPoint call, I got the matching tags (about 1k) retuned in 2-3 seconds. But if asking PI server to return all 25K tags (if the user enter ‘*’ in the filter text box), it will take about one minute.

                     

                    When the user selects a tag name from the tag list, our trending usually shows the trend data very quickly. The 30 seconds delay for retrieving data from a PI server was caused by a slow network connection. One of our customers has a slow network link to the remote Primary PI server in the PI Collective (they have four PI cluster servers in the Collective). They saw the delay in the trending. I modified our code so that they can connect a local PI server (with fast network connection) to get the trending data. The data delay problem was resolved. But that brings another issue. Because I use the MemberOpen() method in IPICollective interface  to open a connection to a collective member (not the Primary server), if this PI server goes down, will the PI Collective manager handle the failover to another member in the Collective? Which members will it failover to (according to the member’s priority?)? The following is our code for the connection:

                     

                    object
                    _ServerName = requestHostName.ToString();
                    piServer = piSDK.Servers[(String)_ServerName.ToString()];
                    PISDK.IPICollective Col = (PISDK.IPICollective)piServer;
                    if (Col.IsCollectiveMember)
                    {
                        PISDK.CollectiveList ColList = Col.ListMembers();
                        PISDK.CollectiveMember ColMember = ColList.get_Item(ref _ServerName);
                        piServer = Col.MemberOpen(ColMember, "ServerRole=Any");
                    }
                    else
                    {
                        piServer.Open("");
                    }

                     

                    The program gets the PI server name from a configure file. I like to know if this is correct way to connect to a collective member. Will it cause any problem?

                     

                    Thank you very much for your helps!

                     

                    Regards,

                     

                    Tina

                      • Re: Getting a tag list from a PI server takes quite long time.

                        Tina Wang

                        I use the MemberOpen() method in IPICollective interface  to open a connection to a collective member (not the Primary server), if this PI server goes down, will the PI Collective manager handle the failover to another member in the Collective? Which members will it failover to (according to the member’s priority?)?
                        No, this is not the right way to connect to a Collective; you should simply connect using Server.Open("ServerRole=Any") and let the PI SDK handle which member it connects to and handle failover. One of the concepts of PI Server Collectives is that they should be "seamless" to client applications (i.e. look like a regular PI Server). You can control what the "preferred server" (i.e. the local one, not the Primary) is via the Priority setting of each member, in the PI SDK Connection Manager.

                         

                        You can take a look at the "High Availability" and the "Open Method (Server object)" articles in the PI SDK programming reference.

                         

                        If I may kindly suggest, I think it would be better to initiate new discussion threads from "derived" questions like this (i.e. not the original question)... with the appropriate subject and "tags", it makes it easier for other people in the community to find the answers they are looking for

                         

                        Tina Wang

                        I tried your suggestion to make the GetPoints call as an asynch call but not successful. I will try again.
                        Alrighty - don't hesitate to create a new discussion thread if you need help on this particular topic

                    • Re: Getting a tag list from a PI server takes quite long time.
                      ken

                      You should use the asynchronous version of the GetPoints call.

                       

                      That will return to your program after retrieving partial results. You can monitor how the call is progressing by looking at the PIAsynchStatus object's properties.

                       

                      Your program can also sign up for events from the PIAsynchStatus object. These events will tell your program when more results have been returned and when the call is complete.

                       

                      If you want to cancel the call, you can call the Cancel method of the PIAsynchStatus object.

                       

                      If there are specific attributes you want to display along with the tag name, you can use the IPIGetPoints2.GetPoints2 call. That allows you to specify which point attributes you want retrieved with each tag. This saves another round trip call to the server to retrieve additional point attributes you need.

                       

                      For more information about these objects and methods, please check the PI-SDK help file.

                       

                      Ken Coates, PISDK team