4 Replies Latest reply on May 14, 2015 8:27 AM by AlistairFrith

    Best way to summarise 70,000 tags?


      We have a need to perform a simple calculation summarising a value over around 70k digital tags. Essentially, we need to keep track the total of a number associated with each tag that is currently Offline. We are not allowed to use AF or ACE. We are thinking of storing the number in UserInt1


      What is the most efficient way...


      A windows service, waking up every few minutes and finding all the Offline tags and adding up the associated values?


      A windows service registering event pipes that adjust the total every time a tag changes to/from Offline?


      something else?


      Thanks for any suggestions


      --- Alistair.

        • Re: Best way to summarise 70,000 tags?

          Hi Alistair,


          The best strategy might depend on several factors:

          • How frequent would you like the total value to be updated (i.e. can you tolerate a few minutes delay? or do you want to most updated value at all times?)
          • How frequent do these 70k tags update or change to/from offline?
          • Do you need a history of these data?


          Please help me understand these tags a little more, are they numerical tag, or digital (e.g. on/off) tag? You mentioned digital tag in your post but then mentioned that you are calculating the total value for these tags, which makes me think they are numerical tag. How do you determine if each tag is online/offline?


          If the end user/application cannot tolerate any outdated values, the best way might be monitor these tags with event pipes so you are not missing any events. If the tags do not update often and you are ok with stale values for a short period of time, perhaps doing a periodic calculation will suffice. One suggestion I would make is to perform the calculation on the server-side if possible to avoid transferring all the raw values over the network.

            • Re: Best way to summarise 70,000 tags?

              The tags are digital tags giving the state of a component (Live/Dead/Isolated/etc.). Each component has a number of dependent customers (which we will be putting in the UserInt1 attribute) and we need to keep track of how many customers in total are 'off-service' at any time.


              The number of dependent customers associated with a component changes rarely and so is held in the UserInt rather than in a separate tag. The tag's value also changes quite infrequently. The total number of users 'off-service' needs to be calculated at least every 5 minutes, so it doesn't have to be instantaneous, but we do need to keep a history of it. It sounds like a Windows Service, waking up every couple of minutes to perform the calculation using current snapshots might be acceptable. I will have to ask whether shorter periods of 'off service' can be ignored.

                • Re: Best way to summarise 70,000 tags?

                  Interesting benchmark for your PI Server


                  Assume 70 seconds full polling time, it results in average 1k/second snapshot reads, in addition to your normal server load. This should be serviced by PI Server without problems.

                  1. Prepare a windows application or service running on PI Server to minimize network overhead.

                  2. Read your tag list (ot PointId list) and userints1 into array in memory.

                  3. Start polling your tags with, say, 14 * 5 seconds intervals, 5k tags each, to avoid peak loads.

                  Use AFSDK, PI-SDK or even PI-API to read snapshots. My experience is that AFSDK may be slowest, PI-API fastest. All deliver snapshot values quick - they are read from memory. Don't convert them to strings, use numeric values (int16) read as digital state numbers. Store them in memory.

                  4. Perform your calculations after 70s cycle and write result (sum of active userints1) in one PI Tag, using anything you want. It is not critical.


                  Install some perfmon tags to check the load of PI Server (total CPU and pisnapss)

                  3 of 3 people found this helpful