6 Replies Latest reply on Mar 7, 2014 5:15 PM by Gregor

    Archive Activity Grid - Performance?

    flost

      Hi all,

       

       

       

      I´v read the information aboput the AAG. It sounds good and I have several Ideas to use the data. But I´m wondering about the warning about heavy resource consumption. I´m not shure if this statement is there since servers were much less powerfull then nowadays or if it is still a problem.

       

      My current productive PI Server has 40 CPU´s (well, 20 cores plus Hyperthreading) and 32 GB of RAM all on bare metal, no VM. I just let the AAG run just to check if there is an rais in ressource consumption but I can´t see anything. Ok, I started with a few minutes but now I have it run for about two hours and I can´t see anything (still 1-5% CPU Usage and 7.10 GB RAM used).

       

      Has enyone of you experience with AAG if it runs for an longer timerange?

       

      If there are no significant ressource needs it would be great to have extensive statistical data of the usage of my PI Points. I´m thinking about award an Tag of the month an Application of the month span="" id="mce_marker" data-mce-type="bookmark">

       

      Regardas

       

      Florian

        • Re: Archive Activity Grid - Performance?
          Rhys Kirk

          This sounds like an obvious statement, but it depends on how much activity is going on with your PI Server. For example, running it on a secondary with fewer users is not the same as the primary where typically majority of user's connections default to. Do you have a large user base or connected applications?

           

          I last used AAG a couple of years back on a heavily used PI Server and couldn't run it for long, i.e. could not run it for the purpose you are referring to.

          • Re: Archive Activity Grid - Performance?

            Hello Florian,

             

            The data collected by Archive Activity Grid is held in memory. Even your PI Data Archive (PI Server) hardware is strong and current usage is at about 20%, using the Archive Activity Grid only temporarily is still recommended. Please allow me referring to Technical Support KB00335.

             

            Rather than looking at the Archive Activity Grid, I suggest collecting some performance counters i.e. those exposed by PI Snapshot and PI Archive subsystem. You don't get the granularity enabling you to select a "tag of the month" but when recording the Performance Counters, you get a good feeling about how the load changes over time. I also suggest recording process counter information like CPU usage, Elapsed Time and Private bytes of piarchss and pisnapss and possibly other PI Subsystems. The Elapsed Time counter of a process is like it's heartbeat. As soon as it becomes stale, it indicates the process is experiencing serious trouble.

             

            For the "tag of the month" competition I would suggest using the offline Archive Check i.e. against backed up archive files. Please see e.g. Technical Support KB00923. The article also suggest using a PI OLEDB (classic) provider query to do some "online" analysis. The result of the Archive Check is a text file containing information about archive record usage. You may want to talk to Technical Support about interpretation and advanced options.

             

            As a site note, with the amount of logical cores available with your PI Data Archive node, you may want to increase the amount of threads for PI Snapshot and PI Archive subsystem. I once more suggest talking to a Technical Support engineer.

             

            I am missing one important figure with the hardware specs that you provided and hope that the storage solution (hard disk or solid state disk in combination with a RAID controller) doesn't turn out to be the bottleneck.

             

            Please let us know if you like to be contacted by a colleague from Technical Support.

              • Re: Archive Activity Grid - Performance?
                flost

                Hi Rhys and Gregor,

                 

                thank you for your answers. There are some interesting topics which I will take up and do some considerations on them.

                 

                My PI System is not really heavily used. In total I have 600 users but if there are more then 20 concurrent Users conneted it would be an exeption. I have more Interface nodes sending data to the server then concurrent users reading data

                 

                I have local sto0radge with an RAID 5. I/O is less than 1MB/sec and the disk queue length is less then 0.01 over all partitions. So I think the hardware is ready for future growth ;-)

                 

                I´m aware of KB00335 but due to the fact that it was last updated nearly vour years ago I wanted to ask for current experiences. I can tell that after running AAG for about seven hours there were no significant changes *CPU is equal, RAM ist raised about 0.02 GB but it could be normal szstem behavier) in ressource consunmption regarding CPU or RAM. Unfortunatelz I did not check my disk parameters so I am not able to compare here. Now it is time to go home and since I have no access to mz PI Server over the weekend I turn AAG off. But i was good to see what happens.

                 

                We have some performance counter with the perfmon interface so we have an good idea of ressource consumption over all. We did an Upgrade (to PI2012 and new Hardware) just a few weeks ago. due to the fact that the server is mainly bored the hint with changing the number of threads is verz good. Is there an "Performance Optioimation Guru" avalible or shall I contact TechSupport in gererall?

                 

                Have a nice weekend!

                 

                 

                 

                Florian

                  • Re: Archive Activity Grid - Performance?

                    Hello Florian,

                     

                    The official recommendation is to not exceed the amount of logical cores with the amount of threads for a single subsystem. My personal recommendation for your system would be increasing PIarchss_ThreadCount and PIsnapss_ThreadCount to 40. I wouldn't touch other subsystems unless you are heavily using them and experience poor performance. Because of the strong hardware resources you may want to increase some other parameters like e.g. Archive_FlushQueueSize too. The best way is discussing this within a Technical Support case. Provide your hardware specs and information about your system usage. PI System Sizing and Configuration has some guidance and spreadsheets that will be useful i.e. Hardware Recommendations. The Technical Support Engineer working with you will know when to escalate. You are not dealing with performance issues that require immediate action. Because of this I would be carefully driving the screws and reserve some good portion of potential.

                     

                    RAID 5 is a very reliable storage solution but not the one providing you with the best performance. 1MB/sec is indeed close to idle. With recent hardware > 100MB/sec should be possible. With 600 users, I suggest collecting disk performance counters and to check them from time to time. 

                      • Re: Archive Activity Grid - Performance?
                        flost

                        Helo Gregor,

                         

                        I have an answer from Techsupport. As you suposed they do not recomend changing the thread copunt unless the existing threads are heavly used.

                         

                        Back to the original topic: The message related to AAG was that the amount of RAM used by AAG will increase over time, but with an relatively small PI System which I have it should net become too large. So, I will test it having it run over 24h and have an deeper loock onto RAM.

                         

                        Thanks a lot for the suggestions.

                         

                        Florian