3 Replies Latest reply on Dec 13, 2012 5:01 PM by Gregor

    PI Testing in a sandbox

    howardh

      We use a virtual sandbox for testing patches and applications before deployment while learning how to get around an updated gui and new/changed features. I recently setup 2 PI server vm's (for PItoPI testing), PI applications and interface vm, existing .Net development vm and an AF SQL2088R2 vm. One of the things we are hoping to accomplish is some performance testing and tuning. Are there any scripts or suggestions on the best way to get this done?

        • Re: PI Testing in a sandbox
          mhamel

          @Dale: We don't have any automated scripts to perform what you want. It would be complicated to aim for a script capable of testing all your requirements and from which we are not aware. My scripts might not be meaningful for you as I might be looking for a different performance than yours. Also some test can be automated and some cannot. When you cannot create automated tests you can write scenario test that you can apply manually and measure the outputs.

           

          As the PI System utilizes different technologies, it would be difficult to come with one or two suggestions on how to perform this testing and tuning. I would like to know what components of the PI System you need to measure performance. Each component would have its series of suggestions or scripts. I can easily think of the following tests:

          • Stress test based on concurrent usage of the PI AF Server. 
          • Narrow but deep (1 attribute over a long time period) data retrieval on AF Attributes performed by a single user.
          • Narrow but deep (1 attribute over a long time period) data retrieval on AF Attributes performed by multiple users. 
          • Wide but superficial (many attributes over a very short time period) data retrieval on AF Attributes performed by a single user. 
          • Wide but superficial (many attributes over a very short time period) data retrieval on AF Attributes performed by multiple users. 
          • Emptying a PI Interface buffer speed after a 3 day disconnection. The goal is to measure the recovery time. 
          • Stress test based on concurrent and sustained queries against the PI Data Archive. An example of this test could consist of sending 10 concurrent PI OLEDB queries or SDK (OSIsoft.PI namespace) complex queries to retrieve many time series data or calculated data and measure the response time of other standard queries against the PI Data Archive. 
          • Try to overwhelm the machine (virtual or physical) containing the PI System components and measure the response time. 
          • Etc.

          What influence the results would be different from one PI System components to another. Some of these parameters are:

          • Tuning parameters table from the PI Data Archive. 
          • SQL Server Settings.
          • Architecture, network topology, etc.
          • Version of software.
          • Virtualization technology.
          • Security schemes (firewall, NAT, GPOs).
          • Operating System and version

          I invite you to share with us what parameters you are looking specifically for your environment. This would more align the type of scripts/scenarios we can propose. I invite other members to share the techniques they use to validate performance of new releases or patches.

          • Re: PI Testing in a sandbox
            howardh

            We have run into issues with the update and network managers due to the number of connections being re-established after a PI Server reboot or stop/start. I have opened up calls beginning back in January of this year. TechSupport Call # 447791is from this month. We have 2 PI servers each running as a vm on VMWare ESXi 4.1. Our entire PI System is running in this environment. There were some technotes applied early on regarding VMWare but did not help. The current 'work around' is to open a vm console to the PI server, disable the active network interface, enable a 'ghost' network interface and reboot the server. Upon startup, wait a few minutes, enable the active network interface and disable the 'ghost'. I want to make sure this is not an issue for us with PI2012.

             

            From what I have read and seen we are really looking forward to the increased performance of PI System 2012.

              • Re: PI Testing in a sandbox

                Hello Dale,

                 

                I took a brief look at call # 447791. It looks to me as if PI Network Manager has become unresponsive, possibly because of network issues. It didn't even accept the RPC from piartool.

                 

                I strongly recommend working with the colleague from TechSupport to identify the root cause. If we don't understand the nature of this issue, we won't be able to propose a fix.

                 

                Did you provide the requested logs for the occasion on 28-Nov?

                 

                Do you possibly have more details about the nature of the issue?

                 

                Did you recognize some pattern or the issue happening under certain conditions or does it just happen randomly?

                 

                Are both collective members affected at the same time or does that happen for one of the nodes only at a time?

                 

                Are you monitoring PI Network Manager performance counters?

                 

                I am very exited about the performance increment of PI Server 2012 too but without understanding what PI Network Manager is / was suffering, you will never gain certainty if the issue is fixed or not.