10 Replies Latest reply on Nov 25, 2011 9:40 PM by Ahmad Fattahi

    PI ACE Start-up time?

    markosem

      Hi,

       

      I have two PI ACE development environments - a developer PC and a test system PC.

       

      On my dev system I have PI Server and ACE all on the same machine. On my test system I have PI Server and ACE running on separate Virtual Machines (using ESXi Server). The test server in general runs faster than my dev PC.

       

      When I start up ACE on my dev PC (by starting the 2.x Scheduler Service), It takes a max of 1 minute for all my contexts to kick into life (approx 5 contexts).

       

      On the test system (again about 5 contexts), it can take up to 5 minutes!

       

      I was wondering what happens when ACE is initialised? What is it doing for that amount of time? My system is quite simple so am a bit baffled to why it takes so long, and why there is such a difference between my dev and test PC.

       

      Is there something I'm doing wrong? Can the start-up time be reduced?

       

      Thanks.

       

       

       

       

       

       

        • Re: PI ACE Start-up time?
          Ahmad Fattahi

          The extra delay on your test machine can be a result of using the virtual machines. Could you monitor the memory/CPU usage on those machines (as well as the hosting physcial machine) to see if that is teh bottleneck?

           

          Also, how populated is the corresponding MDBs? How many tags/aliases through teh MDB are you incorporating in your ACE calculations? MDB performance can get slow as the number of objects grows.

           

          Another place to look is the PI SDK logs. They may show where the time is being spent while the scheduler is starting up.

            • Re: PI ACE Start-up time?
              faisal.ramzan

              Hi,

               

              Thanks for your reply.

              Hardware usage:
              I didn't see any abnormal spikes for memory,CPU, hard disk and network usage.

               

              Number of tags:
              To keep things simple, I am testing it with just one context in this case. My calculation involves 3 input and 7 output tags. There are not that many tags in my PI server (around 150 in total). I deleted all the values in the tags before starting my calculation.

               

              PI Server Message and ACE logs:
              I have put the logs for my ACE and PI server in one file, as I didn't see an option of attaching multiple files to the post. The PIACENetScheduler service is started @ 12:05:04. Looking at PI ACE logs, the actual calculation starts @ around 12:11:30. So it takes around 6.5 minutes to start the ACE calculation, once PIACENetScheduler service is started. I just wondering what is the cause of this delay. Has anyone else seen this before?

               

               

                • Re: PI ACE Start-up time?
                  Ahmad Fattahi

                  Couple of questions:

                   

                  1) What is the frequency and offset of your calculations?

                   

                  2) How is the size/utilization of your MDB? When/if you try to open the MDB, especially the part containing your PI ACE modules, do you see any latency?

                   

                  4) What does your calculation do? Can you start from something very simple, e.g. a calculation that multiplies tag Sinusoid by 2 and writes it back to another tag?

                   

                  If none of these give us a clue you may want to use PI SDK trace logs. For that matter I would suggest you contact our regular Tech Support as they are more equipped to take trace logs and analyzing them. The goal would be to see where the time is being spent.

                    • Re: PI ACE Start-up time?
                      faisal.ramzan

                      >>1) What is the frequency and offset of your calculations?
                      My single ACE context calculation takes in 3 inputs and generates 7 outputs. Each input parameter receives 4 values per second.

                       

                      >>2) How is the size/utilization of your MDB? When/if you try to open the MDB, especially the part containing your PI ACE modules, do you see any latency?
                      My MDB has got 54 modules in it. I don't know how to calculate/find out the size of MDB. Don't see any latency at all while navigating through my MDB.

                       

                      >>4) What does your calculation do? Can you start from something very simple, e.g. a calculation that multiplies tag Sinusoid by 2 and writes it back to another tag?
                      As far as my calculation is concerned, that's the next thing I need to investigate as I see the lag between my inputs and outputs. For the time being I am interested to find out the reason for the delay between the time I start my ACE scheduler service and my first calculation takes place (currently 6.5 minutes).

                       

                      Thanks for looking into this.

                        • Re: PI ACE Start-up time?
                          Ahmad Fattahi

                          Thanks Faisal. Let me make a couple comments about the previous set of questions:

                           

                          1) What I mainly meant was the "scheduling of your calculations" which is a separate configuration than the tags themselves. Do you have it naturally scheduled based on the events happening to a tag or is it clock-scheduled?

                           

                          2) At least on the surface it doesn't look too big of an MDB. We can leave this part on hold for now until we investigate other points.

                           

                          3) Are you able to create a simple calculation that takes Sinusoid, multiplies it by 2 and writes it to another test tag? You can schedule that both ways and see if there is any difference or points of interest/distinction with your original calculation: Clock-scheduled and event-scheduled based on the input tag (Sinusoid). The intention here is to see if the problem is universal across all PI ACE calculations or is specific to your original calculation.

                            • Re: PI ACE Start-up time?
                              faisal.ramzan

                              Sorry, I misinterpret your first question.

                               

                              >>1) What I mainly meant was the "scheduling of your calculations" which is a separate configuration than the tags themselves. Do you have it naturally scheduled based on the events happening to a tag or is it clock-scheduled?

                               

                              My calculation is naturally scheduled based on the events happening to a tag (4 events/sec).

                               

                               

                               

                               

                               

                              >>2) At least on the surface it doesn't look too big of an MDB. We can leave this part on hold for now until we investigate other points.
                              I agree.

                               

                               

                               

                               

                               

                              >>3) Are you able to create a simple calculation that takes Sinusoid, multiplies it by 2 and writes it to another test tag? You can schedule that both ways and see if there is any difference or points of interest/distinction with your original calculation: Clock-scheduled and event-scheduled based on the input tag (Sinusoid). The intention here is to see if the problem is universal across all PI ACE calculations or is specific to your original calculation.
                              Yes, I tried that as per your instructions. The time values shown below represent the time between the start of PI ACE scheduler service and first calculation.

                               

                               

                               

                               

                              Naturally scheduled
                              (1 event/sec)

                              Clock Scheduled
                              (Offset=0, Period=1sec)

                              My simple calculation times

                              6 min 18 sec

                              6 min 8 sec

                              My original calculation time

                              6 min 30 sec

                              -

                               

                               

                              So it looks like the time for simple calculation is pretty much similar to my original calculation. For these simple calculation tests, I also turned off all the security settings (by moving the slider all the way down). This was done to rule out that fact that it might be spending some time trying to authenticate itself somewhere, with all those security settings enabled. But that didn't seem to make any difference.

                               

                               

                               

                              Thanks,
                              Faisal.

                               

                               

                                • Re: PI ACE Start-up time?
                                  Ahmad Fattahi

                                  OK thanks for all the tests. Another point can sometimes result in delays when moving calculations from Development environment to production; that is when the path of some dll's registered and required by PI ACE is different in two environments. Therefore, at startup, PI ACE has to search for the dlls.

                                   

                                  Having said that, I should add that there are multiple other reasons that can be causing this issue. I would recommend you contact our regular Tech Support at this point as they are much more efficient in such troubleshooting issues. They can even start a remote session on your machine to look and resolve on the spot. Also, log analysis is one of the things they can do very efficiently. We will be happy to have more programming-related discussions here on the forum