2 Replies Latest reply on Dec 29, 2017 2:13 PM by gmichaud-verreault

    PI OPC DA interface skipping scans then stops data collection

    caffreys_col

      Hi Everyone,

       

      I have an OPC DA interface running and over the last month or so it has started to exhibit some strange behaviour. The number of scans skipped starts to increase and then it max's out. The heartbeat starts increasing in time scale from 1 second updates to over a minute. The interface then stops collecting data, but the heartbeat continues to work. I can't see anything in the event logs of the server or anything within the PI logs for the interface.

       

      What causes scans to be skipped? Could it be down to the server not running particularly quickly? I've just run pigetmsg and it prints out the lines like an old dot matrix printer, character by character rather than line by line. When we do a reset of the interface software it clears the problem for a while but then comes back after 5-6 days.

       

      The trend bellows shows the change in skipped scans and the corresponding drop in scan counts. I've removed the heartbeat trace as it would block everything else out. The substitution arrows are for the scan count tag mostly, and a couple from the scan skipped tag

       

      Anyone got any ideas as to what might be causing this? I have 3 other OPC interfaces and they are working fine. Below is a screenshot of the results returned from running pigetmsg. The previous summary had scan on time %'s > 97.5% for all classes. In the screenshot below the scans on time are much lower. I'm checking scan class 1 to see if I have any rogue tags configured which may be causing the issue but most of the tags in this group are digital tags so wouldn't be changing at 1 second intervals. Is there anyway to see what tags are sending the most data??

       

      All help greatly appreciated

       

      Col

        • Re: PI OPC DA interface skipping scans then stops data collection
          Steve Boyko

          Typically scans are skipped because the interface can't read all of the tags in the scan class before it's time to read again. This can be caused by too many tags in the scan class and the OPC server can't serve them fast enough.

          Are these polled or advise tags?

           

          What is the OPC server being read by PI-OPC? Does it have logs that you can examine?

           

          When you said "I've just run pigetmsg and it prints out the lines like an old dot matrix printer, character by character rather than line by line", is that being run on the PI interface computer or the PI server? That sure sounds like the computer is running extremely slowly.

           

          Are your other PI-OPC interfaces running on the same interface PC, or a different one?

          • Re: PI OPC DA interface skipping scans then stops data collection
            gmichaud-verreault

            Hi Colin,

             

            Are you seeing any "The OPC server did not respond to the last Refresh call for scanclass N, and has not responded to the previous nnn Refresh call(s)."  messages in the logs? Those would indicate the OPC Server itself being overloaded.

             

            If you are simply seeing scans skipped (as described by Steve Boyko), then you should take a look at pretty much all of your scan classes. In average, you seem to only get 50% of the scans on time.

             

            Suggestions:

            • Depending on your number of tags, you may want to split your interface instance in two separate instances. Since the interface is single-threaded, splitting it in two can make a big difference and allow multiple calls to be made at the same time.
            • Move the majority of your tags in Advise polling instead of polled. Since the tags will be "subscribed" to changes it will also help the interface perform significantly better. If using advise, change your group size (/AM) to a number smaller than default (200-400)
            • Include all of the most time-sensitive tags in advise groups. This should actually reduce overall network load as you are only getting updates from the server when new values happen, whereas Polled tags will make many unnecessary calls back and forth to the server.
            • For your polled tags, the easiest fix is to increase the time for each scan rate. The scan rate is sent to the OPC server, and the OPC server makes a good faith effort to keep data in a cache as fresh as the scan rate. However, whether the OPC server is able to reach out to all of the end devices to do so is not guaranteed. Increasing the scan rate provides the OPC server more time to gather data from the data sources before going back out for another device scan.

             

            Hope this helps,

             

            Gabriel