0 Replies Latest reply on Oct 3, 2013 5:51 AM by kmarsh

    An example highly scalable AFSDK Custom Analytic Application

    kmarsh

      Someone had a requirement to monitor 350,000 things in real-time to generate an alarm when a threshold is exceeded.  Many ideas were kicked around to the conclusion that only a custom AFSDK application could be expected to work at that scale and in real time.  This is an example of such a custom AFSDK application.

       

      Here's how it works.  The main loop is:

       

      ReadIniFile(); Gets started by reading some parameters from an ini file in the same folder as the exe.

       

      while (true) {    

       

         ReadConfigElement();  Reads one AF element with several string attributes to get the rest of the parameters, i.e. using AF like Windows Registry.

       

         LoadUpRelevantElements();  Searches up the list of elements matching a specific category (e.g. the 350,000 things to be monitored).  This is the slow part so it is not done often.  For example, maybe every 8 hours or once per day.  It gets from each element one PI point and one alarm limit value.  Nothing is written back to PI in this example, rather alarms are logged in a text file.

       

         SignalThreadsToStop();  Once the new configuration is loaded from the AF, we need to stop threads that may have been working based on the old configuration.

       

         DivideWork();  The fast work of doing history recovery, signing up for updates, and analyzing the data to generate alarms is done in near real-time by fast AFSDK RDA calls and multiple threads.  This routine divides up the PI points and limit values according to how many threads will run.  This goes fast so the threads are stopped only an instant.

       

         SpawnThreads();  Here the threads are run.  These threads have no interaction with the AF, but use only fast RDA calls to the PI server.  They sign up for updates first, then do history recovery, and then continue working in real-time as the main thread moves through this loop, right up until the next SignalThreadsToStop();

       

         WaitForReloadOfConfig();  Once the threads are running there's nothing for the main thread to do but wait until it is time to reload the configuration.

       

         }

       

      Here is an example ini file:

       

      AFSERVER

       

      VMPI2012

       

      AFDATABASE

       

      MyWater Inc

       

      CONFIGELEMENT

       

      aaaConfigElement

       

      USER

       

      Administrator

       

      PASSWORD

       

      WowCow@%#@!

       

      DOMAIN

       

      VMPI2012

       

       

       

      And here is the AF Configuration element.  All attributes are STRINGS - why?  Laziness on my part I guess.

      AFCategory Attribute Water Meter
      HistoryRecoveryStartTime Attribute *-10m
      HistoryRecoveryYesOrNo Attribute Yes
      LastResultTimestamp Attribute 27-Sep-13 16:48:54
      LimitAttribute Attribute High Limit
      ReloadConfigurationPeriodInSeconds Attribute 86400
      ThreadCount Attribute 10
      UpdatePeriodInSeconds Attribute 15
      ValueAttribute Attribute Flow Rate

       

       

      In this case any AF elements of type "Water Meter" would have a "Flow Rate" attribute that is a PI Point data reference, and a High Limit attribute that is just some number.  The configuration will be reloaded every day, and 10 threads will collect data every 15 seconds and generate alarms if Flow Rate > High Limit.