5 Replies Latest reply on Mar 24, 2015 7:16 AM by Roger Palmen

    How does Analytics use processor cores?

    Roger Palmen

      Hi All,

       

      Just checking (have little resources to test at the moment) how AF Analytics service makes use of Processor cores available.

      • Does it scale to use multiple (or all) cores in parallel?
      • How are the analysis groups (and EvaluationPartitionSize setting) determined in the service and how do they relate to threads created?

       

      In other words, if the bottleneck is on processing power, and not I/O or memory & caching, i can just add more cores to make it run faster / support larger loads?

        • Re: How does Analytics use processor cores?
          bshang

          Hi Roger,

           

          Asset-based Analytics will use multiple cores. The analysis grouping is determined by the time rule and dependency. For example, an analysis that evaluates every 10 seconds is in a subclass of an analysis that evaluates every 5 seconds. Dependent analyses are also grouped and run under the same thread, and the total set of analyses are divided into independent groups based on their dependency rank. I'm not sure about the details, but probably some form of topological sorting is used to determine the level of dependency for a particular analyses. Therefore, independent analyses can be run concurrently on separate threads which belong to the same time rule or class.

           

          Regarding performance, it can be improved by adding multiple cores if that is indeed the bottleneck. However, typically, it will be due to I/O and network, and those calls can be controlled via the AFDataCache used by analytics. The settings are described here Analysis service configuration… which you may have seen already.

           

          The CacheTimeSpanInMinutes (default 5 min) and MaxCacheEventsPerAttribute (default 1024) can be adjusted to control the cache size. For troubleshooting performance, you can look at the Evaluation Skipped Count performance counter.

           

          Note also that the entire data flow path must be considered to determine performance. For example, while CPU could in some cases appear to be the bottleneck, increasing the # cores may not help if buffering also becomes a bottleneck (typically limited to 50-100k events/sec).

            • Re: How does Analytics use processor cores?
              skwan

              Just to add a couple of "tips".

               

              Event triggered analyses are grouped together.

              RAM is your friend for PI Analysis Service.

              BufSS 4.3 (shipped with the latest version of AF Server) is considerably better performing than previous versions.

               

              --

              Steve Kwan

              AF Product Manager

              1 of 1 people found this helpful
              • Re: How does Analytics use processor cores?
                Roger Palmen

                Barry,

                Thanks for your answer. A bit as expected. In theory, adding more cores will yield more processing power, but in practice it's more complex and you need to consider the entire chain of dependencies, RAM, caching settings etc.

                 

                One question remains on the grouping. Even if a have a large volume of calculations, if they are all inter-dependant, will they than still only use one thread? Or if we break the EvaluationPartitionSize setting we will still have multiple threads for interdependant calculations?

                And as a side-note. Skipped event count will only increase if you allow load shedding which in a lot of cases, customers don't want that.

                 

                Thanks to all of you to make this a helpful thread!