3 Replies Latest reply on Mar 26, 2015 6:31 PM by Rick Davin

    Free App - AF Demo With Timings !!!

    Rick Davin

      Free app here!  Get your free app here!

       

      This post contains an attachment to Visual Studio 2013 Project Source Code.  The download link is at the bottom of the post. [EDIT: Version 2 was attached that cleaned up some bad code.  I discuss in my reply below dated March 24, 2015.]

       

      Fellow PI Geeks - I wrote a Winform app that has a lot of goodies in it!  There’s stuff for developers of all skill levels: entry-level, intermediate, and advanced.  There’s some nice AF best practices stuff plus a couple of AF tricks that I discovered. There’s even many good .NET and UI tricks thanks to awaiting many async tasks.


      TL; DR;

       

      If you think the post is too long and don’t want to read it, just jump to the bottom to download the attachment containing the entire project.  Also close to the bottom are links to other project(s) you may want to check out.

       

      FEATURES

       

      Granted PISystemExplorer has many of these features for you to look at info, but this app provides the source code so you fetch the info in your own programs.

       

      Entry-level Developers can use this to:

      • See how a PISystemPicker and AFDatabasePicker can be linked together.
      • See how to detect what AFSDK client version you are running under.
      • See info about your .NET environment.
      • See info about a PISystem and list the Data References being used.

       

      Intermediate Developers can:

      • See how to get quick counts to database objects.
      • See exactly which items are checked-out! (Seriously that’s worth the price of admission.)

       

      Advanced Developers, there’s an entire tab for you, the [Walk Down] tab with an insane amount of code to drool over:

      • How to find and load elements in bulk using a background task.
      • (Cool Tip!) How to load elements twice as fast by using parallel chunking.
      • (Best Practice!) The fastest way to see if an AFAttribute uses the “PI Point” data reference.
      • (Best Practice!) How to extract the PI Point name from an attribute.
      • (Cool Tip!) How to validate PIPoints for a large number of attributes.

       

      General .NET and Winform UI Tips

       

      Beyond AF, there’s plenty to learn from about .NET, Winforms, and general UJ:

      • Using a Console.Beep() (hey, it was new to me!)
      • How to use a BindingSource to easily populate a data grid view.
      • Awaiting async tasks so that the UI remains responsive.
      • How to hide or show a TabPage on a TabControl.
      • How to use Parallel.ForEach.
      • How to use a range partitioner with Parallel.ForEach to process in more manageable chunks.  Or as I call it, “Lose wait now, ask me how!”

       

      Special Feature – Checked Out Items

       

      Sometimes I just want to know if a database has items checked out.  But other times I want to know what items are checked out by who since when.  An example is provided in this app, although obviously your database must have items that are checked-out to see this feature.  Otherwise it would be like looking for a solar eclipse in the middle of the night.

       

      The [Checked Out] tab:

       

      CheckedOut.png

       

      Snapshot: Data References

       

      DataReferences.png

       

       

      Snapshot: Quick Counts

       

      QuickCounts.png

       

       

      Walk Down Example

       

      You may choose a Bulk LoadElements or Parallel.  The only difference is what appears in the gray shaded cells.  This example used Bulk:

       

      Timing

      Comment

       

      AF Server 'RickAF'

       

      Database  'Rick Database'

       

      Issuing elements = AFElement.FindElements.

      00:00.6

      Found 2588 elements.

       

      Issuing Bulk AFElement.LoadElements(elements).

      00:11.1

      Bulk LoadElements Completed.

       

      Counting PI Point attributes.

      00:01.4

      PIPoint Counting Completed.

       

        Ⱶ̶—> Total  Attributes = 604787

       

        Ⱶ̶—> PIPoint Attributes = 213569

       

      Fetching PI Point Tag Names.

      00:38.3

      TagName Walk Down Completed.

       

        Ⱶ̶—> tagnames.Count = 213569

       

        Ⱶ̶—> First Name: \\RickPI\alpha

       

        Ⱶ̶—> Last  Name: \\RickPI\omega

       

      Validating Actual PIPoint(s).

      00:49.5

      Validation of PIPoint(s) Completed.

       

        Ⱶ̶—> points.Count = 198212

       

        Ⱶ̶—> Mismatch!

       

        Ⱶ̶—> Expected  PIPoints = 213569

       

        Ⱶ̶—> Validated PIPoints = 198212

       

      Here’s just the Parallel section from another run on the same database:

       

       

      Issuing Parallel AFElement.LoadElements(elements).

      00:05.7

      Parallel LoadElements Completed.

       


      But, Wait, There’s More!

       

      You even get a chance to read this nifty disclaimer!

       

      *****  N O T I C E *****

      THIS CODE AND INFORMATION ARE PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND,

      EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES

      OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NONINFRINGEMENT.

       

      Other applications by Rick:

       

      Laggy Network FindPIPoints

       

      Tutorial: A faster way to get PIPoints from a large list of AFAttributes

       

      DOWNLOAD VISUAL STUDIO PROJECT FILE HERE:

        • Re: Free App - AF Demo With Timings !!!
          Ahmad Fattahi

          Great post, Rick!

          • Re: Free App - AF Demo With Timings !!!
            bshang

            Awesome post! Can't wait to play around with this

            • Re: Free App - AF Demo With Timings !!!
              Rick Davin

              I’ve posted a version 2 to this app, and removed version 1 due to some problems.  It takes a big man to admit when he’s wrong, and I am a very big man.

               

              Clarification: this app demonstrates keeping the UI responsive during long tasks.  It is particularly intended to be run on slower networks or against AFDatabases with lots of elements and attributes.  If you have a small database or a really fast network, this app won’t do much for you.

               

              What changed:

              • Ver 1 did a simple count of PI Point attributes, but it did not actually generate a list of them.  Ver 2 does, and the count is now fetched from that AFAttributeList.  This was put to answer that need of “How can I tell which attributes have PI Points?”
              • For that tag names dictionary, it’s now an IDictionary<AFAttribute, string> and the tag name is uppercased.  This was put in to answer the need of “How can I get the tag name from an attribute without the expensive of invoking the PI Point?”
              • Validating the PIPoints now simply uses an AFAttributeList.GetPIPoint() call rather than building it from the available tag names.  While I would like to think using the already fetched names would be faster (and would make sense for this very specific app), in general I don’t see someone going about it that way.  Okay, I’ve seen others just want the tag names without hitting the PIPoint.  But I can’t really see someone wanting to get the tag names and then turning around later get the PIPoints.  The new code reflects that if someone eventually wants the PIPoints that they would skip getting the names and get right to grabbing the PIPoints.

               

              AFAttributeList.GetPIPoint

               

              Given:  AFAttributeList attributes = something;

               

              and assuming that it’s not null, that all items are unique, and all items have been verified to use the PI Point data reference, consider this short call:

               

              var lookup = attributes.GetPIPoint().Results;

               

              For the app, I use a lot of await async Task to keep the UI responsive, so this short one-liner becomes:

               

              ValidatePIPointsSimpleAsync(attributes);

               

              With this ‘simple’ method:

               

              private asyncTask<bool> ValidatePIPointsSimpleAsync(AFAttributeList attributes)
              {
                  //===============================================================================
                  // The AFAttributeList.GetPIPoint() method internally employs parallel threads.
                  //===============================================================================
                  await Task.Factory.StartNew(delegate
                  {
                      _cached.PointLookup = attributes.GetPIPoint().Results;
                              
                  });
                  return true;
              }
              
              
              
              
              

               

              No fuss, no muss. Except I have some odd results when running some tests:

               

              Test Group A – 20,898 attributes across a high latency WAN.

                • Simple Run : 55 seconds

              Test Group B – 213,749 attributes across a medium latency LAN.

                • Simple Run : I STOPPED AFTER 5 MINUTES!

               

              So I do my own custom parallel chunking with a rangeSize: 1000 and threadCount: 8:

               

              privateasyncTask<bool> ValidatePIPointsParallelAsync(AFAttributeList attributes, int rangeSize = 1000, int threadCount = 8)
              {
                  //===============================================================================
                  // I employ some custom parallel chunking.
                  //===============================================================================
                  await Task.Factory.StartNew(delegate
                  {
                      var ranges = Partitioner.Create(0, attributes.Count, rangeSize);
                      var options = new ParallelOptions();
                      options.MaxDegreeOfParallelism = threadCount;
              
                      Parallel.ForEach(ranges, options, range =>
                      {
                          var startIndex = range.Item1;
                          var endIndex = range.Item2;
                          var chunk = new AFAttributeList();
                          for (int i = startIndex; i < endIndex; i++)
                          {
                              chunk.Add(attributes[i]);
                          }
                          var lookup = chunk.GetPIPoint().Results;
                          if (lookup.Count > 0) LockAndLoad(lookup);
                      });
                  });
              
                  // Again, to keep UI happy and responsive, and to have
                  // my await work as planned, I need to return something.
                  return true;
              }
              
              // With parallel threads you must avoid concurrency issues.
              // I either would need a ConcurrentDictionary, or lock a regular Dictionary.
              // To minimize lock requests, each parallel chunk will only update once.
              private void LockAndLoad(IDictionary<AFAttribute, PIPoint> items)
              {
                  lock (_cached.PointLookup)
                  {
                      foreach (var item in items)
                      {
                          if (!_cached.PointLookup.ContainsKey(item.Key))
                          {
                              _cached.PointLookup[item.Key] = item.Value;
                          }
                      }
                  }
              }
              
              
              
              
              

               

              Granted, I didn’t really write that in response to the simple timing out.  I actually wrote it, and tested it first before trying the simple way.  My updated timings would be:

               

              Test Group A – 20,898 attributes across a high(?) latency WAN (ping: 80ms)

                • Parallel Run : 19 seconds
                • Simple  Run : 55 seconds
                  • Simple was run 20 minutes after Parallel

              Test Group B – 213,749 attributes across a medium(?) latency LAN (ping: 2ms)

                • Parallel Run (1) : 50 seconds
                • Simple  Run (1) :  I STOPPED AFTER 5 MINUTES!
                  • Simple (1) was run 20 minutes after Parallel (1)
                • Parallel Run (2) : 58 seconds
                • Simple  Run (2) :  I STOPPED AFTER 5 MINUTES!
                  • Parallel (2) was run 20 minutes after Simple (1)
                  • Simple (2) was run 1 minute after Parallel (2)

               

              [EDIT: on 3/26, I added the actual ping times to the respective servers above.  Since the notions of high, medium, or low latency are subjective, I feel the ping times give a more precise view.]

               

              I did it this way because in general, the first run should be the slowest.