Rick Davin

Passing a list of tag names to the FindPIPoints method

Blog Post created by Rick Davin Employee on May 20, 2019

In this blog we will discuss AF SDK's PIPoint.FindPIPoints method when passing in an enumerable collection of tag names.  As far as immediate usage of the method, there is very little to add that isn't already in the help.  A general weakness of many help examples is that the example is intentionally short, simple, and typically has perfect data.  This blog is being written with the specific thought that some of the tag names passed into FindPIPoints intentionally will not find a tag.  We offer a couple of ideas on what you can do to discover which ones were not found, though any good developer should not feel limited to just the ideas presented here.

 

My inspiration for this blog comes from 2 different partner code reviews hosted months apart.  Both applications used the PIDataPipe where a sizeable list of tag names were being read from a text file.  Despite both developers being quite skillful, I was taken aback that both would try to find each tag one-at-a-time.  Overall, their logic was similar:

 

  • Open the text file and read a tag name for each line.
  • Issue a PIPoint.TryFindPIPoint to find the current tag by its name.
  • If the tag was not found, log it for later diagnostics.
  • If the tag was found, issue a AddSignups for that tag, even if it is the only tag in a list.

 

As I mentioned, these developers were quite skilled and knew about bulk calls.  This was apparent in each application where they would later issue a RemoveSignups in bulk.  When I asked one of them why they would read 5000 tags one-at-a-time instead of in bulk, the answer was all about the logging.  It was critical to their application that it logs whenever a tag is not found.

 

The thing about FindPIPoints is that it only returns what is found (doh!).  If you pass in 1000 tag names and 100 of them are not found, then what you get back is 900 PIPoints.  What we will explore here are ways to quickly, easily, and efficiently discover which 100 were not found.  That is the missing piece as to why these developers were not using a bulk call.  This is not a hard task to code up, and the possible solutions covered here rely more upon simple .NET Framework objects, and very little to do with AF SDK (that is, beyond the FindPIPoints call).

 

Case Insensitive Dictionary

The coding challenge is to reconcile the list of tag names you pass into FindPIPoints and quickly determine which ones didn't make the cut.  As an application developer, my first choice to solve this would be a dictionary keyed by the tag name (a string) with a lookup value of a PIPoint.  I would also want this dictionary to be case insensitive just in case your exact casing of the tag name in the text file does not exactly match the casing of the tag name on the PI Data Archive.

 

The logic would be that every tag name you pass in will be a key in the dictionary.  We then will make a one bulk call to FindPIPoints.  We enumerate over the returned list of PIPoint to populate the dictionary with the found PIPoints.  Afterwards, any remaining dictionary entry that does not have a valid PIPoint would indicate a tag name that was not found.  The biggest takeaway is not the mechanics of the dictionary, but rather that we issued only one efficient bulk call to the PI Data Archive.

 

Since my example method calls FindPIPoints and FindPIPoints has an optional 3rd parameter of attributeNamesthen I too want my custom method to mirror the signature of FindPIPoints. 

 

LEGAL DISCLAIMER

Since we will share some sample .NET code, we must provide the obligatory legal notice and disclaimers.  The sample code being shared herein is subject to the following disclaimer:

 

Copyright 2019 OSIsoft, LLC.

 

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

 

http://www.apache.org/licenses/LICENSE-2.0

 

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

 

To put this another way, if you have problems with these sample methods, do NOT call Tech Support, nor create a case in my.OSIsoft.com. The only outlet for discussing issues with this code is here within the PI Developers Club forum.

 

The code being shared below:

  • Has not been put through any regression testing.
  • Has a very limited test profile (only me on my local PI Data Archive)
  • Has not been endorsed by any OSIsoft Product team
  • Is in no way an official offering by OSIsoft
  • Is offered only as a learning example

 

With that said, we strongly advise that you do not use this in your production code.  Keep in mind that we cannot stop you from doing so, but if you do, then it's no longer our code but rather is becomes YOUR code to support, maintain, validate, debug, and assume the full responsibilities as owner of said code.  Such is the nature of an "AS IS" license.

 

C# Sample Implementation Returning a Dictionary

public static IDictionary<string, PIPoint> GetExampleDictionary(PIServer piServer, IEnumerable<string> names, IEnumerable<string> attributeNames = null)
{
    if (piServer == null)
        throw new ArgumentNullException(nameof(piServer));
    if (names == null)
        throw new ArgumentNullException(nameof(names));

    // We will use a case-insensitive dictionary keyed by tag name with a Value of PIPoint.
    // A null PIPoint would indicate a tag name that was not found.
    var dict = new Dictionary<string, PIPoint>(StringComparer.OrdinalIgnoreCase);

    // We want every tag name to have an entry in our dictionary.
    // Initially the PIPoint will be null, which we try to populate shortly below.
    foreach (var tagname in names)
    {
        if (tagname != null)
            dict[tagname] = null;
    }

    if (dict.Count == 0)
        return dict;

    // Live Library: https://techsupport.osisoft.com/Documentation/PI-AF-SDK/html/M_OSIsoft_AF_PI_PIPoint_FindPIPoints_2.htm
    // If a tag name is not found, an exception is not thrown nor is a null PIPoint returned.
    // FindPIPoints will only return PIPoints that were actually found.
    var tags = PIPoint.FindPIPoints(piServer, names, attributeNames);
           
    // Finally we will assign the found PIPoints to its proper entry in the dictionary.
    // Note that some dictionary entries may still have null PIPoints for those tag names that were not found.
    foreach (var tag in tags)
    {
        dict[tag.Name] = tag;
    }

    return dict;
}

 

Great.  We issued a bulk call and thanks to a tiny bit of effort on our part, we now have a handy little dictionary at our fingertips.  Let's see an example on how we could consume this dictionary depending upon a given context of working with tags that have been found & validated, or working with tag names needing to log a "Not Found" message.  For setting up this example, we want to have a few good tag names along with some bad tag names.  For the bad ones, let's include some crazy edge cases of wild card patterns being used, duplicate names with different casing, or even a tag name containing illegal characters!

 

string wicked = "* ? ; { } [ ] | \\ ` ' \" ,"; // Every character not allowed in a tag name
string[] tagnames = new[] { "CDM158", "SINUSOID", "sinusoid", "sinus*", "This tag does not exist", "", "     ", wicked };

PIServer pida = new PIServers().DefaultPIServer;

var dict = LearningExample.GetExampleDictionary(pida, tagnames);
var points = dict.Values.Where(x => x != null).ToList();
var badNames = dict.Where(x => x.Value == null).Select(x => x.Key).ToList();

Console.WriteLine($"PIPoints Found: {points.Count}");
foreach (var point in points)
    Console.WriteLine($"   {point.Name}");

Console.WriteLine($"PIPoints NOT Found: {badNames.Count}");
foreach (var badName in badNames)
    Console.WriteLine($"   '{badName}'");

 

And that sample code might produce the following:

 

Sample Console Output
PIPoints Found: 2
   CDM158
   SINUSOID
PIPoints NOT Found: 5
   'sinus*'
   'This tag does not exist'
   ''
   '    '
   '* ? ; { } [ ] | \ ` ' " ,'

 

The help warns us that we cannot use wildcard patterns, so we understand why 'sinus*' is not found.  Observe that I even threw in some blank tag names, one with zero length and one with many blanks (which is why I quote the ones that are missing).  And the last line tells me that an exception will not be thrown even if you use illegal characters in the tag name being searched.

 

All my consuming application needed to do is take a little handling of whether a PIPoint was found or not.  Simple concept, simple code, but big performance difference when dealing with thousands of tags.

 

VB.NET Sample Implementation Returning a Dictionary

Public Function GetExampleDictionary(ByVal piServer As PIServer, ByVal names As IEnumerable(Of String), Optional ByVal attributeNames As IEnumerable(Of String) = Nothing) As IDictionary(Of String, PIPoint)

    If piServer Is Nothing Then Throw New ArgumentNullException(NameOf(piServer))
    If names Is Nothing Then Throw New ArgumentNullException(NameOf(names))

    ' We will use a case-insensitive dictionary keyed by tag name with a Value of PIPoint.
    ' A Nothing PIPoint would indicate a tag name that was not found.
    Dim dict = New Dictionary(Of String, PIPoint)(StringComparer.OrdinalIgnoreCase)

    ' We want every tag name to have an entry in our dictionary.
    ' Initially the PIPoint will be Nothing, which we try to populate shortly below.
    For Each tagname In names
        If tagname IsNot Nothing Then dict(tagname) = Nothing
    Next

    If dict.Count = 0 Then Return dict

    ' Live Library: https://techsupport.osisoft.com/Documentation/PI-AF-SDK/html/M_OSIsoft_AF_PI_PIPoint_FindPIPoints_2.htm
    ' If a tag name is not found, an exception is not thrown nor is a null PIPoint returned.
    ' FindPIPoints will only return PIPoints that were actually found.
    Dim tags = PIPoint.FindPIPoints(piServer, names, attributeNames)

    ' Finally we will assign the found PIPoints to its proper entry in the dictionary.
    ' Note that some dictionary entries may still have Nothing PIPoints for those tag names that were not found.
    For Each tag In tags
        dict(tag.Name) = tag
    Next

    Return dict
End Function

 

And here's an example of how VB.NET could consume that dictionary:

 

Dim wicked As String = "* ? ; { } [ ] | \ ` ' "" ,"  ' Every character not allowed in a tag name
Dim tagnames As String() = {"CDM158", "SINUSOID", "sinusoid", "sinus*", "This tag does not exist", "", "     ", wicked}

Dim pida As PIServer = New PIServers().DefaultPIServer

Dim dict = LearningExample.GetExampleDictionary(pida, tagnames, Nothing)
Dim points = dict.Values.Where(Function(x) x IsNot Nothing).ToList()
Dim badNames = dict.Where(Function(x) x.Value Is Nothing).Select(Function(x) x.Key).ToList()

Console.WriteLine($"PIPoints Found: {points.Count}")
For Each point In points
    Console.WriteLine($"   {point.Name}")
Next

Console.WriteLine($"PIPoints NOT Found: {badNames.Count}")
For Each badName In badNames
    Console.WriteLine($"   '{badName}'")
Next

 

Alternatively Return a ValueTuple

There are more than one way to do things and the above was my first train of thought.  I am the type of person that ponders other ways to try things to see which might be faster or easier to work with.  The problem I have with a dictionary being returned is that extra care and handling that must be done.  Other than the fact that you send in one list of tag names, what I really want coming back are 2 mostly unrelated things: I want a list of the PIPoints that were found and I want a separate list of the tag names where a PIPoint was not found.

 

Wanting to avoid the cumbersomeness of using out modifiers on the signature of a void method, I next considered tuples.  In particular, the ValueTuple class.

 

If you are using Visual Studio 2017 with .NET 4.6.2, you will need to install the NuGet package for ValueTuple.  If you are using Visual Studio 2019 with .NET 4.7.2, ValueTuple is now a part of mscorlib.  Tip: if you are converting from VS 2017 to VS 2019 and also upgraded the target framework being used, you will need to uninstall the NuGet ValueTuple package.

 

Internally, the methods will still use a case insensitive dictionary.  The only difference is what type of object we will return.

 

C# Sample Implementation Returning a ValueTuple

public static (IList<PIPoint> FoundList, IList<string> MissingList) GetExampleValueTuple(this PIServer piServer, IEnumerable<string> names, IEnumerable<string> attributeNames = null)
{
    if (piServer == null)
        throw new ArgumentNullException(nameof(piServer));
    if (names == null)
        throw new ArgumentNullException(nameof(names));

    // We will use a case-insensitive dictionary keyed by tag name with a Value of PIPoint.
    // A null PIPoint would indicate a tag name that was not found.
    var dict = new Dictionary<string, PIPoint>(StringComparer.OrdinalIgnoreCase);

    // We want every tag name to have an entry in our dictionary.
    // Initially the PIPoint will be null, which we try to populate shortly below.
    foreach (var tagname in names)
    {
        if (tagname != null)
            dict[tagname] = null;
    }

    if (dict.Count == 0)
        return (new PIPointList(), new List<string>());

    // Live Library: https://techsupport.osisoft.com/Documentation/PI-AF-SDK/html/M_OSIsoft_AF_PI_PIPoint_FindPIPoints_2.htm
    // If a tag name is not found, an exception is not thrown nor is a null PIPoint returned.
    // FindPIPoints will only return PIPoints that were actually found.
    var tags = PIPoint.FindPIPoints(piServer, names, attributeNames);

    // Finally we will assign the found PIPoints to its proper entry in the dictionary.
    // Note that some dictionary entries may still have null PIPoints for those tag names that were not found.
    foreach (var tag in tags)
    {
        dict[tag.Name] = tag;
    }

    // The variable 'tags' has the found PIPoints.
    // Let's create a list of tag names of any PIPoints that were not found, if any.
    var missing = dict.Where(x => x.Value == null)?.Select(x => x.Key);

    // Return a tuple of the found tags and the missing tag names.
    // The consumer can pass tags directly into a PIDataPipe.AddSignups,
    // and log the missing tag names.
    return (tags, missing.ToList());
}

 

C# allows different ways we can consume this object.  Both C# and VB.NET allow you to grab the returned ValueTuple as an singular object and then use its different fields:

 

var tagLookup = pida.GetExampleValueTuple(tagnames);

Console.WriteLine($"PIPoints Found: {tagLookup.FoundList.Count}");
foreach (var point in tagLookup.FoundList)
    Console.WriteLine($"   {point.Name}");

Console.WriteLine($"PIPoints NOT Found: {tagLookup.MissingList.Count}");
foreach (var badName in tagLookup.MissingList)
    Console.WriteLine($"   '{badName}'");

 

In the first line above, the tagLookup.FoundList will be a IList<PIPoint> and the tagLookup.MissingList will be an IList<string>.

 

The other nice way (which VB.NET lacks BTW) is to decompose the fields into individual variables as they are declared:

 

var (points, badNames) = pida.GetExampleValueTuple(tagnames, null);

Console.WriteLine($"PIPoints Found: {points.Count}");
foreach (var point in points)
    Console.WriteLine($"   {point.Name}");

Console.WriteLine($"PIPoints NOT Found: {badNames.Count}");
foreach (var badName in badNames)
    Console.WriteLine($"   '{badName}'");

 

In the first line above, the variable points will be a IList<PIPoint> and the variable badNames will be an IList<string>.

 

VB.NET Sample Implementation Returning a ValueTuple

<Extension()> Public Function GetExampleValueTuple(ByVal piServer As PIServer, ByVal names As IEnumerable(Of String), Optional ByVal attributeNames As IEnumerable(Of String) = Nothing) As (TagsFound As IList(Of PIPoint), MissingTagNames As IList(Of String))

    If piServer Is Nothing Then Throw New ArgumentNullException(NameOf(piServer))
    If names Is Nothing Then Throw New ArgumentNullException(NameOf(names))

    ' We will use a case-insensitive dictionary keyed by tag name with a Value of PIPoint.
    ' A Nothing PIPoint would indicate a tag name that was not found.
    Dim dict = New Dictionary(Of String, PIPoint)(StringComparer.OrdinalIgnoreCase)

    ' We want every tag name to have an entry in our dictionary.
    ' Initially the PIPoint will be Nothing, which we try to populate shortly below.
    For Each tagname In names
        If tagname IsNot Nothing Then dict(tagname) = Nothing
    Next

    If dict.Count = 0 Then Return (New List(Of PIPoint)(), New List(Of String)())

    ' Live Library: https://techsupport.osisoft.com/Documentation/PI-AF-SDK/html/M_OSIsoft_AF_PI_PIPoint_FindPIPoints_2.htm
    ' If a tag name is not found, an exception is not thrown nor is a null PIPoint returned.
    ' FindPIPoints will only return PIPoints that were actually found.
    Dim tags = PIPoint.FindPIPoints(piServer, names, attributeNames)

    ' Finally we will assign the found PIPoints to its proper entry in the dictionary.
    ' Note that some dictionary entries may still have Nothing PIPoints for those tag names that were not found.
    For Each tag In tags
        dict(tag.Name) = tag
    Next

    ' The variable 'tags' has the found PIPoints.
    ' Let's create a list of tag names of any PIPoints that were not found, if any.
    Dim missing = dict.Where(Function(x) x.Value Is Nothing).Select(Function(x) x.Key)

    ' Return a tuple of the found tags and the missing tag names.
    ' The consumer can pass tags directly into a PIDataPipe.AddSignups,
    ' and log the missing tag names.
    Return (tags, missing.ToList())

End Function

 

As mentioned above, VB.NET is unable to decompose the fields on one single line. Thus you must use the entire ValueTuple object and work with its fields.  Here is a simple example:

 

Dim tagLookup = pida.GetExampleValueTuple(tagnames)
Dim points = tagLookup.TagsFound
Dim badNames = tagLookup.MissingTagNames

Console.WriteLine($"PIPoints Found: {points.Count}")
For Each point In points
    Console.WriteLine($"   {point.Name}")
Next

Console.WriteLine($"PIPoints NOT Found: {badNames.Count}")
For Each badName In badNames
    Console.WriteLine($"   '{badName}'")
Next

 

Conclusion

The code above has not done anything special with the FindPIPoints method.  In fact, we are using it exactly for the purposes of which it was intended, and also exactly in the manner in which it was intended.  All we really did was considered various ways to take the results returned by FindPIPoints and transform them into another object more conducive to certain applications.  We then threw both good and bad cases at it to see if that transformation behaves as we expected.  These sample methods just ever so slightly touched upon AF SDK, and the subsequent transformation relies on very common, very routine .NET objects.

 

We showed 2 possible ways for you to transform the results.  And I am sure these are not the only ways.  I encourage you to contemplate these and other techniques to find what works best for you and your company based on your coding experience, preferences, style, and company policies.

Outcomes