Skip navigation
All Places > PI Developers Club > Blog > Authors Sam Pride

PI Developers Club

5 Posts authored by: Sam Pride Employee
Sam Pride

I'm Back! Did you miss me?

Posted by Sam Pride Employee Dec 2, 2010

After 15 months "out in the wild", I've returned to OSIsoft to start a new life as a Centre Of Excellence Engineer !!

 

It's great to be back, I've missed vCampus (I wasn't as successful as I thought I could  be in convincing my last post to give me a vCampus license). But wow, this place has grown over the last year and it's certainly been in great hands. I've learnt a lot whilst I was away and hope to do so as a member of the CoE team. You can be certain, anything I learn will find it's way here (just don't tell my boss!). As a CoE engineer, I'm on the cutting edge of PI technology and get to play with some really interesting PI implementations. What more can a PI Geek ask for?

 

Again, great to be back. You'll see me lurking on the forums, responding when I can, and blogging here when I have something of interest. You might also see me at the vCampus, UC and regional conferences, so be certain to come and say hello! As always, happy to respond to questions, comments either to the forum/blog post or by email: sam@osisoft.com.au

 

 

As promised in my last blog post, we’ll start the discussion of Manual data entry in a HA PI System with the most popular Data Access technology; the PI-SDK.

 

The PI Software Development Kit is a very common technology for accessing the data in PI. The majority of PI Clients use the SDK, many custom applications are built using the SDK and there are countless little code snippets in ProcessBook displays at most PI sites. As such a prevalent technology, it is important to understand how the PI-SDK behaves in a High Availability (HA) environment.

 

Before we get too in-depth with the PI-SDK, we should determine what we really need for HA-based Manual Entry Data (MDE).  Once we have done this, we can easily see how the PI-SDK “measures-up” and what we can do to get it into shape!

 

Ensuring that data sent to a PI server is consistently and reliably stored within the PI Archive is our primary goal. When working with a collective, we need to ensure that data is sent to both members of the collective consistently and be able to ensure that all modifications to the PI Archive are processed on each server. Furthermore, to maintain a Highly-Available system, the final MDE solution should not introduce a “single point of failure”. Concisely, we need Replication and Buffering.

 

The PI-SDK is “collective aware” in the fact that it can seamlessly failover a connection to multiple servers within the collective. Replicating updates to the PI Archive (inserting, editing or deleting events) through the SDK, however, is not as straight-forward. In its current release, the PI-SDK does not support replication or buffering of events sent to PI straight out of the box. We can, however, design a solution using the PI-SDK that satisfies the goals of our MDE system with relatively little effort.

 

As we are all aware, before you can use the PI-SDK for updating the data in the PI Archive, we need to initiate a connection to the PI server. Prior to the PR1 SDK release, calling the open method of a server object would open a connection to that server. PR1 changed this behaviour; the Open method now creates a connection to the collective, instead of a named server. Whilst you may instantiate the server object with a specific server name, there is no guarantee that the SDK will open a connection to that server. The SDK will attempt to connect to each server in the collective until it successfully connects, in order of the server priority. Any modification made to data is only performed against the connected collective member and isn’t replicated to the others. To achieve our first goal, replication, we will need to handle the PI Server connection slightly differently.

 

The PR1 release of the SDK included a new Interface, IPICollective. This interface extends the functionality of the Server object and exposes a series of methods and properties that allow us to properly handle a collective. The MemberOpen method will allow us to attempt to connect to a specific member of the collective (instead of the collective itself). This method accepts the same parameters and behaves similar to Server.Open before the PR1 release. If you cannot connect to the server, it will throw an error instead of failing over to a secondary. Once you have opened a connection to the server, you can use the returned server object as you would normally.

 

Here’s a quick example of connecting to a named PI Server using the IPICollective.MemberOpen method:

 
ProtectedFunction OpenMember(ByVal _serverName AsString, _
              ByVal
_connectionString AsString) As PISDK.Server
 
        Dim srv As Server
        Dim sdk As PISDK.PISDK
        Dim col As IPICollective
        Dim colList As CollectiveList
        Dim colMember As CollectiveMember
 
        'Get a handle on the SDK
        sdk = New PISDK.PISDK()
 
        'set the server to point to our desired server
        srv = sdk.Servers(_serverName)
 
        'Check to see if it is a collective
        col = srv
        If col.IsCollectiveMember() Then
            'If so, locate the correct Collective Member
            colList = col.ListMembers()
            colMember = colList(_serverName)
 
            'Open the member (not the collective)
            srv = col.MemberOpen(colMember, _connectionString)
        Else
            'It is a normal server, open it as you normally would.
            srv.Open()
        EndIf
 
        Return srv
 
    EndFunction

Once we have a server object, we can cast it as an IPICollective object and manage the connection as collective. We can check to see if the server is indeed a collective before trying to open it using the collective-specific functions. Once we have ascertained that it is a collective, we can list all the servers in the PI Collective and get a handle on our specific server. Once we have this, it is just a matter of calling MemberOpen and specifying this handle with the standard Server.Open connection string. The server object can now be used to store and retrieve data using the standard PI-SDK methods.  For more information on the IPICollective and its methods/properties, please consult the PI-SDK documentation. You will also need to ensure the “Replication_AllowSDKWrites” tuning parameter is set on all secondary members (it is not enabled by default)

 

So, using the MemberOpen method, we can now connect to each member of the collective and send data to PI. Each time we send data to PI, we will need to repeat the method for each member of the collective. We can, therefore, satisfy our first goal; Replication. We can only ever have one open connection to the server per process, attempting to open an additional connection will result in an error “This server is already open under a different connection string”.

 

 There are, however, some downsides to calling MemberOpen for each collective member. Each time you close a connection and open a new one with MemberOpen, it takes some time. Constantly switching servers adds overhead and may result in a visible performance loss as the SDK needs to authenticate each time a connection is made. If you are using event pipes, these will be lost when you switch server which may result in lost data or updates. Additionally, each time you open a connection to another server, you potentially orphan existing PI-SDK objects. These orphaned/zombie objects use memory and will increase the working set of your application. As these are COM-based objects, there is a good chance they won’t be cleaned up (Garbage Collection) until the application exists. These issues can be mitigated with good application design.

 

You may be tempted to modify the priority of the server within the collective and then use the SwitchMember method to connect to different servers. Whilst this procedure would still allow you to connect to the other collective members, you have no guarantee that the connection is made against the desired server. If SwitchMember cannot connect to the desired server, it will attempt to make a connection to all others in the collective until it can (ordered by the priority).

 

Now that we can effectively replicate data to the collective, we need to implement a buffering strategy to store writes when a collective member is not available. This is another in-depth topic and, if covered here, would make this quite a long blog post. As such, I will discuss some options in my next blog post in this series.

 

If you have any questions, please feel free to use the comments, the discussion forum or email the vCampus team/myself directly and we will gladly help out.

 

 

As PI Developers, we are often required to bridge the gap between the PI Infrastructure and the customer requirements. A common task is developing solutions for storing "special case" or manually-entered data. Whilst the PI Server provides mechanisms for getting this data into PI, we cannot cater for every possible scenario. There will always be the need for custom development of some description.

 

Prior to the PR1 release of PI, developing these solutions was relatively straight forward.  When PR1 was released (Jan 07), the concept of a Collective was introduced and somewhat complicated the task of persisting manual and special case data in PI. Now, instead of writing to one server, you will need to consider the Collective itself as well as each individual Collective member. Manual Entry/Special Case data is not as straight forward as it used to be.

 

To help clear up the concerns and misconceptions around Manual data Entry, the vCampus team are releasing a whitepaper on the ins-and-outs of Manual entry data in a High Availability (HA) environment. To help kick this off (and get some discussion/thinking started on the topic), I will be releasing a series of Blog Posts outlining the various key concepts covered in the whitepaper.

 

These posts will not replace the content of the whitepaper, they will enhance and demonstrate the concepts presented. And, for those busy people out there, they'll give you the abridged "Too Long, Didn't Read" (TLDR) version.

 

So, watch this space over the next few weeks, there will be plenty to see!

 

P.S. Don't forget to register for the PI Programming 101 Webinar on the 28th of May. You might be surprised by what you learn!

 

 

Sam Pride

vCampus - Australian Style!

Posted by Sam Pride Employee Apr 17, 2009

The first 5 months of the vCampus have been very promising. We’ve already had plenty of great contributions from our user-base, from every corner of the globe. As such, the vCampus team have been looking for a team member from the AsiaPacific region. And they found me….

For those of you who have not met me, I’m based in the OSIsoft Australia office (In sunny Perth, Western Australia). I have been with the company for 3 years now, in a Field Service/Tech Support/Training role and really enjoy working with our customers and the challenges they face.

Before joining OSIsoft, I was a Software Engineer, both contracting out to various clients and as a .Net developer at a local communications company. During my time as a developer, I became quite involved with the local community and have always seen the benefits that user groups and networking provides. The lack of a developer/user community at OSIsoft was one of the first things I noticed when joining the team; I’m glad that vCampus has been established and even happier that I can be a part of it!

Keep a look out for me trolling the forums, responding when I can. I also have a few great ideas (at least I think so) in the pipeline, so “watch this space”. Feel free to email myself or the vCampus team any questions or comments and we’ll do our best to answer them in a prompt manner.

Manual entry data is an important aspect of many customer's PI servers, as is the need for Highly-Available and reliable systems. Unfortunately, using the PI-SDK to store manual entry data is not that straight-forward in a PI-HA environment. 

In the current release of the PI-SDK and the PI-Server, there is no out-of-the-box mechanism in place to replicate SDK writes to all members of the collective. Whilst there are features planned for future releases that will overcome these issues, we need to be able to ensure our manual entry data mechanisms work well with the existing PI version. The good news is, implementing a robust solution for Manual Entry Data (or any PI-SDK writes) in a HA environment is not overly complicated.  This blog post will hopefully outline the considerations and technical details for achieving this (and hopefully not confuse you too much!)

As mentioned previously, the PI-SDK will only write to one collective member at a time. This is further complicated by the fact that, when connecting to a named server, the SDK actually connects to the collective itself and there is no guarantee you will actually connect (or stay connected) to your desired server. To overcome this limitation, the PI-SDK provides a new Interface, IPICollective, which allows us to gather information on the Collective and target an individual server.

The IPICollective interface is a secondary interface supported by the PISDK.Server object that provides extra functionality for dealing with PI Collectives. The IPICollective allows us to access the Collective-specific information (such as the Collective Name, the members in the Collective etc) and also provides some handy methods for establishing and managing connections to various members of the collective.  We will investigate the interesting/important members of IPICollective through a quick example.

Example

The following snippet of code contains a function called OpenMember. This function will try to connect to a collective member if the server name is a member of a collective, else it will connect directly to the PI Server. As this method returns a connected (hopefully) PISDK.Server object, you can use it in place of your regular Server.Open() calls.

 

 

 

.csharpcode, .csharpcode pre {      font-size: small;      color: black;      font-family: Consolas, "Courier New", Courier, Monospace;      background-color: #ffffff;      /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt {      background-color: #f4f4f4;      width: 100%;      margin: 0em; } .csharpcode .lnum { color: #606060; }

 
ProtectedFunction OpenMember(ByVal _serverName AsString, ByVal _connectionString AsString) As PISDK.Server

        Dim srv As Server
        Dim sdk As PISDK.PISDK
        Dim col As IPICollective
        Dim colList As CollectiveList
        Dim colMember As CollectiveMember

        'Get a handle on the SDK
        sdk = New PISDK.PISDK()

        'set the server to point to our desired server
        srv = sdk.Servers(_serverName)

        'Check to see if it is a collective
        col = srv
        If col.IsCollectiveMember() Then'If so, locate the correct Collective Member
            colList = col.ListMembers()
            colMember = colList(_serverName)

            'Open the member (not the collective)
            srv = col.MemberOpen(colMember, _connectionString)
        Else'It is a normal server, open it as you normally would.
            srv.Open()
        EndIfReturn srv

    End Function



The IsCollectiveMember property of IPICollective will return true if the underlying server object is a collective member (or a collective in its own rights). You should always check this property before trying to use the Collective-specific functionality of the IPICollective interface, as any unsupported request (such as determining the Collective name) will throw an error.

Once we have determined that the PI Server is a member of a collective, all we need to do is pass the appropriate CollectiveMember object (which we obtained through the ListMembers() function) to the MemberOpen() method. This behaves like the standard Server.Open() method you most likely have called in the past, meaning you can still supply a connection string. The PISDK.Server object that was returned can be used exactly as you would before.

You may be tempted to use the IPICollective.SwitchMember() method to connect to another PI Server. Whilst this method will allow you to connect to another a member, you have no real control over which member you connect to. The SwitchMember() method uses the CollectiveMember's priority when failing over the connection; it will attempt to connect to the highest priority server other than itself. Calling this method multiple times would result in the connection failing between the Primary and the first secondary, starving any additional secondaries. The only workaround to this is to programmatically update the priority for each member of the collective before switching, which is not ideal.

The final thing you will need to do before you can use this connection to send data to your Collective members is to ensure that the Replication_EnableSDKWriteValues PI Tuning parameter is set to 1. This is disabled by default (it doesn't even appear in the list) and will cause any PI-SDK writes to fail. The parameter does not appear in the SMT Tuning Parameters plug-in by default; you will need to right-click on an existing entry and chose "new".

There are, however, some drawbacks to connecting and disconnecting to the various members using MemberOpen. Steve Pilon eloquently outlines some of these drawbacks in his post on the discussion forums: http://vcampus.osisoft.com/forums/p/97/302.aspx#302. Instead of duplicating Steve's comment I suggest you have a quick read and possibly join in the discussion if you have further questions in this regard. Nevertheless, these drawbacks may be outweighed by the benefits obtained through replication of data and there are also some things you can do to mitigate their effect.
Next Steps

Unfortunately, getting a connection to an individual collective member is only one part of the puzzle. Your application will need to be modified to write to all the SDK members. Furthermore, you will need to account for buffering data to SDK servers that are unavailable. There is a lot to consider when implementing replication of Data, even more so if you wish to pass the ACID test (http://en.wikipedia.org/wiki/ACID). There are many options when choosing how to implement a local cache, such as Database, Flat-Files, far too many to list here. I will address these concerns and the various options in another post (blog or discussion forum).

ACE

Another viable alternative when implementing manual entry data for a HA environment is to use ACE. ACE uses the API to send data to PI which means data updates pass through the buffer. This will allow you to replicate data to multiple PI Servers, yet retrieve data using the PI-SDK (giving you data-access failover). It's important to remember that only updates using the PIACEPoint are sent using the API; any direct PI-SDK calls still behave as they would in your own custom app.

Unfortunately, using the IPICollective methods discussed earlier will not quite work in PI-ACE. The PI-SDK only allows one open connection per application. When you call IPICollective.MemberOpen(), you will most likely receive an error message "The server is already open under a different connection string", as the one PIACEClassLibraryHost process may be shared by multiple contexts (threads) . You should not attempt to close the existing connection, as this is used by PI-ACE to retrieve Data and configuration information and any premature interruption may cause crashes and unexpected behaviour. The bottom line is, when using PI-ACE to write data to PI, use the inbuilt PIACEPoint functionality.

 

 

Final Thoughts


Thanks for sticking through such a long post. I hope it has been useful to somebody! This is not the final word on this subject, there are plenty of slight details I have glossed over (which I hope to cover in a later post) and there are some great products/features in the engineering plan that will greatly improve this problem.

As always, if you have any questions or comments, please feel free to post them here or respond to me personally or the rest of the vCampus team if you would prefer to remain private.  If anybody has any alternate solutions, please, post them up; I'm sure they will be well received.


P.S. All this talk about Collectives has made me think of the Borg - looking forward to the new StarTrek movie (Geek and proud!)

Filter Blog

By date: By tag: