10 Replies Latest reply on Mar 5, 2012 5:33 PM by pcombellick

    AF Data Access pattern inconsistency






      I recently stumbled upon an issue I would like some insights from the experts on..


      It looks as if the way the AF SDK returns retrieves data from AF and sources differs depending on the type of source.


      It seems that if you query for attribute value that referrers a PI tag the locally running AFSDK client instance in reality gets a reference to the PI tag, after which the AFSDK itself connects to that PI server and than retrieves the data.


      (this behavior can be verified by using a port blocker to block the PI server port 5450 and fire up the AF explorer) ) .


      The behavior for other/relational sources is more as expected: data access is done via AF server and it security mechanism.


      Following picture is a ‘standard’ setup of an AF implementation and a description of the challenge.




      I see the following issues with this behavior:


      1.       No ‘real’ data federation/virtualisation where all data access goes via the AF layer, this violates one of the key attractions of AF.


      2.       (Possibly) conflicting security mechanisms between AF security and client-to-PI security


      3.       IT infrastructure challenges due to :


      a.       configuring PI servers for clients since a consumer of an asset model in AF does not per-se need to be aware of the underlying sources


      b.      Network setup in enterprise environments.




      Assuming my assessment is right (please tell me it is not …)


      The solution to this challenge is quite easy: I ask OSIsoft to offer us a PI Point datarefence that uses the same security mechanism and data access pattern as the table lookup data reference



        • Re: AF Data Access pattern inconsistency

          My 2 cents:


          AF DataReferences are always executed on the client side. This indeed explains the behavior of the PIPointDataReference. The configuration is used to retrieve values directly on the PI server. (not via the AF Server). This behavior is the same for every DataRefence, custom or the ones that come with PI AF.


          The table lookups are a bit different, they make use of an AF mechanism called AFTables. These are tables that reside on the AF Server. This explains the behavior of the TableLookup DataReference. 


          You indeed need to have your security configured on both PI AF and PI Archive server. This has to do with the (very dynamic) nature of PI AF. You might say that at this point in time, PI AF (and the AFSDK) are 'loosely' coupled with the PI Archive Server. This allows for some really awesome flexibility, but also requires added security configuration.


          Next to that, this architecture allows for a very safe environment. PI AF offers the flexibility of custom DataReferences. This also poses quite a risk. A poorly performing, or poorly developed AFDataReference will always be executed on the client (unless you access it explicitly on the server). This way it cannot easily bring the server down, or have severe performance impacts on the server *. There is quite some risk involved with having these custom datareferences on a server, and I personally like the 'safety' of having them run on clients.


          I would also add to this argument that the current security model is a very secure and flexible one. Identities are propagated to both the PI AF and PI Archive Server (if you use the AFDataRefence). 


          You may find the topic about security in the new CTP release of AFSDK interesting, and you could possibly also be interested in this [DEAD LINK] webinar tomorrow.


          *: Disclaimer: never say never, but it is much more unlikely.

            • Re: AF Data Access pattern inconsistency

              Hi Aldo.  This topic receives quite a bit of discussion internally at OSI as well.  .


              As with every design, there are trade-offs.  Here is the reason we have the connect from the AF SDK and not from the AF Server:


              1) PI SDK and AF SDK often exist in the same product (e.g. ProcessBook, PI Data Services – which is utilized by PI Web Services, PI Coresight, and PI Webparts).  Moving PI access from within the AFSDK to the AF Server would require completely independent security mechanisms and access paths by these same products when using AF ASDK vs. when using PI SDK (In other words, it would introduce conflicting security mechanism you mention in #2, not resolve it)


              2) If moving the user identity of the client user to AF Server is required (impersonate or its equivalent), then certain PI authentication methods become problematic (explicit login, trust by a mechanism other than user account). For Windows impersonation, you would be left with requiring a Kerberos delegation configuration by your Domain Administrator. 


              3) Communication with PI Server can be chatty.  Requiring the extra hop from AFServer to PIServer would decrease performanc.  AFTable is designed to be chunky, and thus, the extra hop is not as impactful.


              4) Often, large enterprises may have a central AF Database, but distributed PI Servers.  The extra hop to the AF Server to get to the PI Server could be a large one (the assumption here is that users are accessing their local PI Data most frequently).  


              5) Increased load on the AF Server box for processing all client data requests.


              6) PI WebServices provides a partial solution, by removing AF SDK from your client, but I realize this is limited in its AF Capabilities.


              Note in your drawing you have a line between “AF” under Sharepoint and PI Servers.  If “AF” is representing the AF Server as it exists today, there should be no line.  Sharepoint uses PI-DS which uses AF SDK and PI SDK to retrieve PI Data, hence the line would be the dotted line around the AF Server.  In this role, AF SDK is executing in a server role for Sharepoint.  (Hence, one man’s client is another man’s server).


              However, all that being said, I understand your use case and why you may wish to execute the Data Reference on the AF Server without having to wrap all AF access in another service application.  The problem is also not specific to the PI Point Data Reference – Certainly it would exist for any Data Reference that accesses non-PI Data.  In addition to your benefits ( 1 and 3), the central access would provide an opportunity for caching common data request (PI-Data Services does this).  In summary, your suggestion of an alternate data reference is interesting and we will give it consideration.

                • Re: AF Data Access pattern inconsistency

                  Hi Chris,


                  Thanks for the reply, this is a good discussion…..I want to comment on some of the points you make because I think you answers are based on the assumption that the OSI products are pervasive on the organization.




                  My point is that (I think) most large enterprises would like to consider AF as a federated information integration layer…Where consumers of AF (read: can be analytics applications, or end-users) only deal with AF.


                  Regarding your ‘bullets’


                  1)      Lots of our clients are currently doing (or moving into pro-active production monitoring) in the office environment, integrating information from various sources in order to apply extra ‘intelligence’,  


                  they see the need and value of a federated information model, so the concept of AF is certainly gaining momentum....


                  BUT…These kinds of organizations often tend to have large architect communities that are focusing on consistent integration patterns and loosely couple solutions that minimize configuration.


                  It think we must recognize that AF caters for a completely different use case: federated data access for production monitoring purposes


                  So I do not agree with you on the point you make about independent security mechanisms, I think AF is the logical way to bring structure in access management, certainly since PI carries its historical baggage in terms of security.


                  2)      True and agreed, but access to AF works with windows authentication anyway so implementation of Kerberos is kind of inevitable, certainly for non-PI data sources


                  3)      .


                  4)      Could be…when it comes to scale-out scenarios and distribution AF needs some work in that area anyway to make it a real enterprise strength system.


                  5)      True,  see my remarks at (4)


                  6)      As you said PI webservices are very limited, and only cather for simple value queries where the asset structure is already know, meta-data services and model support is lacking


                  7)      Regarding your remarks on my drawing and sharepoint: The drawing assumes custom webparts that use the AFSDK to retrieve data from AF.


                  The standard OSI webparts only cater for the most basic use-cases…in all cases I have thus far experienced clients were forced at some point in time to move away from them and build their own.






                  Aldo Braam 

                    • Re: AF Data Access pattern inconsistency

                      Aldo - did you participate in the roundtable discussions at the last vCampus Live! 2011 conference?


                      We brought up exactly the same points - to position AF as an Enterprise Integration Platform, and we attracted much attention from OSIsoft and the crowd to that subject.


                      We are also seeing a strong and increasing demand in that sort of a Data Federation tool, and already building solutions based on the AF assuming a role of the Integration Platform, where PI Data is only small percent of the total data presented on that level.


                      One of the subjects of that discussion back in November was to consider a common server-side caching mechanism in the AF for any data references, including custom ones. So that all data queries would go through the AF Server and hit data sources only if cache for them has expired. Pretty much like AF Tables/TableLookup now, although there is no server-side data caching mechanism for them now.


                      All that being said, I understand Chris's concerns on high data throughput requirements, which may bring the AF Server down. We also discussed that back in November, and one of the possible ways to handle that could be Load Balancing of AF Collective on top of the standard AF HA architecture.


                      Security concerns in the enterprise, raised by Aldo - are very important.


                      Particularly when you need to centralize configuration of user access to the Data and restrict it on the level of "Asset Hierarchy" rather than PI Tags. So that particular users may only see certain parts of the Asset, including all the PI Tags linked to that.


                      Kerberos in most cases is not a problem, as we have to configure it anyway, when we work in multi-level environment, with Sharepoint, AF, PI and other datasources, where double-hop issue is out there by definition.