My 2 cents:
AF DataReferences are always executed on the client side. This indeed explains the behavior of the PIPointDataReference. The configuration is used to retrieve values directly on the PI server. (not via the AF Server). This behavior is the same for every DataRefence, custom or the ones that come with PI AF.
The table lookups are a bit different, they make use of an AF mechanism called AFTables. These are tables that reside on the AF Server. This explains the behavior of the TableLookup DataReference.
You indeed need to have your security configured on both PI AF and PI Archive server. This has to do with the (very dynamic) nature of PI AF. You might say that at this point in time, PI AF (and the AFSDK) are 'loosely' coupled with the PI Archive Server. This allows for some really awesome flexibility, but also requires added security configuration.
Next to that, this architecture allows for a very safe environment. PI AF offers the flexibility of custom DataReferences. This also poses quite a risk. A poorly performing, or poorly developed AFDataReference will always be executed on the client (unless you access it explicitly on the server). This way it cannot easily bring the server down, or have severe performance impacts on the server *. There is quite some risk involved with having these custom datareferences on a server, and I personally like the 'safety' of having them run on clients.
I would also add to this argument that the current security model is a very secure and flexible one. Identities are propagated to both the PI AF and PI Archive Server (if you use the AFDataRefence).
*: Disclaimer: never say never, but it is much more unlikely.
Hi Aldo. This topic receives quite a bit of discussion internally at OSI as well. .
As with every design, there are trade-offs. Here is the reason we have the connect from the AF SDK and not from the AF Server:
1) PI SDK and AF SDK often exist in the same product (e.g. ProcessBook, PI Data Services – which is utilized by PI Web Services, PI Coresight, and PI Webparts). Moving PI access from within the AFSDK to the AF Server would require completely independent security mechanisms and access paths by these same products when using AF ASDK vs. when using PI SDK (In other words, it would introduce conflicting security mechanism you mention in #2, not resolve it)
2) If moving the user identity of the client user to AF Server is required (impersonate or its equivalent), then certain PI authentication methods become problematic (explicit login, trust by a mechanism other than user account). For Windows impersonation, you would be left with requiring a Kerberos delegation configuration by your Domain Administrator.
3) Communication with PI Server can be chatty. Requiring the extra hop from AFServer to PIServer would decrease performanc. AFTable is designed to be chunky, and thus, the extra hop is not as impactful.
4) Often, large enterprises may have a central AF Database, but distributed PI Servers. The extra hop to the AF Server to get to the PI Server could be a large one (the assumption here is that users are accessing their local PI Data most frequently).
5) Increased load on the AF Server box for processing all client data requests.
6) PI WebServices provides a partial solution, by removing AF SDK from your client, but I realize this is limited in its AF Capabilities.
Note in your drawing you have a line between “AF” under Sharepoint and PI Servers. If “AF” is representing the AF Server as it exists today, there should be no line. Sharepoint uses PI-DS which uses AF SDK and PI SDK to retrieve PI Data, hence the line would be the dotted line around the AF Server. In this role, AF SDK is executing in a server role for Sharepoint. (Hence, one man’s client is another man’s server).
However, all that being said, I understand your use case and why you may wish to execute the Data Reference on the AF Server without having to wrap all AF access in another service application. The problem is also not specific to the PI Point Data Reference – Certainly it would exist for any Data Reference that accesses non-PI Data. In addition to your benefits ( 1 and 3), the central access would provide an opportunity for caching common data request (PI-Data Services does this). In summary, your suggestion of an alternate data reference is interesting and we will give it consideration.
Thanks for the reply, this is a good discussion…..I want to comment on some of the points you make because I think you answers are based on the assumption that the OSI products are pervasive on the organization.
My point is that (I think) most large enterprises would like to consider AF as a federated information integration layer…Where consumers of AF (read: can be analytics applications, or end-users) only deal with AF.
Regarding your ‘bullets’
1) Lots of our clients are currently doing (or moving into pro-active production monitoring) in the office environment, integrating information from various sources in order to apply extra ‘intelligence’,
they see the need and value of a federated information model, so the concept of AF is certainly gaining momentum....
BUT…These kinds of organizations often tend to have large architect communities that are focusing on consistent integration patterns and loosely couple solutions that minimize configuration.
It think we must recognize that AF caters for a completely different use case: federated data access for production monitoring purposes
So I do not agree with you on the point you make about independent security mechanisms, I think AF is the logical way to bring structure in access management, certainly since PI carries its historical baggage in terms of security.
2) True and agreed, but access to AF works with windows authentication anyway so implementation of Kerberos is kind of inevitable, certainly for non-PI data sources
4) Could be…when it comes to scale-out scenarios and distribution AF needs some work in that area anyway to make it a real enterprise strength system.
5) True, see my remarks at (4)
6) As you said PI webservices are very limited, and only cather for simple value queries where the asset structure is already know, meta-data services and model support is lacking
7) Regarding your remarks on my drawing and sharepoint: The drawing assumes custom webparts that use the AFSDK to retrieve data from AF.
The standard OSI webparts only cater for the most basic use-cases…in all cases I have thus far experienced clients were forced at some point in time to move away from them and build their own.
Aldo - did you participate in the roundtable discussions at the last vCampus Live! 2011 conference?
We brought up exactly the same points - to position AF as an Enterprise Integration Platform, and we attracted much attention from OSIsoft and the crowd to that subject.
We are also seeing a strong and increasing demand in that sort of a Data Federation tool, and already building solutions based on the AF assuming a role of the Integration Platform, where PI Data is only small percent of the total data presented on that level.
One of the subjects of that discussion back in November was to consider a common server-side caching mechanism in the AF for any data references, including custom ones. So that all data queries would go through the AF Server and hit data sources only if cache for them has expired. Pretty much like AF Tables/TableLookup now, although there is no server-side data caching mechanism for them now.
All that being said, I understand Chris's concerns on high data throughput requirements, which may bring the AF Server down. We also discussed that back in November, and one of the possible ways to handle that could be Load Balancing of AF Collective on top of the standard AF HA architecture.
Security concerns in the enterprise, raised by Aldo - are very important.
Particularly when you need to centralize configuration of user access to the Data and restrict it on the level of "Asset Hierarchy" rather than PI Tags. So that particular users may only see certain parts of the Asset, including all the PI Tags linked to that.
Kerberos in most cases is not a problem, as we have to configure it anyway, when we work in multi-level environment, with Sharepoint, AF, PI and other datasources, where double-hop issue is out there by definition.
No I did not participate......I was however involved in the product selection and later on implementation of an enterprise Data Services Layer (as we call it) for a large dutch oil & gas firm ;-) ...so I am talking from experience.
You have my vote in asking for a server side cache....Especially the lack of an embedded cache with various cache refresh strategies is what is really lacking in AF
As far is the AF HA is concerned.....problem with that is that it is based on sql server replication where there can be only one active 'master'.
AF HA is one of several deployment topologies for AF.
SQL Server replication can support multiple master replication. A big challenge with this configuration is how to resolve conflicts such as primary keys, unique name constraints, etc.
Can you explain your availability requirements for AF? It is possible that one of the supported AF deployment topologies will meet your requirements.
What we are looking for is a sort of Load Balancing idea in AF HA and PI HA configurations, where performance of a single server can be addressed by putting in another server in collective and automatically load balance data reads from the collective.
I agree with Alex, what I was thinking of is this:
In short: a Federated/Sharded database with massive scale-out capabilities.
I myself I am very much looking towards the auto-sharding features to be found in MongoDB..
AF has long supported load balancing of the AF Server. Multiple AF Servers can be configured behind a software or hardware load balancer to communicate with a single SQL SErver cluster or mirror. An AF HA Collective, does support multiple read-only secondaries (AF SErver / SQL Server pairs). Federation as implemented in SQL Azure, does not support dynamic load balancing. It may be possible to configure multiple AF HA Secondary SQL Servers behind a load balancer, but that is an exersize left to the reader.
Future AF may have ... (come to UC 2012).
Incidently, I too am interested in sharding. I can see how to shard EF or searching, but I have not heard of a reasonable algorithm to shard AFElements. Any suggestions?