27 Replies Latest reply on Nov 10, 2011 3:54 PM by MvanderVeeken

    AFPlugin configuration as CLR object

    MichaelvdV@Atos

      I'm (still) fairly new to AF development, but something came to mind which I would like to share.

       

      It seems that the only way to store configuration information is trough the configString property. You can save the information as a string, and you have to parse and construct the string to extract the 'property' information.

       

      For my own convienance I have came up with a simple routine that serializes a CLR configuration object (which has properties) to a string, and stores that into the ConfigString property. This object is then used in my custom DataReference configuration form.

       

      I find this very handy, and better to use then then the option used in the examples, where you have to parse and construct the string.

       

      What do you guys think? It seems something like this could save some development time. Could something like this be implemented in AF itself?

      #region Configuration
      private string ConfigString_backing;
      public override string ConfigString
      {
          get
          {
              return ConfigString_backing;
          }
          set
          {
             ConfigString_backing = value;
             SaveConfigChanges();
          }
      }

      public TSDMDataReferenceConfiguration GetConfiguration()
      {
          DataContractSerializer serializer = new DataContractSerializer(typeof(TSDMDataReferenceConfiguration));
          using (MemoryStream s = new MemoryStream(Encoding.ASCII.GetBytes(ConfigString)))
          {
              TSDMDataReferenceConfiguration config = serializer.ReadObject(s) as TSDMDataReferenceConfiguration;
              return config;
          }
      }

      public void SetConfiguration(TSDMDataReferenceConfiguration configuration)
      {
          DataContractSerializer serializer = new DataContractSerializer(typeof(TSDMDataReferenceConfiguration));
          using (MemoryStream s = new MemoryStream())
          {
              serializer.WriteObject(s, configuration);
              using (StreamReader reader = new StreamReader(s))
              {
                  s.Seek(0, SeekOrigin.Begin);
                  string serialized = reader.ReadToEnd();
                  ConfigString = serialized;
              }
          }
      }
      #endregion

        • Re: AFPlugin configuration as CLR object
          cescamilla

          I will take this example and move it out of context.

           

          I really like it, in fact, I like it enough I'll try and create my own implementation and play with strings instead of values for a test project on Databases (yes, I'm planning on doing something fancy like Multiple view conversions, multi-variable columns, partition, multiple keys, etc) and this will come in handy!

            • Re: AFPlugin configuration as CLR object
              MichaelvdV@Atos

              I'm glad you like the idea! I thought it would save a lot of implementation time where you don't have to parse and construct everything :)

                • Re: AFPlugin configuration as CLR object
                  andreas

                  After I got my head around that I really like it - something I will keep in memory for the time I will need it . Thanks for sharing!

                    • Re: AFPlugin configuration as CLR object
                      MichaelvdV@Atos

                      I think it's really easy to use (and maybe should be included as an option in AFPlugin base class ). You basically 'misuse' the ConfigString (which is there basically for technical reasons instead of usability/features I imagine) to store a more user-friendly configuration object.

                       

                      The simplest form could be to basically store a serialized Dictionary<string, string> for key-value pairs.

                        • Re: AFPlugin configuration as CLR object

                          fyi - I recall that AFConfigString is limited to 3500 unicode chars.

                            • Re: AFPlugin configuration as CLR object
                              MichaelvdV@Atos

                              Yes, there is a limitation. I have encountered that using this technique.

                               

                              My remedy was to use the DataContractJsonSerializer. JSON has far less 'overhead' than XML does. It is way more compact, and still very good readable if formatted right.

                                • Re: AFPlugin configuration as CLR object
                                  pcombellick

                                  the limit in AF 2.3 is 3500 UNICODE characters.

                                   

                                  In AF 2.4, the limit has been increased.

                                    • Re: AFPlugin configuration as CLR object
                                      Bannikov

                                      Paul Combellick

                                      In AF 2.4, the limit has been increased.

                                       

                                       

                                       

                                      How many characters would be this limit now? Or it's a very secret?

                                        • Re: AFPlugin configuration as CLR object
                                          pcombellick

                                          Yes, it is a secret, until AF 2.4 is released.

                                            • Re: AFPlugin configuration as CLR object
                                              pcombellick

                                              Server: Limits on Configuration String lengths for Data References, Analysis Rules, Delivery Channels, and Time Rules have been lifted.  An updated client is also required to set a configuration string beyond the previous limits.

                                               

                                              (at the database level, the limit is 2GB .  1 UNICODE char = 2 bytes).

                                                • Re: AFPlugin configuration as CLR object
                                                  cmanhard

                                                  While the limit is raised, restraint is advised.  It doesn't take too many attributes with a configuration string of 10K unicode characters before we are talking about serious amounts of data needing to be transferred from the database to the client - ultimately causing performance issues.  If possible, use non-overridden template configurations so that the configuration can be shared amongst many elements.  

                                                    • Re: AFPlugin configuration as CLR object
                                                      MvanderVeeken

                                                      Chris Manhard

                                                       If possible, use non-overridden template configurations so that the configuration can be shared amongst many elements.  

                                                       

                                                      Could you give us some more detail about that?

                                                        • Re: AFPlugin configuration as CLR object
                                                          pcombellick

                                                          If you don't override certain properties on an AFAttributeTemplate, AF doesn't have to store anything in the database or use memory in the AFSDK for each AFElement that references this AFAttributeTemplate.

                                                           

                                                          For example, if an AFAttributeTemplate has a datareference config string that is 1KB, and there are 1 million AFElements that reference this AFAttributeTemplate and you don't override AFAttributeTemplate config string in any of the AFElement attributes, you will save about 1GB in the database and will save 1KB per element in the client's memory.

                                                          • Re: AFPlugin configuration as CLR object
                                                            cmanhard

                                                            For example, if we have a templated attribute which includes a long formula, where the formula uses other attributes of the same template, and this formula string will be valid for most elements created for the instance, the AF will only transmit that formula from server to client once for the template, and not every element instance.  Additionally, the AFSDK will only hold one string in memory, no matter how many elements from that template are created.  In this case, the 10K used for the formula is not that significant.

                                                             

                                                            Alternatively, if you don't use templates, or, if you redefine the configuration string to be unique for each element, then AF cannot share the configuration amonst element instances.  Thus, as an example, if you attempted to load 50,000 AF elements, each with a single attribute with a 10K in length configuration string - we are suddenly up to 1 GB of data just for these configuration strings(50K*10K*2 bytes per unicode character)

                                                             

                                                            As a more concrete example, I have seen users code a rollup KPI using Formula Data Reference.  In this case, the user is specifically coding the attributes of specific child elements to rollup with the formula.   Thus, every parent element has a unique configuration string.  Thus, a lot of data and memory can get used quickly.

                                                              • Re: AFPlugin configuration as CLR object
                                                                Bannikov

                                                                Chris Manhard

                                                                As a more concrete example, I have seen users code a rollup KPI using Formula Data Reference.  In this case, the user is specifically coding the attributes of specific child elements to rollup with the formula.  

                                                                 

                                                                I think that this is a lack of features in AF itself. We've developed about half-dozen Data Reference plug-ins for AF to perform data aggregation. For example, we have atribute in our parent element named 'Power'. And we have similar attribute in each child element of this parent. The number of children can be vary.

                                                                 

                                                                The value of parent 'Power' should be total (or, in some cases average) of all child elements 'Power' attributes. If we use standard Formula plug-in, then we're forced to make unique tuning for each parent element. With our plug-in we can leave this at template side because configuration of such aggregation is equal for all elements and is template-based.

                                                                 

                                                                 

                                                                  • Re: AFPlugin configuration as CLR object
                                                                    MvanderVeeken

                                                                    Sergey Bannikov

                                                                    I think that this is a lack of features in AF itself.

                                                                     

                                                                    I'm not sure if I'm stating the truth here, but isn't that also the goal of datareferences? To also provide roll-up, averages, etc. from other parts of the model? I'm not sure it is a 'lack of features' in the Asset Framework.

                                                                     

                                                                    It would be however nice to see some more 'standardized' datareferences for this purpose. I think that's also why vCampus hosts community projects. AF is providing us with a Framework, and it is up to us (as developers) to create the plugins to suit our needs. We can share on thoughts and functionality and collaborate to create a 'set' of multi purpose datareferences to fill these requirements.

                                                                     

                                                                    How would you envision the ability to provide aggregates other than with a DataReference in AF ? This is a honest question, as I personally see DataReferences as the ultimate tool to provide flexibility and functionality when leveraging the AF model.

                                                                      • Re: AFPlugin configuration as CLR object
                                                                        Bannikov

                                                                        Michael @ OSIsoft

                                                                        I'm not sure if I'm stating the truth here, but isn't that also the goal of datareferences? To also provide roll-up, averages, etc. from other parts of the model? I'm not sure it is a 'lack of features' in the Asset Framework

                                                                         

                                                                        Hello Michael, you're absolutely right. I've actually mean the lack of 'standard' data references because I suggest that such aggregation features are useful for many applications. And having some 'standard' DR is preferred. I think that aggegation should be done using AF DR. But (because AF DR are executed on client side, bnot on server side) I suggest that OSIsoft can develop some aggregation DR that perform their tasks in cooperation with AF Server (for example, working as front-end to server-side aggregation features).

                                                                          • Re: AFPlugin configuration as CLR object

                                                                            I agree with Sergey here and wanted to re-iterate something I am sure OSIsoft hears a lot; when a customer purchases AF and then installs AF the first thing they want to do is exchange information with System A, System B, System C and finally have some calculations, which is not always possible.  

                                                                             

                                                                            The problem is that out of the box AF needs to ship with more data references as a base layer to connect to different systems (inc. other Process Historians), provide calculations (okay, this one is coming), etc so that deployment of AF is much greater and quicker.  I have seen some projects switch to other integration layers (e.g. Siemens XHQ) because "off-the-shelf" they connect to more systems and require less custom development.

                                                                             

                                                                            I can see the OSIsoft interface developers are going to switch to AF based interfaces soon (if not already) and they could in theory provide some of these 'optimised' data references to other systems, rather than using generic data references...

                                                                             

                                                                            Will we see an OPC UA data reference developed by OSIsoft?

                                                                              • Re: AFPlugin configuration as CLR object
                                                                                cmanhard

                                                                                There is no question about the need for rollup's within AF, and work is in progress to provide this capability out-of-the-box through configured analytics.  Essentially, the experience will not be dissimilar from Data References.  You will be able to configure the rollup at the template level, and have it automatically applied to a hierarchy of elements. Configured Analytics will be able to leverage your existing Data References as well.  Configured Analytics also moves calculations server side - making functions like trending a rollup - more scalable for operations like trending.  

                                                                                • Re: AFPlugin configuration as CLR object

                                                                                  Rhys @ Wipro

                                                                                  when a customer purchases AF and then installs AF the first thing they want to do is exchange information with System A, System B, System C and finally have some calculations, which is not always possible.

                                                                                  The problem is that out of the box AF needs to ship with more data references as a base layer to connect to different systems (inc. other Process Historians), provide calculations (okay, this one is coming), etc so that deployment of AF is much greater and quicker. I have seen some projects switch to other integration layers (e.g. Siemens XHQ) because "off-the-shelf" they connect to more systems and require less custom development.

                                                                                  I think there are a few mixed topics here... "Data References (surfacing data) vs. PI Interfaces (copying data into PI)", "data in (DRs or interfaces) vs. data out (PI Data Access)", and "AF vs. integration layers".

                                                                                   

                                                                                  The good news is, these are all great topics and I am sure Rhys and I are not the only ones interested...
                                                                                  The bad news is, it's going to be difficult to cover these topics effectively in a single discussion thread... especially under "AFPlugin configuration as CLR object" as a unifying theme ;)

                                                                                   

                                                                                  While we could create separate threads on these topics (and I suggest we do), I think we should spend  some quality time discussing this during the upcoming vCampus Live! event... informally throughout the event, but also during the "PI System Integration" roundtable discussion, held on Wednesday at 6pm.

                                                                                  • Re: AFPlugin configuration as CLR object

                                                                                    Rhys @ Wipro

                                                                                    Will we see an OPC UA data reference developed by OSIsoft?
                                                                                    Why a data reference, why not a PI Interface? (note, this should probably have its own discussion thread, as I mentioned in my previous post a minute ago)

                                                                                      • Re: AFPlugin configuration as CLR object
                                                                                        Rick Davin

                                                                                        Pity, Steve.  I understand the need to have an ordered forum on discrete topics.  While the current discussion has grown beyond the original post so much that it could now be deemed ‘off topic’, I’d like to go on record as saying this is one of the most interesting threads in vCampus.  It’s almost as if a narrow topic can give a narrow response but a broader topic can provide a much broader response!

                                                                                         

                                                                                        Maybe we should just retitle this thread ‘A Very General, Ongoing Discussion About AF’?  That way, nothing is off-topic.  But if we must wait for vCampus Live for a more real-time, interactive discussion, I’d be very happy to participate.  I’m sure there will be a lot of interesting topics (and elbows) flying around.

                                                                                          • Re: AFPlugin configuration as CLR object

                                                                                            Rick Davin

                                                                                            I’d like to go on record as saying this is one of the most interesting threads in vCampus.
                                                                                            Agreed! And that's why I'd like to give it more visibility... and I think we can achieve that by generating new discussion threads specifically for the topic. That way other users (not just those those this conversation) can find it and chime in. And those in need for an answer around "AFPlugin configurations as CLR object" will not be confused (d'oh, too late!)

                                                                                             

                                                                                            That said, it's just a suggestion... I'll chime in either way

                                                                                             

                                                                                            Rick Davin

                                                                                            It’s almost as if a narrow topic can give a narrow response but a broader topic can provide a much broader response!
                                                                                            Well I think this is clearly an advantage of vCampus over regular Technical Support (no offense to them, really): having the entire community behind a question (rather than a single Tech Support engineer, even if very knowledgeable), has better likelihood to generate such interesting discussions. I just think we can give these discussions higher visibility by organizing those appropriately (again, just a suggestion

                                                                                             

                                                                                            Rick Davin

                                                                                            Maybe we should just retitle this thread ‘A Very General, Ongoing Discussion About AF’?  That way, nothing is off-topic.
                                                                                            Maybe! We'll see what the vCampus team wants to do, as part of their "moderation duty"

                                                                                             

                                                                                            Rick Davin

                                                                                            But if we must wait for vCampus Live! for a more real-time, interactive discussion, I’d be very happy to participate.
                                                                                            Oh we don't have to wait, we can discuss here... I just think the discussions at vCampus Live! may be even more valuable because such a broad topic generally benefits from in-person conversation...

                                                                                             

                                                                                            Rick Davin

                                                                                            sure there will be a lot of interesting topics (and elbows) flying around.
                                                                                            elbows flying around??  nah...