5 Replies Latest reply on Sep 7, 2010 4:00 PM by wpurrer

    Future of PI / Future Data

    jlakumb

      Note, this is a continuation of this discussion thread

       

      Ok, fine, I can take a hint.  I'll weigh in here on the future of future data, and provide a little more transparency on what is happening with "Verrazano" (code word for our next release).

       

      It turns out we have a number of competing forces for Verrazano (not surprisingly).  On the one hand, there is urgent need to scale up to support higher tag counts for some near-term AMI opportunities.  On the other hand, we have a commitment to support future data for our primary markets, especially T&D industry.

       

      Our strategy is to embark on both efforts simultaneously.  We plan to continue these on parallel paths for now.  Since the PI Server team is switching to Agile development, we have more opportunities to "check in" and determine whether we potentially have a shippable version.  This is one of the reasons why there is no release target for Verrazano (also the fact that we are early in development phase).  Rest assured though that we are currently working on both of these initiatives...

       

      To respond to Rhys, the high level design you describe is very much how we plan to implement future data support.  So when you are ready for a job at OSI, just let us know. ;-)  And Michael if you could expand on the PB "weirdness" with future data currently, please let me know.  One of the things we are planning to do very soon is test our PI clients and data access handling of future data to see whether there are any discrepancies or differences in behavior.  If you guys are aware of any, that is useful information.

       

      At OSIsoft vCampus Live! 2010, Denis and I will be presenting on PI System Tuning and Optimization.  This would be a good opportunity to share any input regarding performance/scalability.  It would also be a good time to discuss your thoughts on the user experience with future data.  I plan to have a questionnaire available for folks to comment.

       

      Hope that helps.  Looking forward to seeing you guys next month.

       

      Regards,

       

      Jay

        • Re: Future of PI / Future Data

          Jay Lakumb

          To respond to Rhys, the high level design you describe is very much how we plan to implement future data support.  So when you are ready for a job at OSI, just let us know. ;-)

           

          Everyone has their price (Oh, and great minds think alike.)
          Actually, future data storage would be one nice project especially expanding the scope to the client PI trend control and Event Frames.

           

          Jay Lakumb

          At vCampus Live, Denis and I will be presenting on PI System Tuning and Optimization.

           

          Denis Vacher?  I would have liked to have met him (unless I already have and don't know it ).

           

          Thanks for taking the hint to respond.

          • Re: Future of PI / Future Data
            MichaelvdV@Atos

            Jay Lakumb

            Ok, fine, I can take a hint.  I'll weigh in here on the future of future data, and provide a little more transparency on what is happening with "Verrazano" (code word for our next release).

             

            Thanks for taking the time to respond!

             

            As for the codename, I guess it's named after Giovanni da Verrazzano, who came to a somewhat horrific demise. Let's hope this doesn't happen to the release

             

            Jay Lakumb

            And Michael if you could expand on the PB "weirdness" with future data currently, please let me know. 

             

            We are working on documenting the 'weirdness'. My collegue is working together with Martin Freitag (OSIsoft) to explore the issues. However, at this point we are not certain it's the COM connectors 'fault', or it's something in PB. We will let you know!

             

            Jay Lakumb

            Looking forward to seeing you guys next month.

             

            Same here, still too bad Rhys can't come

            • Re: Future of PI / Future Data
              mhalhead

              Jay Lakumb

              On the one hand, there is urgent need to scale up to support higher tag counts

               

              I recall another UC presentation showing PI doing 23,000,000 tags. On comparitively modest hardware. I'm curious how big OSI is aiming for. My company hasn't even breached the million mark collectively.

               

              It is good to hear about the future data work; this will definitely help. I won't ask when!

                • Re: Future of PI / Future Data
                  mhalhead

                  Michael @ Atos Origin

                  The discussion has moved away from 'AF and Silverlight' to 'future improvements of the PI system'. Maybe we should continue this most interesting discussion in a new topic.

                   

                  I noticed this but the discussion was interesting. So I continued with it. As a side note I did resolve my Silverlight issue.

                   

                  I'm including the whole of Jay's post for clarity from the AF and Silverlight discussion.

                   

                  Jay Lakumb

                  Michael, I agree that manageability is tightly coupled to scalability. So we are also considering the system administration experience when we look at ways to scale the PI System to 20MM+ tags.  That said, I would like to ask a couple questions about tag tuning just to make sure I understand what problem you are trying to solve.  First, what are the reasons for tuning tags to begin with?  I can think of a couple (e.g. reduce disk storage, increase query performance), but want to confirm since you mentioned this is to "make these extremely large systems manageable". Can you please explain how you see that tag tuning might help with manageability?  Second, you mentioned the PI Tuning Tools from Exele, which seem to meet your needs.  Perhaps you could explain what tasks in these tools are difficult/repetitive/manual which would be best to automate (and perhaps suggest to Exele)?

                  By the way, we are well aware of the, shall we say, less than optimal performance/scalability of our various analytics tools. This is a key are of focus in our roadmap. Look to hear more about this (probably starting next year).

                   

                  Starting with the first question; the reason for tuning tags. PI has good data compression but it can be brutal if you get the settings wrong and the data comes out as flat lines. Which is hardly ideal. Storing the data with compression turned off seems a little silly; I don't care much about hard disk space as this is cheap but under compressed data will make querying slower and will impact the performance of ACE, AF, performance equations and summary calculations. One of the biggest attractions to PI for me is the speed (and the excellent tech support).

                   

                  How tuning relates to manageability: In the end of the day it is the data that is important; that is what the users see. I have a small team that looks after the whole process information side of the company; my team consists of 4 people (all engineers mostly chemical engineers) including me. This team manages everything from roll out to day to day maintenance to projects. Our PI systems aren't small ranging in size from ~50k to ~100k (most are in this category) we currently have 8 PI systems and will increase to 11 shortly (once I deal with the documents). We mostly use Siemen PCS7; an average setup of PCS7 will contain about 400k-600k tags. Most of these tags are rubbish or unused. So we've written some tools to help import and update tags directly from the control system based on block types. Unfortunately, not all the settings in the DCS are ideal for logging. Which results in a post import cleanup. This is quite time consuming and to be honest no engineer like do this. A lot of the tags aren't looked at on a daily basis so no one notices that they are over compressed until someone needs that data. Therefore improving this would definitely improve the system.

                  • Automated reporting on PI tuning; including Bad and Stale tags. The current bad and stale tags module in SMT isn't great. I've been considering setup up a simple SQL database that collects the average archive event counts over a time window and generates a report every morning showing the 10 worst under compressed (high point count) and ten worst over compressed (low point count) tags. This would be a single report covering all our PI servers. The reason for the time window is to average plant disturbances.
                  • Centralising the logs particularly the interface logs so that the PI admin can see all the information in one place.

                  I've decide that I'll investigate the Exele tool a bit more.

                    • Re: Future of PI / Future Data
                      wpurrer

                      I want to remember there are some posts in the past...

                       

                      http://vcampus.osisoft.com/forums/p/957/4614.aspx#4614
                      http://vcampus.osisoft.com/forums/t/1039.aspx

                      And a couple more which i don't find anymore (hijacking my posts ,...)

                       

                      About future of PI....

                       

                      I think it would be very important to get a roadmap... the roadmap now is very boring and very very slow
                      ... and there are so many things open.  The only thing which was new is the AF 2 MDB synchronisation ... where i don't know if there is a real value.

                      I don't really now for now what value do i get out of the pi - Systems in terms of update and enhancements.

                       

                      the processbooks release cycle from one new version every year is also to slow  (especalliy when there are a couple of bugs and features requests open)

                       


                      if you use AF  - Modeller the axis doesn't work anymore (with some timesettings),...

                       

                      or

                       

                      for example "the osisoft products are still not Office 2010" ready,... (Batch,...)

                       


                      Maybe it would be a good a idea to start something like the connect.microsoft.com very every user can posts enhancments,... and vote for it. (Get Feedback ... ) and this should not be on the vcampus. The mechanism which is used now is non transparent.