12 Replies Latest reply on Aug 1, 2012 5:31 PM by Lonnie Bowling

    Sample data




       I was wondering if there is any way to create/simulate sample data in PI. 







        • Re: Sample data

          There are random/rampsoak interfaces shipped with the PI system to create "moving" PI tags.  The manual "PI_Random.doc" manual should help you out with those tags.

          You can (depending on your licencing) use OSI's Chocolate Milk Factory simulator for examples/test applications OR write your own simulator (PISDK).

          There is even the option of connecting an OPC server and getting a simulator that talks OPC and in turn to PI.

          • Re: Sample data



            during the installation of the PI server we create a couple of simulation tags (simply search for tags with the pointsource 'R').


            Personally I find it usefully to make copies of these tags, especially SINUSOID and CDT158, to get more tags with simulated data. As Rhys said the manual of the random and rmp_sk interface can help you understand how to create tags that simulate data more customized to your needs.



              • Re: Sample data

                [Administration note: Moving this discussion thread to the "PI System Integration and Administration" forum]


                Just to add on what Andreas and Rhys said, the manuals for the Random and RampSoak Interfaces are installed with the PI Server and are located in the \PIPC\Interfaces\Random and \RmpSk folders.


                As per Andreas' suggestion, it is relatively easy to duplicate tags with the PointSource attribute set to 'R' or '9' - such as CDT158 - in multiple copies to get data generated around various center points and following various patterns. This is easily feasible using the PI TagConfigurator add-in to Microsoft Excel.



              • Re: Sample data

                I've written a utility that I find very useful for demoing systems/prototypes that are getting data from a PI Server. It takes some command line arguements (the name of a .csv file (could be .xls, txt, etc.), a time interval, etc.), then opens the file, gets the tag names from the first line of data (if they don't exist in the PI Server, it creates them), the sets up a timer (using the command line interval) and every timer tick, reads the next line of data from the file and pokes the values into the PI tags (current timestamp). My current incarnation stops at the end of the data but you can have it loop back to the first data record to keep the simulation going.


                You can pull the data of interest out of a real PI archive and then feed it back in as "simulated" data at whatever playback rate you want.

                • Re: Sample data

                  Just to add on all these valid suggestions by RJK, Andreas, Richard Marcell and myself, I think the solutions really depends on your needs for creation/simulation of data. Take the "amount of data" factor: I would definitely suggest a different solution for a "5 tags, 5 days worth of 5 minute interval data" case, than for a "5000 tags, 5 years worth of 5 seconds interval data" case.


                  Another important factor is the frequency at which you have to perform that generation of data. 1 time only? Frequently and on-demand? Daily? Every minute? On an ongoing basis?


                  Again, this is in addition to other valid solutions that were suggested (e.g. Random and RampSoak Simulator Interfaces):

                  • Small loads of data, "1 time only" or "on-demand": PIConfig scripts
                    (see Chapter 10 "The PIConfig Utility" of the PI Server System Management Guide, under the vCampus Library)
                  • Bigger loads of data, frequent/ongoing needs: custom application or, for the sake of simplicity, performance and reliability: the PI Universal File Loader (UFL) Interface. (see its Product Page for more information and access to its user manual).
                    Or the older PI BatchFile Interface, although the PI UFL Interface should be used instead.

                  In any case, do not hesitate to request assistance from OSIsoft! The vCampus discussion forums for programming/scripting and our Technical Support team (techsupport@osisoft.com - +1-510-297-5828) for regular PI Interface concerns.

                    • Re: Sample data
                      Nick D'Orazio

                      Richard: very neat application. That must be great for training.


                      I think one of the best generators of simulated data is the one Buck Bard prepared for instructors to use when teaching Batch on the server and BatchView clients. You can find it at http://training.osisoft.com > Downloads > Quick Start Kits.  (The download link is on the right side of the page.) It produces regular mock process data that repeats in batches, so that the data actually looks like process data that repeats at intervals but never the same way twice.


                      Along these lines, there will be a webinar on April 16th on handling recalculations in Performance Equations and ACE. One of the things I'll demonstrate is how to fill a gap in the history of a calculation, which is somewhat close to this topic on Sample Data. You could for example create a dummy calculation tag and then produce data for a period of time in the past using a calculation going against existing archive data or even generating mock data using trig functions (sin, cos, etc). Here's a description of the webinar:


                      "In this webinar we teach how to use the Performance Equations Recalculator and the PI ACE Manager to recalculate previously calculated and stored values when the input values of those calculations have changed. We also teach how to manually force a recalculation to existing calculated values after you have changed the logic of the calculation expression. Finally, we will teach the procedure recommended to fill a gap in the history of a calculated tag using recalculation."


                      It will be available as a replay after the 16th.  


                      ** update **


                      Registration is now open for the webinar. You can register at http://training.osisoft.com/Downloads/Webinars/Training_Webinars.htm. The training webinar series is available to customers with a current Software Reliance Program (SRP) agreement. You will be prompted for your Tech Support SSO (single sign-on) account credentials when you register so that we can verify SRP status.





                        • Re: Sample data
                          Pablo Benvenuto

                          The link mentioned above by Nick to the simulated data is no longer available.  However, you can visit the Batch class webpage located here: http://learning.osisoft.com/Batch_Class.aspx.  On the right margin of that page you will find a link to download the class material.  The class material includes all the simulation tags and everything else you need for the class exercises.  The link can be access directly here: techsupport.osisoft.com/.../DownloadCenter.aspx

                            • Re: Sample data

                              I came across this thread while searching for a way to playback or simulate real data for applications that don't have a concept of history. An example of this would be any application that is connected to the PI OPC DA server, or in my case, writing to a different vendor's OPC DA server using the OPC interface.


                              Essentially you can do this with UFL as suggested by Steve, and use a bat file to feed the files into the UFL directory at the desired data frequency to achieve realistic looking playback. UFL is devastatingly fast and for processing 1000+ playback tags, there was no user noticiable delay.


                              The bat file generate the input files themselves, from a big file that contained all the data (from a piconfig dump). The big dump file was sorted by timestamp, with a 1 second sample for each of the 1138 tags. OPC_Subsea_1138 is the last tag in a sampling set that indicates a new file should be generated:


                              set /a c=1


                              setlocal ENABLEDELAYEDEXPANSION


                              for /F "tokens=1,2,3,4 delims=," %%G in (28Jun.csv) do (


                              echo %%G,%%H,%%I,%%J >> d:\simulation\files\!c!.txt


                              IF %%G==OPC_Subsea_1138 set /a c=c+1








                              The bat file to feed the files numbered 1->1380 to UFL, the ping command adds a 1sec delay:


                              FOR /L %%G IN (1,1,1380) DO (


                              copy /Y %%G.txt UFL


                              PING -n 1 -w 1000 >NUL




                               Finally the UFL .ini file uses the Now() function so all the values go in with the current snapshot:


                              StoreInPI(Tagname, ,Now(),Value, , )


                               Anyway, this was much quicker to me that writing a some PI SDK code, so I thought I would share for future people that might have the same problem.