10 Replies Latest reply on Nov 2, 2009 6:05 PM by danielproulx

    Loading old Data

    mfalkowski

      I have some old data in an excel Spreadsheet, is ther any way to import it into the Pi Server

       

       

       

      Thanks

        • Re: Loading old Data

          Depending on how much data you have in there, the simplest and cheapest way is most likely to use our PIConfig command prompt utility (located in \PI\adm on a PI Server, and in \PIPC\bin on any "PI-but-not-PI-Server" machine)

           

          In a first place, it is good to note tha PIConfig is very powerful and gives access to many tables in PI. You can find great documentation about PIConfig in Chapter 10 of the "PI Server Management Guide", available in the vCampus Library under "vCampus PI Products Set > Server Products > PI Server".

           

          As for backfilling data specifically, there is a nice article called "Backfilling with PIConfig" about that on our Technical Support website, under "Knowledge Center > System Manager Resources". It provides all the commands (which you can script in .bat file) you need to do so.

           

          Hope this helps!

           

           

           

           

            • Re: Loading old Data

              One detail I omitted (although it's explained in the "Backfilling with PIConfig" article): you first need to export the data contained in your Excel files to CSV files (Comma-Separated Values), with the format "tag,time,value". You can easily do that by using "Save As" in Excel, with the .CSV type.

              • Re: Loading old Data
                mfalkowski

                Steve,

                 

                I have access to one of our clients who is willing to spit out as much data as we need to import into our Test Pi Server.  If I just ask them to make a backup file and then Restore that file on our server will that work? Or is it more complicated?

                 

                Thanks

                  • Re: Loading old Data
                    cescamilla

                    You could copy a full backup of the pi server and then recover that in the vCampus server.

                     

                     

                     

                    It shouldn't be too complicated, except that you won't be able to use the interfaces that the original PI Server has, so you will lack real time data.

                     

                     

                     

                    But it will work.

                      • Re: Loading old Data
                        MikeReid

                        What is the relative performance of PIConfig v SDK / API / ...?

                         

                        Does PIConfig apply any form of compression?

                         

                         

                         

                        I have had a few questions asked about doing this, but no one has come up with any data to be laoded yet...

                         

                        My initial thinking was to write some VBA / SDK code in Excel to do it, and apply a simple deadband filter to reduce the total volume of data written into PI.

                         

                         

                          • Re: Loading old Data

                            Mike Reid

                            What is the relative performance of PIConfig v SDK / API / ...?

                             

                            While it depends on a number of variables (type of calls you are using, type/precision of data you are writing to PI, network latency variability, etc.), I did a quick benchmark to give you an idea: writing 10,000 integer values (from 1 to 10,000) for tag Sinusoid, 1 minute apart.
                                -> With PIConfig: 35 seconds
                                -> With PI SDK: 4 seconds

                             

                            PIConfig is really meant to be the "low-effort, low-cost" solution for small, typically 1-timer, data backfilling operations.

                             

                            Mike Reid

                            Does PIConfig apply any form of compression?

                             

                            Yes, if you tell it do to so
                            Essentially, it depends on the table and the mode you use, and on whether the data you are sending to PI is the most recent data for this particular tag. The techsupport knowledge base article mentioned above shows how to do it with compression (using the PISnap table). You could also make use of the PIArc table to write events directly into the Archive, bypassing the PI Snapshot Subsystem (which handles Compression).

                             

                            Hope this helps!

                              • Re: Loading old Data
                                danielproulx

                                Hello all

                                 

                                Thanks for the PI config pointers. I am new to the PI system and have been given the task of loading instrument data from a client to a new system.

                                 

                                If I have a CSV file from a digital tag can Piconfig recreate the data properly in a PI archive if I rebuild their Digital States table in my system?

                                 

                                Currently the CSV file reads

                                 

                                tag, date+time, numeric value, text

                                 

                                Numeric value is 0 for all the digital tags and the text reflects the digital state

                                 

                                Is there a different method the client could use to export a digital tag in order for someone to load it properly?

                                 

                                Thanks for any help you can provide

                      • Re: Loading old Data
                        cescamilla

                        This could start a flame war!!!

                         

                        Anyway there are a lot of methods to input data into the PI Server, you could use PIDDE Interface, the good old (and almost forgoten) piconfig.exe, PI SMT, an Excel Spreed Sheet exporter, etc...

                         

                        Depending on how many data you have, and how can you actually sort it out your options will reduce and your choice will be easier.

                         

                        I would recomend piconfig.exe as it is really flexible and simple. I'll look for a link on how to importing data into a PI Server with piconfig.exe

                         

                         

                         

                        -- Edit --

                         

                        Steve got there first!