5 Replies Latest reply on Mar 23, 2015 5:15 PM by gachen

    Looking for easiest way to bulk load data into 2012 PI Server for demo

    fdeschenes

      Hi,

      I have a PI Server 2012 I'm using for a proof-of-concept as part of a larger EA solution and I want to create 300 or so tags and load them up with a few days of 1 second values. I can create the tags with the Excel plugin easily, but what is the best way to generate data in the archives for these tags?

      Thanks in advance.Frederick

        • Re: Looking for easiest way to bulk load data into 2012 PI Server for demo
          stang

          Hi Frederick,

           

          I'm assuming you are looking to populate your tags with random values. You can pretty much do this with any of the PI System access products, but I'll show you an example with the PISDK in a c# console app.

           

          The below works if you want to create events for every point on the server; you can also modify the GetPoints search string if you have an easily searchable mask.

           

                      PointList allTags = mySrv.GetPoints(@"tag = '*'");
                      DateTime endTime = DateTime.Now;
                      Random rnd = new Random();
                      foreach(PIPoint pt in allTags)
                      {
                          PIData dat = pt.Data;
                          PIValues vals = new PIValues();
                          vals.ReadOnly = false;
                          for (int i = 0; i < 1000; i++)
                          {
                              double val = rnd.NextDouble();
                              vals.Add(endTime.AddSeconds(-i),val,new NamedValues());
                          }
                          dat.UpdateValues(vals,DataMergeConstants.dmReplaceDuplicates);
                      }
          

           

          It creates a number of events one second apart going backwards from a specified end point.

          If you want to pass it a list of tags to create events for, you can try something like this:

           

                      var readFile = System.IO.File.ReadAllLines(@"C:\taglist.txt");
                      List<string> tagList = new List<string>(readFile);
                      DateTime endTime = DateTime.Now;
                      Random rnd = new Random();
                      foreach (string tagName in tagList)
                      {
                          PIPoint pt = mySrv.PIPoints[tagname];                        
                          PIData dat = pt.Data;
                          PIValues vals = new PIValues();
                          vals.ReadOnly = false;
                          for (int i = 0; i < 1000; i++)
                          {
                              double val = rnd.NextDouble();
                              vals.Add(endTime.AddSeconds(-i),val,new NamedValues());
                          }
                          dat.UpdateValues(vals,DataMergeConstants.dmReplaceDuplicates);
                      }
          

           

          Where taglist.txt contains all the tagnames separated by carriage returns.

          1 of 1 people found this helpful
          • Re: Looking for easiest way to bulk load data into 2012 PI Server for demo
            jose_dai

            Hi Frederick,

             

            Another option that you could you apply is with AF Analytics, you should create a AF Structure with your 300 tags using a template solutions and Analytics template. In my company we did something similar.

             

            You must create a template with the following attributes, Tag Value, Max Value and MIn Value( zero and Span), into the same template you must create the analytics template (Rand()*('Max Value'-'Min Value'))+'Min Value' and the Output Attribute is Tags value. You must include manually the values of Max Value and Min Value. Set the Scheduling about your requirements.

             

            You must do the same with Digital Value, but only you must create a template with one attribute Tag Value, and the analytics must be the following If(Rand() < 0.5) then 0 else 1, and the output is the same Tag Value. The digital state will depend of the tag that you  put into pi point data reference.

             

            Regards.

             

             

            • Re: Looking for easiest way to bulk load data into 2012 PI Server for demo
              dng

              There are indeed many ways to generate test data! Let me propose another option which you can consider. This option is useful if you would like to have random values for the 300 tags moving forward, but this will not help if you would like to backfill data.

               

              With every PI Data Archive installation, some interfaces are installed by default. The PI Random Interface is one of them. Default tags like "sinusoid" uses the random interface to generate sine waves. You can configure your 300 tags to be using the random interface (point source "R"), and set different location values to suit your needs for the data range and randomness (see the live library document here; user guide download here). Setting the scan class to 1 second will allow you to generate 1-second data. Once the points are set up properly, you will have random data for those tags moving forward.

              • Re: Looking for easiest way to bulk load data into 2012 PI Server for demo
                gachen

                If you want just purely random data, then Sam and Jose's suggestions are both good options. I like Daphne's suggestion because the Random interface is good for situations where you need simulated process data.

                 

                I just wanted to also point out that the Random interface has a little-known switch "/NoGap", which essentially lets you backfill data to these random points.

                 

                Example of steps:

                1. Stop the Random interface
                2. Create/configure your 300 tags
                  • Put the points on point source 'R'
                  • If you want values every 1 second, it's probably best to turn off exception and compression
                  • Configure the location codes according to what kind of random data you want
                  • Put the points on a 1-second scan class (make sure the scan class exists in the interface batch file)
                3. Add the "/NoGap" switch to the interface batch file (along with a 1-second scan class if necessary)
                4. "/NoGap" looks at the points' last snapshot values and times to determine the backfill start time. Since this is 2012, with tuning parameter "Snapshot_DoNotReplacePtCreatedOnOOO" disabled (default), you can just write another "Pt Created" value to all of these tags at a past timestamp (make sure archives exist back to this point). The new "Pt Created" event that you write should replace the existing "Pt Created" event.
                5. Start the Random interface, you should now see data being backfilled for your Random tags back to the new "Pt Created" event.