4 Replies Latest reply on May 18, 2015 7:32 PM by voo33

    Massive data dump from PI to Excel

    voo33

      Hi PI genius,

       

      We're working on a report, which requires us to provide 10 minute interval data for a few AF attributes on monthly basis. In each monthly report, we have to cover all the months since the beginning of the term year. Beyond the 10 minute data, we have to also provide the calculated results in the same Excel file where the raw data lives, and the customer will verify our calculation results based on the raw data provided.

       

      Since we're asked to provide the report automatically every month, we're thinking to use SSRS just as our other projects. I'd imagine at some point during the term year, the data volume will be a huge issue for OLEDB enterprise and SSRS.  Has anyone ever run into similar scenarios? Is there any suggestion what is the most efficient approach to provide both raw and calculated data?

       

      Thanks,

      Shanshan

        • Re: Massive data dump from PI to Excel
          Lonnie Bowling

          Hi Shanshan,

           

          A quick question to clarify. For the first month of the term you have to dump 10 minute intervals which is considered raw data. The customer then looks at your calculated values (from AF?) and validates your calculations using the raw data. The second month both the first and second month's data (in 10 minute intervals) is pulled in and the same process is repeated. My question is why is the first month validated a second time? Is there year to date calculations? If so are all the calculations year to date?

           

          Guess that was actually more than one question

           

          Lonnie

            • Re: Massive data dump from PI to Excel
              voo33

              Hi Lonnie,

               

              Unfortunately this data validation process is specifically required by our customer, and in reality, the data validation is 'not' completed until the end of term year, when the Excel is filled up with 12 months raw and calculated data. Beyond that, they also want to see the 'change of the data' in the report. Those're the reasons why the data is repeatedly showing in the report, and it's required to provide such report on a regular basis.

               

              Thanks,

              Shanshan

                • Re: Massive data dump from PI to Excel
                  Lonnie Bowling

                  I was just thinking that maybe there is a way that you don't have to dump the same data over and over, just pull the monthly data in once and add to the spreadsheet, you could have a worksheet for each month. Maybe update the previous month if data was changed or added. This could be done in the spreadsheet with scripting or a scheduled C# service that could automate the process if there are several customers that required this.

                   

                  Lonnie

                    • Re: Massive data dump from PI to Excel
                      voo33

                      Hi Lonnie,

                       

                      1. Although it’s one particular customer requesting it, I think we should try your way, which is easier and more efficient than dumping the same data over and over again. We’ll discuss further and maybe I’ll ask for your advice again if we run into any issue. J

                       

                      Thanks soooo much for the idea!

                       

                      Regards,

                      Shanshan