10 Replies Latest reply on Feb 21, 2017 2:43 PM by jdryden

    Bandwith Consumed By DataLink

    Fuad

      Hi My Team has raise a concern on the Loading time of PI Datalink. Since they are located offshore, it makes sense that the time taken longer due to the slow connection.

      Is there a way for me to observe the bandwidth needed by Datalink to load the data? So i can present to the IT team, to justify the need to improve the network.

       

      i have understand OSIsoft has release below sheet, but thats for the Data writing to the Server. Can i use it to calculate the Bandwidth needed for Datalink?

       

      pastedImage_0.png

        • Re: Bandwith Consumed By DataLink
          Roger Palmen

          I think the sheet works ok for interfaces, as it includes the overhead introduced by the PI-API used by interfaces. Datalink uses the PI-SDK and would therefore have a different overhead. In other words, this will not give an accurate picture.

           

          When you mention "the loading time of datalink", is that the time to load the plugin, or the time to load one a specific dataset from the PI server? The latter is very dependant on your specific situation.

            • Re: Bandwith Consumed By DataLink
              Fuad

              I would like to know the bandwidth need to load a specific dataset from PI Server.. I am calling data for around 450 points. As run the Datalink from my office, it would only take around 2-3 minutes, but at Offshore, it took around 25minutes..

                • Re: Bandwith Consumed By DataLink
                  Roger Palmen

                  One of the things you can look at is the type of calls you make. When you say offshore is slower, i assume that the connection has a high latency. This making one large call to PI would improve performance over multiple separate calls. One call roughly equates to one function in Datalink.

                  It is quite difficult to improve this, as this would mean tuning to your specific situation. That would involve a lot of measuring, testing, etc.

                  • Re: Bandwith Consumed By DataLink
                    gregor

                    Hello Fuad,

                     

                    Since a few versions (I believe starting with version 5.0), PI DataLink is using AF SDK under the hood. This is also indicated by the fact that you can refer Attributes within your PI AF structure alternatively to just PI Points.

                     

                    What version of PI DataLink are you using?

                     

                    Your question for the data volume PI DataLink requires is indeed an interesting one and of special interest when operating with limited network bandwidth. I have reached out to the product specialists within OSIsoft Technical Support organization and hope they can suggest a general approach to either measure or calculate the data volume used by PI DataLink.

                    1 of 1 people found this helpful
                    • Re: Bandwith Consumed By DataLink
                      jdryden

                      Hi Fuad,

                       

                      I've performed a quick empirical test of retrieving large quantities of data using PI Datalink 2016. In this test, I retrieved compressed values for Float32 and Int32 tags, 6 tags at a time. I then measured the volume of data that was received over the network by Excel while retrieving these events.

                       

                      As is expected, the volume of data varies very linearly with the number of events retrieved:

                       

                      For 32-bit values (often the most common types), each event requires 17.6 Bytes transferred. Let's call it 18 Bytes for estimation.

                       

                      We can use this number of bytes/event to perform some back-of-the-hand estimates of data that will be retrieved by PI Datalink. If you have 450 tags and are retrieving about 1000 events per tag (on average), this will send 8.1 MB over the network. If you are retrieving 100,000 events per tag, this number jumps to 801 MB (that's a lot of data in a spreadsheet).

                       

                      To be clear, these estimates are for data volume. I am guessing that you are more concerned about the time it takes to load the Datalink values. To estimate the time it will take to send the values, we can divide the data volume by the network bandwidth, keeping in mind that 1 MB/s = 8 Mbps. So if you have on average a 1 Mbps connection and are retrieving 1000 events per tag, you can expect your transfer time to be:

                       

                           8.1 MB / ( 1 Mbps * (1 MB / 8 Mb) ) = 64.8 seconds

                       

                      If the number you determine from this estimate is unacceptably high, that may be sufficient evidence for your IT team.

                       

                      Disclaimer: This quick analysis only covered the case of large volumes of compressed data, not other types of calls (or many smaller calls). It also does not account for issues like latency, which (as Roger pointed out) can make many small calls take longer than one larger call.

                       

                      Let me know if you have any questions about the above!

                       

                      -James

                      5 of 5 people found this helpful
                        • Re: Bandwith Consumed By DataLink
                          gregor

                          Thank you James,

                           

                          Very insightful!!

                          • Re: Bandwith Consumed By DataLink
                            Fuad

                            As per mentioned by Roger, i notice that it take longer time, making a large call. As it is retrieving the value for next cell, its also refresh the Datalink formula on previous cell, which already been populated. So it keep on refreshing and it takes a longer time.

                             

                            to overcome this, i created a loop in the vba to retrieve the value. After each value retrieved, it will convert to value, and remove the formula.

                            ''

                            For l = 3 To 25

                                  For i = 1 To 51

                                  Sheet2.Range("AA11").Value = Sheet2.Cells(8, 3).Value & wellsheet.Cells(i + 12, 4).Value & Sheet2.Cells(9, 3).Value & _

                                        wellsheet.Cells(12 + i, 5).Value & "\Well Test" & Sheet2.Cells(14, l + 25).Value & ".DOR"

                                     

                                        Sheet2.Range("I7").Formula = "=PIArcVal( AA11,Z" & i + 3 & ", 0,Sheet2!$C$7,""auto"")"

                                        If Sheet2.Range("I7").Value <> "Pt Created" Then

                                        wellsheet.Cells(i + 12, l).Value = Sheet2.Range("I7").Value

                                        End If

                                  Next i

                              Next l

                             

                            "

                            Sheet2.Range("AA11").Value holds the tagname (its actually an AF attribute) . It combine few static string ( such as server) , platform, and the table headers ( attribute name).

                             

                            This manage to cut the loading time a lot. But it still consuming at least 5 minutes.

                             

                            Back to James explanation, with my way above. Since im calling the event one by one ( one tag on event), would it be safe to say the bandwidth consume only 18 Bytes at one time? So the way to calculate it should be as below?

                             

                            18Bytes * number of tags / ( 1 Mbps * (1 MB / 8 Mb) )? Assuming im on average a 1 Mbps connection

                              • Re: Bandwith Consumed By DataLink
                                jdryden

                                Hi Fuad,

                                 

                                Apologies for the slow reply. Am I understanding correctly that you are using a PIArcVal() function to individually retrieve the value for each cell one-by-one? If this is the case, then my bandwidth testing does not apply very well to your situation. My test focused on bulk data retrieval for a small set of tags. Your application seems to involve retrieving a small number of values for a larger number of tags.

                                 

                                Making individual calls to PIArcVal() will result in significantly more data volume per event than using Array functions and bulk data calls. In addition, since each cell will require a new call to the PI Data Archive, a high latency environment (like an off-shore connection) will be even slower. I am not sure how your report is being formatted and if this is feasible, but you may wish to reformat the display to make use of these bulk calls to improve performance.


                                Let me know if you have any further questions about this topic.

                                 

                                 

                                -James

                                2 of 2 people found this helpful
                                  • Re: Bandwith Consumed By DataLink
                                    Fuad

                                    Hi James,

                                     

                                    I did try making a bulk call, but what happen is the call keep making the Excel files Not Responding, and at the end of the day, the Excel will crash.

                                     

                                    Plus i can utilize the AF element by calling one by one. Example as below :

                                    2017-02-16 10_32_25-Utility.xlsm - Excel.png

                                     

                                    The tagname for PI Arcvalue will be "tag string =E8+E9+C17+D16+D17+E16 ( \\VCENPCSBPIAPP02\DOR\DOR\REGION\PLATFORM\DP-A\\Utilities\Utility Sotck\Diesel|Items"

                                     

                                    i have created a loop for it to go every row and table headers. Its easier so that i dont have to assign tag for every cells.

                                      • Re: Bandwith Consumed By DataLink
                                        jdryden

                                        Hi Fuad,

                                         

                                        Thanks for the details on the design of your spreadsheet. I think I understand why you are using the macro to run the Datalink functions, as it makes the report more dynamic and easier to generate.

                                         

                                        In this situation however, it is difficult to estimate how much bandwidth will be required since there is increased overhead in making individual calls to Datalink functions. I would recommend that you perform further testing on data usage using your spreadsheet if you would like a more accurate estimate.

                                         

                                        -James

                                        1 of 1 people found this helpful