20 Replies Latest reply on Feb 25, 2016 1:37 PM by gregor

    DST Issue

    Rashmi

      Hi,

       

      Please can anyone advise on the below scenario with regards to DST issue?

       

      1. PI data gets created on SMT, which is on UK server

      2. My script (PowerShell) runs on US server to fetch the data from UK server (PI SMT tool)

      3. the time difference between US and UK is now 6hrs gap (US is ahead of UK)

      4. When I run the script today (19th feb), the file name gets printed as 18th Feb 000000 and the 2nd file name will be 19th feb 001000, 3rd file 19th feb 00300 and so on till 235000 timestamp (as per the code written inside powershell script)

      5. 18th feb 000000 file contains the attribute values of 11th feb 18PM data on UK server (since time gap is 6 hrs) and the next file value map to 610,6:20 and so on.(as per timestamp variable inside the sql query).

       

      6. Now the DST is going to change on 13th March 2016 for US server. The clock will forward 1 hr at 2 AM night, hence it will display as 3 AM.

       

      My doubts.

       

      I underrated that time difference will be 7 hrs after 13th March although, I have to handle 2 scenarios here on 13th March.

       

      1. SInce the file name is printing as back date 000000 ,001000,002000  etc. when the DST will change on 13th, I think from 12 AM to 2 AM, the file names willbe printed as normal. but what will be the file name after 2 AM. At what exact time stamp it will print after 2 AM. when the script will run on 14th March?

      2. after 2 AM, which timestamp data it will pick up from UK server or there will be no impact here since the DST on UK server is going to change on 27th March?

       

      Please advise.

       

      PS: $timeStamp = [DateTime]::Today.AddDays(-1)  -   this logic is for file name to generate

      $timestamp1 = [DateTime]::Today.AddDays(-1).AddHours(-6),   -  this varibale i pass inside the sql query to fetch the data from PI SMT (UK server)

        • Re: DST Issue
          Roger Palmen

          My favorite, timezones and DST!

           

          In general, any application using PI will perform translations of timestamps to the local timezone. In the scenario you describe, it seems you have a powershell script running in a server set to a US timezone, and you will retrieve data from a server in the UK timezone. The filename is also based on the UK timezone.

           

          Now adding hours in the script just adds hours, but makes life difficult as you don't know the current offset between the US server and the UK server. So you should look for a function that changes the US timestamps to UK timestamps.

          I don't understand the the scheduling of your scipt, so it's dificult to assess on what clock and using which rules you run the script. And in what format (and which UTCoffset) the filename is supposed to be.

           

          In general if you deal with multiple offsets between parts of your systems, some best practices i use: 1) base all your code on UTC timestamps. You can request PI data in UTC in almost any client. 2) ensure all timestamps formatted in strings either include the offset (read: ISO 8601 - Wikipedia, the free encyclopedia ), or the requirements cover all DST scenarios so you can use rules to cover the DST switches. NEVER build logic yourselves, it is too complex. Rely on specialised libraries.

           

          Hope this helps!

           

          PS: UK is ahead of US. Weekend already started here!

          4 of 4 people found this helpful
            • Re: DST Issue
              Lonnie Bowling

              Roger,

               

              I think I'm going to print your post and paste it onto the forehead of each my my engineers. In-fact, maybe a tattoo would be in-order, very well said!

               

              Lonnie

              • Re: DST Issue
                Rashmi

                Hi Roger,

                 

                Just to explain you with an example:

                 

                Suppose I run the Powershell script today (22nd Feb US date/time), the file name starts like 21st Feb 000000 (first file), 2nd file will be 21st feb 001000, 3rd will be 21st feb 002000 and o on. The data is on UK server gets generated in every 10 minutes. the file name 21st feb 000000 will have 20th Feb 6 PM data on UK server since there is 6 hrs time difference now. ON UK machine the data we are picking up from 6PM to next day 5:50 PM . So the file name will start from 20th feb 000000 to 21st feb 235000 PM. These file names are according to US time.

                 

                DST is going to change on 13th March at 2 AM (Clock will show 3 AM in US). There is no DST changes in UK on 13March.

                when I run the script on 13th March, it should pick up file from 11th March 6 PM to 12th march 5:50 PM from UK machine. And when I run on 14th, it should pick up 12th 6 PM to 13th 5:50PM ...Since DST in changing at 2 AM in US on 13th March, how the actual file names/data are generating from 2 to  3 AM? Does this mean in this time whatever files have to be generated with No Data (NaN)?  Yet again, on 13th March , 12 to 1:50 AM Many files would have generated already anyway in 10 minutes gap, but when DST changes at 2 AM and clock shows 3 AM , does this mean my script code should change accordingly such that when time reaches at 1:59AM , script should pick up data as per 7 hrs time? If so, then there might be a problem when DST changes in reverse order in Novemeber. That time the actual data will be present between 2 to 3 AM, but when clock reversed back from 3 to 2 AM,I'm not sure how to handle that time. My above approach is not suitable answer for the client to change the script each time .

                 

                Regards,Rashmi

                  • Re: DST Issue
                    gregor

                    Hello Rashmi,

                     

                    It can be quite a challenge to think about how to align timestamps among time zones an how to deal with Daylight Saving changes scheduled for different dates in different time zones. We do have some resources like KB00876 - Handling DST on PI Server, PI Interface, and PI Client nodes on the subject.

                    Likely the most simple approach is to always express timestamps in UTC as suggested by Roger. No matter what time zone a computer is in and if it participates in DST changes or not, it should at any time be possible to calculate from local time to UTC and vice versa.

                    1 of 1 people found this helpful
                      • Re: DST Issue
                        Rashmi

                        Did you and Roger advice the UTC timestamps has to be set on PI server itself ?

                          • Re: DST Issue
                            gregor

                            Hello Rashmi,

                             

                            No, the suggestion is to always "normalize" to UTC.

                              • Re: DST Issue
                                Rashmi

                                Hello Gregor,

                                 

                                I changed my script as :

                                $timestamp1 = [DateTime]::Today.AddDays(-1),

                                PS D:\Windpower\SSE\Scripts> $timestamp1

                                Sunday, February 21, 2016 12:00:00 AM

                                $dtu=$timestamp1.ToUniversalTime(),

                                PS D:\Windpower\SSE\Scripts> $dtu=$timestamp1.ToUniversalTime()

                                PS D:\Windpower\SSE\Scripts> $dtu

                                 

                                Sunday, February 21, 2016 6:00:00 AM

                                PS D:\Windpower\SSE\Scripts> $dtu.Kind

                                Utc

                                 

                                I ran my script, now file names are 21st feb 000000,001000 etc.) . When I checked the data on PI SMT (ON UK machine), 000000 AM data is matching with 6AM timestamp for 21st Feb.

                                 

                                In my normal script, the data for 12 Am on 21st feb was mapping to 20th 6PM data on UK server, which was correct .for DST to apply,  Is my above logic incorrect? I'm not sure whether data on PISMT is generated as per UTC timestamp. I'm given access to only PI SMT tool.

                                 

                                PS: On US server, the time zone says UTC 6 (US and Canada).

                                  • Re: DST Issue
                                    gregor

                                    Hello Rashmi,

                                     

                                     

                                    [DateTime]::Today.AddDays(-1) refers yesterday at midnight and is dependent on what time zone the machine is in it is generated this way. Since your server in the US is 6 hours behind the server in the UK, the meaning of the expression [DateTime]::Today.AddDays(-1) is different on both machines by the 6 hours time difference. When creating the UTC representation on each machine, you will also find the 6 hours difference.  

                                     

                                    I recognize that I don't have the full understanding of what you are trying to accomplish an how PI SMT fits into the scenario. Can you please elaborate a bit more detailed?

                                      • Re: DST Issue
                                        Rashmi

                                        Hello Gregor,

                                         

                                        Here is the detailed explanation.

                                        PI SMT generates data in every 10 minutes and it's present on the server UK

                                        I login to an US server and run my powershell script.

                                        I was asked to develop the script such that even though I run the script today, the file name will create a back dated name starting from 000000hrs and the files will be created in every 10 minutes with back date. Hence I had applied this logic.

                                         

                                        $timeStamp = [DateTime]::Today.AddDays(-1),

                                        $timestamp1 = [DateTime]::Today.AddDays(-1).AddHours(-6),

                                         

                                        The variable timestamp1 I call inside the sql queries so that it will pick up data from PI SMT since there 6 hrs time gap between US and UK. Hence, with this logic my script is running successfully and so far so good.

                                        But now I have been asked to think for DST scenario for March 13th.

                                          • Re: DST Issue
                                            gregor

                                            Hello Rashmi,

                                             

                                            PI SMT refers PI System Management Tools.

                                            My understanding is that

                                            • Your PI System is installed on a host in the UK with time zone settings set up accordingly.
                                            • You are using PowerShell and SQL access to the PI System to extract data and store them in files.
                                            • The machine that you are using for data extraction is located in the US (UTC-6).
                                            • Scripts are running each 10 minutes
                                            • Filenames are backdated by 24 hours. The filename codes the data query start time.
                                            • You are now concerned about what will happen with the DST in the US scheduled for March 13th.

                                            When forming the filename, you are using the 6 hours UTC offset ($timestamp1 = [DateTime]::Today.AddDays(-1).AddHours(-6)). Starting March 13th 2:00 AM clocks will change by 1 hour to 3:00 AM resulting into a new UTC offset of 5 hours. To get the UTC offset at any time, you can use the following PowerShell snippet:

                                             

                                            [System.TimeZoneInfo]::Local.BaseUTCOffset.Hours
                                            
                                              • Re: DST Issue
                                                Rashmi

                                                Hello Gregor and Roger,

                                                 

                                                Many thanks for all your help.

                                                 

                                                I added the additional varibale as advised.

                                                $timestamp1 = [DateTime]::Today.AddDays(-1).AddHours(-6),

                                                $timestamp2 =[System.TimeZoneInfo]::Local.BaseUTCOffset.Hours,

                                                 

                                                So when DST changes, $timetamp2 should print me -5.

                                                 

                                                I ran my script and PI SMT readings are appearing with -6 hrs difference now.

                                                 

                                                Many thanks again!

                                                1 of 1 people found this helpful
                                                • Re: DST Issue
                                                  Rashmi

                                                  $DST_UTC =[System.TimeZoneInfo]::Local.BaseUTCOffset.Hours,

                                                   

                                                  $timestamp1 = [DateTime]::Today.AddDays(-1).AddHours($DST_UTC),

                                                   

                                                  I amended the 2 lines of code  by calling first variable

                                                    • Re: DST Issue
                                                      Roger Palmen

                                                      Good work!

                                                      Using the timezone logic of .NET is a sure way to go.

                                                        • Re: DST Issue
                                                          Rashmi

                                                          Hello Roger and Gregor,

                                                           

                                                          The above command will show the time difference between Local time and UTC time and that will surely work from 13th March. However, the server is in UK (PI SMT ). the DST gets changed on 27th March in UK. I think UK will not follow UTC at that time. It would be 1 hr away from GMT ..

                                                           

                                                          Do we have any powershell command where we get the time difference between UTC and UK? I did nt find any answer in google....and still searching....

                                                            • Re: DST Issue
                                                              gregor

                                                              Hello Rashmi,

                                                               

                                                              I believe that you are thinking way too complicated. Your approach uses UTC time and should work on all machines, no matter in what time zone they are . It should even work with DST change being disabled.

                                                              Besides that I don't think that there's build in mechanism to calculate a "foreign" DST offset but it is probably possible to get that information from a machine that is in UK time zone.

                                                              1 of 1 people found this helpful
                                                                • Re: DST Issue
                                                                  Rashmi

                                                                  Gregor,

                                                                   

                                                                  I'm sorry if I'm asking too many questions.

                                                                   

                                                                  Somewhere my mind says if the time of tags (PI SMT) is in UK local time it will not work - if it is in UTC it will work.  Timestamp shows UTC+1 for UK. Please could you advise if I'm wrong?

                                                                   

                                                                  UTC.jpg

                                                                    • Re: DST Issue
                                                                      Rashmi

                                                                      I think I got your point what you were trying to explain me.

                                                                       

                                                                      I'm just imagining the graph where UTC and UK is at 0 and US is at -6 at left side. after 12th March when DST changes in USA, time gap will become -5 hrs. Again, after 27th for UK DST, UTC will move from 0 to 1 and the total time difference will show -6 again. Hence, I dont have to change anything in my power shell script and your earlier advice about UTCoffset code will always work throughout the year. Please correct me if my imagination is right.

                                                                        • Re: DST Issue
                                                                          gregor

                                                                          Hello Rashmi,

                                                                           

                                                                          I don't feel like I have yet the complete picture of what you are trying to accomplish. Therefore please don't expect me to guarantee it will be working.

                                                                          The idea however is to normalize all timestamps to UTC before storing them. This should always work with local timestamps because the machine / Windows has its offset to UTC at any time.

                                                                          When reading UTC normalized timestamps you should always be able to either use them as they are, or to convert them to local time.

                                                                          The PI Data Archive internally stores Timestamps in UTC, independent from where the PI Data Archive node is installed. Clients usually have the option to either request timestamps as Server or as local (client) time.

                                                                          Dealing with timestamps across time zones under consideration of all DST change rules isn't trivial and we are facing misunderstandings and issues because of this. In many cases because people make too many adjustments.

                                                                          1 of 1 people found this helpful
                                                                        • Re: DST Issue
                                                                          Roger Palmen

                                                                          Maybe this reading will clarify how .NET DateTime works: Choosing Between DateTime, DateTimeOffset, TimeSpan, and TimeZoneInfo

                                                                          If you look at the constructor for AFtime (AFTime Constructor (DateTime) it shows that this is treated as UTC if the "kind" (DateTime.Kind Property (System) ) is not supplied. So using that knowledge you should be in control of what timestamp (UTC or offset) you send to PI, and this is unambiguous to PI.

                                                                           

                                                                           

                                                                          Dealing with DST and timezones can  be very confusing and requires a lot of attention to get right and not make mistakes, or interpret something good as a mistake.

                                                                          1 of 1 people found this helpful