7 Replies Latest reply on Mar 22, 2017 4:38 PM by gregor

    Programmatically redirecting output of pigetmsg -f

    John Messinger

      So I have a requirement to programmatically capture PI message log information in near real time on a continuous basis. As there doesn't seem to be any SDK functionality to access the server (or local) message log (other than the PI-SDK, and I don't really want to go down that path), I've been mucking around with trying to redirect the output of pigetmsg, running in follow mode (-f switch). I'm doing this from some C# code, snippet as follows:

       

      static void Main(string[] args)
      {
        string sourceServer = "demo-pi";
        string authenticationMethod = "Windows";
        var commandArgs = new StringBuilder();
        commandArgs.AppendFormat("-node {0}", sourceServer);
        commandArgs.AppendFormat(" -{0}", authenticationMethod);
        commandArgs.Append(" -f");
        var pigetmsg = new Process();
        var startInfo = new ProcessStartInfo();
        startInfo.WindowStyle = ProcessWindowStyle.Hidden;
        startInfo.CreateNoWindow = true;
        string cmdDirectory = Environment.GetEnvironmentVariable("PIHOME") + "\\adm\\";
        startInfo.FileName = string.Format("\"{0}pigetmsg.exe\"", cmdDirectory);
        startInfo.Arguments = commandArgs.ToString();
        startInfo.RedirectStandardOutput = true;
        startInfo.UseShellExecute = false;
        pigetmsg.StartInfo = startInfo;
        pigetmsg.OutputDataReceived += new DataReceivedEventHandler(MessageLogOutputHandler);
        Console.WriteLine("Commencing process {0} {1}", startInfo.FileName, startInfo.Arguments);
        pigetmsg.Start();
        pigetmsg.BeginOutputReadLine();
        pigetmsg.WaitForExit();
      }
      
      private static void MessageLogOutputHandler(object sendingProcess, DataReceivedEventArgs outLine)
      {
        if (!string.IsNullOrEmpty(outLine.Data))
        {
        Console.WriteLine(outLine.Data);
        }
      }
      

       

      What I've found though is that the output from pigetmsg doesn't actually seem to get captured as redirection of standard output. If I substitute the startInfoArguments as per the following snippet, the code works exactly as expected, in that the output of the pigetmsg command is redirected and I can capture it and output it into my console app window:

       

      var commandArgs = new StringBuilder();
      commandArgs.AppendFormat("-node {0}", sourceServer);
      commandArgs.AppendFormat(" -{0}", authenticationMethod);
      commandArgs.Append(" -st *-10m -et *");
      

       

      So my question here is, does pigetmsg -f not write to stdout? After scratching my head for a couple of days (and reading many SO posts), I'm starting to wonder if it's possible to even capture the output of pigetmsg this way when running with -f.

       

      Appreciate any thoughts or suggestions from the community (and any OSI devs, if you're watching).

       

      Cheers,

      John

        • Re: Programmatically redirecting output of pigetmsg -f
          gachen

          So when you run with the '-f' switch, nothing displays in the console (apart from the "commencing" message)?

           

          Your original snippet with the '-f' switch seems to work fine for me (VS 2015/pigetmsg 3.4.405.1161). What is the version of pigetmsg you are running, and which version of Visual Studio are you using/what version .NET is the solution targeting?

            • Re: Programmatically redirecting output of pigetmsg -f
              John Messinger

              Hi Gavin,

               

              Yes, that's correct - nothing apart from the "commencing" message.

               

              I'm running pigetmsg v3.4.405.1161, with Visual Studio 2015, targetting .NET 4.6.2.

              As an update, running the code again, and leaving it for much longer, I am now seeing message log data come through, however it appears to be buffered for a period of time, and then written to my application console in one large chunk (ie, several minutes worth of messages), rather than as messages arrive (which is what I would expect, as this is how pigetmsg seems to usually behave).

               

              I also noticed on the server that my remote pigetmsg session has been disconnected, with the following message recorded:

              Deleting connection:  pigetmsg(9584):remote(9584), Asynch send failed. [121] The semaphore timeout period has expired.

               

              My code is connecting to PI running on a VM which is using bridged networking to the same network as my development machine, but I will look into possible networking issues all the same.

               

              Out of curiosity, when you ran my code snippet, was data output almost immediately, and in 'streaming' fashion as is usual for pigetmsg, or was there a significant delay before any data was received?

               

              Cheers,

              John

                • Re: Programmatically redirecting output of pigetmsg -f
                  John Messinger

                  So I'm going to abandon the idea of (ab)using pigetmsg this way, as I just seem to have these weird buffering issues with the redirected output. Will probably go down the path of writing a PowerShell script to call Get-PIMessage cmdlet.

                    • Re: Programmatically redirecting output of pigetmsg -f
                      gregor

                      Hi John,

                       

                      Please accept my apologies but I saw your question not before the latest update. Can you please explain more detailed what you like to do?

                      I am not sure if you are aware that since quite some time, pigetmsg.exe supports remote connections which makes it even more handy. Please see the following command as an example to connect against a remote PI Data Archive (MY.PI.DATA.net) using Windows Integrated Security for authentication and redirecting the output of the continuous reading to a text file:

                       

                      pigetmsg -node MY.PI.DATA.net -Windows -f > c:\Temp\MY.PI.DATA.net.txt
                      
                        • Re: Programmatically redirecting output of pigetmsg -f
                          John Messinger

                          Hi Gregor,

                           

                          No worries. The basic concept behind what I was trying to do was continuously read pigetmsg output that I could redirect to another application for parsing and analysis. Yes, I'm aware that pigetmsg supports remote connections using Windows security (you'll see that's exactly what I was doing in my code snippet).

                           

                          My experience with redirecting this output to a text file as you mentioned was that no data was ever written to the file until the process was terminated - seems like it was buffered in memory and not flushed to disk until the process ended. Likewise, when I attempted to redirect the output programmatically in some C# code, I found that there was a significant delay (in the order of several minutes) until I saw any redirected output appear in my app console, even though messages were being written to the log during that whole time (the delay wasn't due to simply no messages being logged). Further, the pigetmsg process would just stop sending any more data after that initial dump of delayed messages. Only seemed to happen in the context of my application code - I don't see this behaviour when running pigetmsg with a remote connection directly from a normal command window.

                           

                          If I can find a way to make this work, I might continue pursuing this option, but I'm consistently seeing this behaviour, so my feeling is that maybe I should use PowerShell, as we can execute the script in a loop to just read the past n seconds or minutes of log messages.

                        • Re: Programmatically redirecting output of pigetmsg -f
                          gregor

                          Hi John,

                           

                          Please accept my apologies. When looking at your code more detailed, I recognized that you've been aware of the option to use pigetmsg.exe remotely. However, I am still interested to learn more about your use case.

                           

                          I've done some testing meanwhile and it appears that process events are maintained pretty lazy. The code that I've used is similar to yours:

                           

                                  static void Main(string[] args)
                                  {
                                      string piDataArchive = "BECKPI2015R2";
                                      string piHome = Environment.GetEnvironmentVariable("PIHOME");
                                      var pigetmsg = new Process
                                      {
                                          StartInfo = new ProcessStartInfo
                                          {
                                              FileName = piHome + @"\adm\pigetmsg.exe",
                                              Arguments = "-f -node " + piDataArchive + " -Windows",
                                              UseShellExecute = false,
                                              RedirectStandardOutput = true,
                                              CreateNoWindow = true
                                          }
                                      };
                                      pigetmsg.Start();
                                      while (!pigetmsg.StandardOutput.EndOfStream)
                                      {
                                          string line = pigetmsg.StandardOutput.ReadLine();
                                          Console.WriteLine(line);
                                      }
                                      Console.ReadKey();
                                  }
                          

                           

                          After running this for a few minutes, the console was still empty and I thought it could help sending some "spam". Because of a missing equivalent in AF SDK, I've used PI SDK to write against the PI Message Log on the PI Data Archive:

                           

                                  static void Main(string[] args)
                                  {
                                      PISDK.PISDK piSDK = new PISDK.PISDK();
                                      PISDK.Server piSrv = piSDK.Servers["BECKPI2015R2"];
                                      piSrv.Open();
                                      for (int i = 1; i < 1000; i++)
                                      {
                                          string sMsg = string.Format("This is message # {0} and there might be more .. ", i);
                                          piSrv.MessageLog.PutString(sMsg);
                                          Thread.Sleep(100);
                                      }
                                      piSrv.MessageLog.PutString("This was it. We will now quit .. c u ");
                                      Thread.Sleep(1000);
                                      piSrv.Close();
                                  }
                          

                           

                          I saw now the console of the PI Message Log "observer" updating with each 20~25 messages. My conclusion is that Windows is somehow maintaining the process and does not signal the "news" unless there's enough activity to do that frequently. I saw similar behavior with the redirected pigetmsg.exe command posted above before. The referenced log file remained empty until I interrupted the process. I conclude that both tested options are likely not useful for you. What may work however, is using the Process class  to query for news every 2 seconds because this is what pigetmsg -f is doing.