Skip navigation
All Places > PI Developers Club > Blog > 2010 > March

StreamInsight Samples

Posted by egove Employee Mar 30, 2010

We are adding three more StreamInsight samples to compliment our Adapter CTP (Community Technology Preview).  If you are interested, you should begin with Spike Counter.  This is  a sample that explores some of the simple LINQ queries and a potential methodology to use to develop CEP solutions.  The PE Samples is very immature and will be added to as time permits.  It replicates a number of the samples provided with the PE documentation.  The final sample shows the use of User-defined Aggregates and User-defined Operators.


All of the samples provided contain detailed comments to explain the various sections of code.  The samples augment Microsoft’s StreamInsight information.  The Microsoft help and discussions provide greater detail than our samples.


For a quick introduction to or review of the PI StreamInsight Adapters, check the StreamInsight webinar, recorded on Feb.25th, 2010.


You can download these samples here (they are also available on the vCampus Download Center, under the "Supporting Files" category).

The PI Notifications Developers Kit includes a custom delivery channel sample that includes a delivery channel and an endpoint editor control.  Some may also want to customize the subscriber editor.  We've uploaded an addendum to get you started here:


DownloadCenter -> Extras -> PI Notifications XML Delivery Channel Code Samples


In addition to adding that control to your custom delivery endpoint project, you'll need to add an override in your custom delivery channel specifiying which control to use.  For the XML sample, this will suffice:


     public override Type ContentEditorType
               return typeof(XMLDCContentEditor);

Since presentation at RSA last year, I've been a fan of Fortify's "Building Security in Maturity Model" (BSIMM). As you might recall the model was based on security development practices observed at leading companies.  Categories and maturity rankings were assigned so BSIMM is especially interesting as a benchmark for SDL practices.


An update was presented at RSA this year. The BSIMM empirical data set now includes over a 100 companies.  Analysis with Fortify has lead to a simplified security development lifecycle document (17 pages) recently released by Microsoft. Below are my perspectives on the ‘optimized' process.


To cut to the chase, the simplified SDL is useful but comes up short. Much of the simplification appears to come from favoring practices at the front end of the process. Addressing security early and often is important and arguably the most efficient approach. However, security in manufacturing sectors is dominated by operational aspects and extended support lifecycles.


It was of little surprise the first few sections are ‘disclaimer-ish'. The introduction suggests the simplified process as a minimum core set of practices. Furthermore, the simplified approach aims lower; targeting a progression from basic to standardized to advanced stages.  The full SDL is needed to target a more dynamic security level characterized by actively minimizing risk for customers. Dynamic security certainly aligns with building resiliency into highly serviceable PI Systems. We like to think of the potential for managed PI to deliver state of the art incident response and prevention service level objectives.


Ok, so not all SDL initiatives are created equal and there is the significant issue of bootstrapping the process. Consider this quote from the simplified SDL: "Integration of secure development concepts into an existing development process can be intimidating and costly if done improperly."  Based on my experience with the BSIMM benchmark criteria, the simplified process is indeed optimized to help get started and can be useful to accelerate existing SDL initiatives.  For instance it's easy to spot voids in the 5 process phases then consult the full SDL or examples from the BSIMM report for details on the missing activities.  Microsoft offers many useful SDL resources.


To get started I suggest using the sample implementation flow chart (attached) in an honest self assessment. How many short-cuts are you taking? Is pen testing a security crutch for your products?  How do you go about security training and establishing security experts?


Perhaps true for developers everywhere, attack surface reduction is an especially challenging security practice at OSIsoft. For instance, when to move a project to a more secure compiler that means dropping support for a legacy platform? What platform features should we leverage and which ones should we disable? How should we react to externalities like the recent announcement about Windows Help? What about PI itself... should we really start all services? If not, which ones should be disabled by default? How easy is it to use the system without super user privileges?


Essentially if we get the surface area wrong, users will not get the expected value from the PI system.  We must build-in security with maturity to protect your data and critical infrastructure but not get in the way of a rewarding experience for at least 80% of users. Indeed the 80/20 rule is part of the SDL, but try to measure that one!


In summary, a simplified security development lifecycle is not the whole SDL story. Over the long haul, I believe a dynamic level of security can only be achieved with close collaboration on operational aspects.  OSIsoft vCampus provides a unique community forum to see many sides of our shared challenges including security. At a minimum your ideas help shape surface area decisions and guide us toward getting security done right!  Thank you in advance for using the SDL process.


Announcing PI OLEDB Enterprise

Posted by jlakumb Mar 18, 2010

As my good friend Ken so astutely observed, a supported scenario with PI OLEDB 4.0 Beta 2 is for use in PI WebParts. This allows folks to bring more AF data into PI WebParts, than it currently supports through its native AF data access. For example, PI OLEDB supports more AF data references such as Formulas and Tables, as well as UOMs and other AF functionality. If you have been waiting for thin client visualization of AF, here is your chance to leverage the powerful combination of PI WebParts with PI OLEDB.


Question: Are there other PI Clients that can take advantage of this technique? Answer coming in a moment...


For those of you who watch the Engineering Plan like hawks (you know who you are), you may have noticed a name change for PI OLEDB 4.0 to PI OLEDB Enterprise (aka PI System OLEDB). This change implies that PI OLEDB Enterprise is designed for use with AF (as well as some PI data), and is a separate product from the existing PI OLEDB 3.x Provider. In addition, PI OLEDB Enterprise will be licensed under the new PI System Access (PSA) license, which was announced at vCampus Live (this was also renamed from PAL to PSA). Unfortunately this reverses my earlier statement on licensing; however, you should find that the PSA provides much, much more. Since this is not a forum to discuss sales issues, I encourage you to watch the PI Data Access product page for the Datasheet and PSA FAQ document, then contact your Account Manager for any follow up questions.


Answer: Given the ubiquity of OLEDB integration in Microsoft tools, it is also possible to bring AF data into Excel/Excel Services using PI OLEDB Enterprise. The External Data feature of Excel makes it relatively easy to write SQL queries and bring in tables of AF data, where this can be combined with DataLink/DataLink for Excel Services to create reports, charts, and dashboards with PI/AF data.


Bonus: Any idea how to do this with ProcessBook? (Hint: use ODBC datasets). If you figure this out, please test it out and post your results in the Comments section.


Now for some March Madness - Go Tennessee Vols and St. Mary's Gaels!


UPDATE 5/25/10: Full credit goes to Matt Ziegler (DataLink PM) for putting together a nice video ( Jay Lakumb) showing how to use PI OLEDB Enterprise in Excel 2010 and PowerPivot. Note, this approach of accessing external data works in previous versions of Excel as well.

Hello, This is your PI System calling to inform you…

Wouldn’t it be cool to get a call from PI Notifications?


Phone calls convey a much higher sense of urgency, and you don’t have to worry about it they were read in time or not! Modern IP telephony provides endless possibilities to relate information in a more timely manner.
PI Notifications comes armed with the perfect delivery channel: web services! Around here, we use Skype(tm) as a convenient and cost effective communication tool, but it could be other tools as well. As an example, let us go ahead and write a little web service that takes a message from PI Notifications and delivers it via Skype.

  1. The first thing to do is create a web service project, so open Microsoft Visual Studio 2008, select File>New>Project>Visual C#>ASP.NET Web Service Application. I decided to call mine MySkypeWebService and I have chosen to write it in C#.
  2. VisualStudio.jpg

  3. Now, we want to call a contact, so we have to create a WAV file and deliver this via Skype. That means we have to add some references to our project:
    1. A .NET reference to PresentationCore and to WindowsBase (MediaPlayer requires this and MediaPlayer is an easy way to get the length of the WAV file).
    2. A .NET reference to System.Speech.Synthesis to make our WAV file.
    3. A COM reference to SKYPE4COMLib for Skype.
      (you first need to download and install the Skype4COM API Wrapper from the Skype Developer Downloads page).

  4. To improve readability of the code, add some “using” statements for the System.Windows.Media, the System.Speech.Synthesis and the System.Speech.AudioFormat namespaces.

  5. Note 1: Making a WAV file and placing a call takes time. For file I/O and other time consuming operations it is recommended to create asynchronous web services. There are various resources available describing how to create an asynchronous Web Service and provide all the necessary information.

  6. So what about the code? Here is how we start with the declaration of the web service:

    using System;


    using System.Collections.Generic;


    using System.Linq;


    using System.Web;


    using System.Web.Services;




    // Add .NET Reference to PresentationCore for Media Player


    // Add .NET Reference to WindowsBase for Media Player


    // Add COM Reference to SKYPE4COMLib for Skype


    // Add .NET Reference to System.Speech for the speech engine


    using System.Windows.Media;


    using System.Speech.Synthesis;


    using System.Speech.AudioFormat;




    namespace MySkypeWebService




        /// <summary>


        /// Summary description for Service1


        /// </summary>


        [WebService(Namespace = "")]


        [WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]




        // To allow this Web Service to be called from script,


        // using ASP.NET AJAX, uncomment the following line.


        // [System.Web.Script.Services.ScriptService]


        publicclassService1 : System.Web.Services.WebService







  7. Now let us start with the WAV File. The following code consists of a method that will create a WAV file in the appropriate format for Skype to understand. As parameter I have chosen the text and the filename:

    // Creates the WAV from Text


    privatevoid Text2WAV(string strOutput, string strFilename)








            // The SpeechSynthesizer


            SpeechSynthesizer _synth


                = newSpeechSynthesizer();


            DateTime _time = DateTime.Now;


            // Set the format to something SKYPE likes


            // 16KHz, 16bit, mono


            SpeechAudioFormatInfo _format


                = newSpeechAudioFormatInfo(16000,






            _synth.SetOutputToWaveFile(strFilename, _format);


            // Slow down a bit, the voice is too fast for me


            _synth.Rate = -2;


            // Speak!








        catch (Exception _ex)




            System.IO.StreamWriter _StreamWriter


                = new System.IO.StreamWriter










            DateTime _time = DateTime.Now;




            _StreamWriter.WriteLine("<!-- " + _time.ToLocalTime().ToString() + " -->");



















  8. Now that we have the WAV file, we will create another method that performs the Skype call. For this we need to know the filename and the Skype contact:

    privatevoid MakeSkypeCall(string strFilename, string strContact)




        // Need to allow ASP.NET to "run as" system


        // and enable IIS to interact with Desktop






            // A workaround to get the length of the audio file


            // is to use MediaPlayer


            MediaPlayer _mplayer = newMediaPlayer();


            Uri _path = newUri(strFilename);




            while (_mplayer.DownloadProgress < 1.0)




                // Need to wait until everything is settled,


                // before to ask for the duration of the file








            System.Windows.Duration _duration = _mplayer.NaturalDuration;






            // Get the Skype Object


            SKYPE4COMLib.Skype _oSkype


                = new SKYPE4COMLib.Skype();


            // Search for the user in the contacts


            SKYPE4COMLib.User _oUser


                = _oSkype.SearchForUsers(strContact)[1];


            // Create the SkypeCall


            SKYPE4COMLib.Call _oCall


                = new SKYPE4COMLib.Call();


            _oCall = _oSkype.PlaceCall(strContact, "", "", "");


            // Wait until the call is picked up or 180 seconds


            int _i = 0;


            while ((_oCall.Status !=




                    & (_i < 1800))










            if (_i < 1800)






                // Play the wav








                // Wait for the WAV file to be finish playing before to hang up




                    (Convert.ToInt32(_duration.TimeSpan.TotalMilliseconds) + 500);


                // Hang up










                System.IO.StreamWriter _StreamWriter;


                _StreamWriter = new System.IO.StreamWriter










                DateTime _time = DateTime.Now;




                _StreamWriter.WriteLine("<!-- " + _time.ToLocalTime().ToString() + " -->");




















        catch (Exception _ex)




            System.IO.StreamWriter _StreamWriter;


            _StreamWriter = new System.IO.StreamWriter










            DateTime _time = DateTime.Now;




            _StreamWriter.WriteLine("<!-- " + _time.ToLocalTime().ToString() + " -->");



















  9. So let us recall here: we need something to send from PI Notifications to this web service. I have chosen to take four parameters: the Element (in fact I will use the Notification instead of the Element as it is more user-friendly than an element including path), the trigger time, a description and finally the Skype contact. The following code consists of a method that creates my WAV file (we can keep it for further documentation), creates a reasonable message from the element, the trigger time and the description, and calls the code to create my WAV file and do the Skype call.

    publicstring Wav2Skype(string strElement,


                            string strTime,


                            string strDescription,


                            string strContact)




        DateTime _time = DateTime.Now;


        string strFilename;


        strFilename = System.Web.Hosting.HostingEnvironment.MapPath




                         + _time.ToString("dd_MM_yy_HH_mm_ss")


                         + ".wav");




                 + ", " + strTime


                 + ", " + strDescription,




        MakeSkypeCall(strFilename, strContact);





  10. Looks like all the heavy-lifting is done! All we need now is we need to expose a web service. Based on the information from MSDN here, we need a delegate and two web methods as below:

    // Delegate


    publicdelegatestringWav2SkypeAsyncStub(string strElement,


                                              string strTime,


                                              string strDescription,


                                              string strContact);




    // Actual method which is exposed as a web service




    publicIAsyncResult BeginWav2Skype(string strElement,


                                       string strTime,


                                       string strDescription,


                                       string strContact,


                                       AsyncCallback asyncCallback,


                                       object asyncState)




        Wav2SkypeAsyncStub _asyncStub


            = newWav2SkypeAsyncStub(Wav2Skype);


        // Using delegate for asynchronous implementation  


        return _asyncStub.BeginInvoke(strElement,












    } // BeginWav2Skype






    publicstring EndWav2Skype(IAsyncResult result)






    } // EndWav2Skype

  11. Build MySkypeWebService using the Build>Build MySkypeWebService menu in Visual Studio.

  12. At this point our web service would now work - well almost. Skype is a nice tool, but keep in mind it is an interactive desktop application which leads to two issues:
    1. We have to open Skype and log in (or set Skype to automatically start and log in at Windows startup).
    2. By default neither IIS nor ASP.NET can interact with the desktop of a user.

    The first issue cannot be solved: you have to be logged in into Skype on the machine providing the web service. The second issue can be solved by doing the following:
    1. In the Services applet, open the properties for the IIS Admin Service service and check the Allow service to interact with desktop option on the Log On tab.
    2. IIS.jpg

    3. Edit the following file
      by locating the line
      <processModel />
      and changing it to
      <processModel autoConfig="true" userName="SYSTEM" password="AutoGenerate"/>

    Note 2: If you want to test your web service with the Echo / Sound Test Service (echo123) you will not hear any echo in your speakers while playing the WAV file.
    Note 3: You will have to set a break point in the web service to not start playing before you are instructed to speak by the Echo / Sound Test Service.

  13. Publish the MySkypeWebService using the Build>Publish MySkypeWebService menu in Visual Studio.

  14. Now we should be free to use the web service in PI Notifications:


  15. And receive a call from PI Notifications:
  16. ASCalling.jpg

Challenge for the community

  1. The above solution requires someone to be logged in to run Skype. I did not find a way to programmatically log in to Skype to avoid this.
  2. We have the wav file. We deliver it via Skype - how about a voice modem as another option?

Recent discussions on using the BLOB (Binary Large Object) point type had me wonder: what do people use BLOBs for?


I'd like to invite you to share your thoughts on what you use BLOB tags for, or what you think would be a good potential usage of those.
And what are your thoughts on Annotations? Annotations vs. BLOB?


Looking forward to reading you!

My new DVR arrived just in time for the 2010 Olympics. BTW...hats off to our northern neighbors for going all out on the games and hosting effort!


The DVR delivered functionality as promised...ability to fast forward over the talking heads while stopping to take in some of the awesome BC panorama and of course thrilling competitions. I didn't precisely measure time to view compared to recorded hours but it was roughly half ... the savings easily should have translated to some much needed beauty sleep.


Well, why didn't it? An unfortunate mix of being a light sleeper and Disk Thrashing Phobia (DTP) has made for many sleepless nights.  I'm sure the DVR must be defective, it's not an uber elite device provisioned with a server class drive spinning at 15K where one would expect high decibels.  Gasp...yes, the DVR is in the bedroom.


But, years of service at OSIsoft can change a person - it's easier to filter out the drone of a jet engine than the unwarranted seeking of a disk. Why is there so much I/O all the time?


The obvious solution is the DVR has to go.  Maybe I'll just run it until the drive self destructs. Tinkering with the DVR is not an option but very tempting...maybe something as simple as a cron job is kicking off defrag.  Now we're finally getting to the point of this blog post, eh! 


How do you really know what's running on your PI Servers and when?


Such a question is of interest in security monitoring and regulatory circles. There are several interesting approaches.  Arguably the built-in Windows Management Instrumentation provides good methods to monitor the internals of a server using a common information model (CIM).  CIM Studio, available in the WMItools download provides a way to browse the very rich and extensive CIM model to get a feel for data available in the Windows implementation.


For this discussion, open CIM studio and connect to namespace root\CIMV2. Then search tool for Win32_Process (or browse logical classes under CIM_ManagedElements for the CIM_Process subclass).  Then pick instances on the right hand panel tool bar to return a data set for all running processes, essentially providing a point in time query of task manager GUI information.  Observe that not all properties are valid for each row.  While this kind of empirical validation of CIM implementation details is less than ideal, in practice it is a good reason to test with CIM studio.


The OSIWMI interface can issue WQL (SQL for WMI) in a style similar to the PI relational database interface. WQL queries support placeholder substitution with values provided by PI.  For example, this query selects processes started since the last query:


WQL=Select * from Win32_Process where CreationDate> '#SST# '; TS= CreationDate; Property=Caption; OrderBy= CreationDate


OSIWMI interface configuration for the above query should target a string tag with the extended descriptor (ExDesc) holding the WQL statement. At runtime, the point snapshot timestamp is substituted for the #SST# placeholder in the where clause.  The query result will be sent to PI using the process creation date for the timestamp.


In a default configuration each image name (CIM Caption) returned in the record set is written to PI as separate events. Various format and concatenation options are set using Location 2 and 3 attributes.  Both columns and rows can be combined into a single event if desired.


If you want to monitor more than the PI server, just change the InstrstrumentTag attribute to identify the target node and CIM model path. Enable the windows firewall rules for WMI and away you go.  Like any Windows RPC over DCOM, this approach works best within a domain.


For NERC CIP responsible entities, WMI provides classes useful for near real time monitoring of ports and services.  In addition to driving notifications, archiving this kind of information in PI can also help remove data retention worries in the event of a cyber incident that isn't immediately recognized.  Hat tip: Bruce at Triencon presented his approach for EMS users in Portland last year (extranet for registered T&D SIG members).


The CIM classes described above perform quite well on my test systems but your mileage may vary.  However, efficient WQL for monitoring the file system (CIM_Directory or CIM_DataFile classes) remains elusive.  If any of you find the secret sauce please let me know.


For those without access to the OSIWMI interface, programmatic access to WMI is wide spread. Some of the more interesting examples are now using Powershell.  I suspect some objectives are better served using a programmatic approach because of WQL limitations, it is not full SQL.


For others who ask isn't there a simpler way?  Maybe, Windows Software Restriction Policies (SRP, or sometimes called SAFER) provides an advanced logging feature that could be enough for this purpose.  SRP was offered starting in Windows 2000, provides a built in whitelist and blacklist technology to help prevent malware.  Configuring these technologies is the challenging part.  You need to know exactly what is supposed to run (or not run).


SRP advanced logging helps gather data to set and troubleshoot configuration. Add or remove a registry key to enable or disable the log.  See technet "Using Software Restriction Policies to Protect Against Unauthorized Software" for complete details.


The log includes image activation events and matches them to SRP rules but unfortunately, timestamp is not part of the record.  Of course PI-UFL could frequently process the log and Windows will create a new one as needed.  There is a caution about not letting the log grow too large but this has not been a problem for me.  Besides PI-UFL does the required housekeeping.


So what IS running on your servers in the middle of the night?  Do you really know?  Sleep well everyone!



Bad news: We are planning to repurpose the machine which is currently hosting the PIWS CTP. This means the CTP will no longer be accessible after this week.


Good news: We are moving forward with our plans to productize PIWS and are working on a Beta release which will be available on vCampus probably in April.


We hope this is a net positive. Please continue to share your feedback on PIWS in this forum, so we ensure we are developing the right product for your needs.

If you're like me, you probably downloaded and installed Microsoft Visual Studio 2010 (VS2010) as soon as it became available (CTP, Beta1, Beta2, RC1 for partners...). And if you are using PI ACE, you might have noticed that the PI ACE Wizard does not appear under the Tools menu like it did in VS2003, VS2005 and VS2008.


Of course, the next release of PI ACE will incorporate nicely in VS2010, but the obvious question is: can I use PI ACE in VS2010 today?


And you probably guessed the answer, thanks to the title of this blog post: yes you can!


First, copy/paste the appropriate files from the %MYDOCUMENTS%\Visual Studio 2008\Addins folder to the %MYDOCUMENTS%\Visual Studio 2010\Addins folder - see screenshot below:




Then you need to edit the PIACEWizard.AddIn file as follows, using your favorite text/XML editor:




And voilà, you now have a nice "PIACEWizard" item that shows up under the Tools menu in VS2010! Just make sure you target the right version of the .NET Framework, the same used by your PI ACE Scheduler (most likely 3.5, if you are using VS2008 to develop your ACE calculations today).



Filter Blog

By date: By tag: