Evaluating code in .NET and building your own Calculation Engine

This is the second post in a series about Project Roslyn. You can find the first blogpost in this series here.

 

To provide a short recap of the first post: Project Roslyn is basically a re-imagined C# and VB.NET compiler. In this series I'm focussing mainly on the C# language. Roslyn provides an API into the C#/VB.NET compilation process. This opens up a lot of new possibilities. In the first blogpost we already saw a prime example of this new technology. With Roslyn it is possible to create an interactive C# or VB.NET console. If you download and install the Roslyn CTP, you already get a console (with Intellisense!) in Visual Studio. If you want to get started with Roslyn, please go to the first blogpost for the instructions.

 

In this post we are going to have a look at code evaluation with Roslyn. Please remember that C# and VB.NET are compiled languages, so the code is compiled 'on the fly' rather then interpreted. This is a big difference between using Roslyn, or using an eval function in Python, JavaScript or PHP.

 

Why is code evaluation a 'big thing' you might ask? Well, you might have encountered a situation where you simply wanted to calculate the result of the string '100 / 5 + 25'. This seems a very simple task, but it is very cumbersome with a compiled language. There are a few options though: the most difficult would be to write a parser and interpreter (for instance using the Interpreter Design Pattern) for this syntax. There are numerous papers written about writing parsers, but it still is a very difficult, big and risky task. It will take up a lot of time for your project for something that seems so trivial.

 

The previous example only talks about simple arithmetic. What if you wanted to further enhance your expressions, so that you could calculate something like '100 + Sin(5) + PIValue('cdt158')' ? This would mean a big change to your custom interpreter, and what about '100 + Sin(100 + 4 + PIValue('cdt158'))'. That would mean you have to build a full-blown language interpreter!

 

There is the option of using the CSharpCodeProvider for C# or the VBCodeProvider for VB.NET. These classes have to ability to compile C# and VB.NET code, and give output to your application. (I have used the CSharpCodeProvider a lot when building extensible applications where the goal was to let users 'script' some application behavior). The issue with this technique is performance, overhead and transparancy.

 

Here is an example of using the CSharpCodeProvider to evaluate the expression '100 + 5 + Sin(2)'.

 

2477.snippet1.png

 

And the output would be

 

6840.output1.png

 

The CSharpCodeProvider basically is a wrapper around the C# command line compiler csc.exe. It is not a compiler (API) in itself. Performance is meager, and there are always assemblies created on disk. (Don't let the 'GenerateInMemory' option of the CompilerOptions fool you: it only means that a temporary assembly is created on disk). This means that performance is slower than expected.

 

There is much overhead: you cannot simply tell the CSharpCodeProvider to compile the string '100 + 5 / 2'. You have to generate some wrapper class, with a method that returns your code.  You then have to use Reflection to get the type and a methodinfo to call the method. This generates a lot of overhead in your application. My guess is that Roslyn does generate some wrapper classes and methods when you evaluate a simple expression, but this is done 'under the hood'.

 

When you create an object with the CSharpCodeProvider, you will get your CLR object back, just like you would in 'normal code'. Other than using Reflection, there is no way of knowing what the object is and what the statements are in a method. With Roslyn, you have transparancy: you will have access to the entire Syntax Tree (this will be discussed in a future blogpost).

 

So, where does Roslyn come in: it solves all the issues described above! It's fast, it has no overhead for the developer (that's us), and it is very transparant!

 

 Lets have a look at how we can achieve the same thing, with Roslyn

 

0724.snippet2.png

 

And the result is exactly the same

 

1614.output1.png

 

This seems clear: we can easily evaluate (C#) expressions with only 3 lines of code! With the CSharpCodeProvider it took about 5 times more.

 

Let's take a look at a more real-world example on how we can use Roslyn. We are going to create an application that can calculate expressions and use PI values. To configure these calculations we will be using a simple XML file.

 

As mentioned in the previous blogpost, you will need the following to get started with Roslyn:

 

Microsoft Visual Studio 2010 SP1 (download here)
Microsoft Visual Studio 2010 SDK SP1 (download here)
Microsoft ‘Roslyn’ CTP (download here)

 

Once you have everything installed, lets fire up Visual Studio 2010 and start a new Console project.

 

1207.screenshot1.png

 

 After that, make references to the following Roslyn libraries, they are typically located in:

  • C:\Program Files (x86)\Reference Assemblies\Microsoft\Roslyn\v1.0 for x64 systems
  • C:\Program Files\Reference Assemblies\Microsoft\Roslyn\v1.0 for x86 systems

 8547.screenshot2.png

 

First, we will create our XML configuration file to the project, let's call it 'Calculations.xml'. In this XML file we have configured two calculations: 'Sinusoid Delta' and 'Athmospheric Tower Vapor Test'. For the first calculation there are two parameters defined, called 'Current' and 'Before'. These parameters refer to PI Tags (in this case 'sinusoid') with a timestamp. The expression is simple: subtract the 'Before' parameter from the 'Current' parameter.The same concept applies to the second calculation.

 

 5353.snippet3.png

 

In our project we will create two classes that represent the information from the XML document. A class named 'SimpleCalculation' to store the expression and the parameters, and a class called 'SimpleParameter' to store parameter settings.

 

6318.snippet4.png

 

 8358.snippet5.png

 

 As you can see, these are very simple classes to hold the information from the XML document. Now, let's go back to our 'Program' file and create a method to read the XML file into a collection of 'SimpleCalculation'. The goal here is to create a static method that reads our 'Calculations.xml' file, and creates an easy to use list with our collections. We will use LinqToXml for this example:

 

4540.snippet6.png

 

 If you are unsure of what the 'yield' keyword does, have a look at this blogpost. This post explains in detail what the 'yield' keyword does. So, this method will provide us with a collection of SimpleCalculation. Let's add a RunCalculations() method in the Program class, and call it from our Main method.

 

8688.snippet7.png

 

 We have not implemented anything yet to really calculate anything, but just to make sure we are on the right track: set a breakpoint in RunCalculations() and hit F5 (run). If you step through, you will see the process of getting the calculations from the XML file at work. If everything runs ok, lets continue to actually run these calculations.

 

Let's first make sure we can get values out of our PI System. Add a reference to OSIsoft.PISDK, and let's create a 'Helper' class to let us retrieve and write values from the PI System.

 

0511.snippet8.png

 

This class will help us with some of our logic when reading and writing data with the PISDK. Offcourse you want to build in better exception handling and value checking, but in order to keep this example simple: let's go with something easy.

 

Let's modify our SimpleParameter class to make use of this PIHelper class, so that it will be able to retrieve the parameter value from the PI System. Your SimpleParameter class should look like this:

 

3644.snippet9.png

 

Whenever we will call GetValue() on one of our Parameters, it will use the PIHelper class to retrieve that value.

 

Now onto the Roslyn 'magic'. We want to be able to evaluate the Expression bit of our SimpleCalculation class. For this we need to modify the SimpleCalculation class to look like the following code snippet. A description of the statements is provided with the sourcecode.

 

4810.snippet10.png

 

The last thing we need to do now is modify the RunCalculations() method in the Program class to actually run the calculations.

 

1732.snippet11.png

 

We are done! Our little calculation engine is completed. Run the application (F5) and see your calculations with PI data being performed.

 

5381.output2.png

 

The full source code is attached to this blogpost. You can now add some calculations and try out different expressions and options with the ScriptEngine.

 

I would really value some feedback on this topic. Is there interest in this? If there is, I would be happy to dive deeper into Roslyn and we can have a look at the Expression Trees, creating Visual Studio plugins and Refactorings, etc. Let me know!

 

Please note that this tutorial and the provided source code is for educational purposes only, do not use this in any production situation.