Project Roslyn - Opening up the .NET Compiler Part I

Blog Post created by MichaelvdV@Atos on Oct 27, 2011

Introduction to Project Roslyn

I have to admit that I was one of those people that were quite worried before the big Microsoft BUILD event. There were a lot of rumors around Microsoft killing off Silverlight and WPF, and going with JavaScript and HTML as the preferred application development tools.


After I watched some of the videos of the BUILD conference I was more at ease, but still quite worried about the direction that was taken. This was my state until I watched this presentation by Anders Hejlsberg.


Anders talks about future directions for C# and Visual Basic. There is a lot of interesting stuff happening. The adoption of (true) async programming in the .NET framework is very promising. One thing really made my day, and got me that same enthusiasm back as when Microsoft announced Silverlight 2. Like back then, I was happy like a little puppy.


I remember I saw something like this at the local Microsoft DevDays when the beta of .NET 4 was announced. The presenter showed a (very simple) C# interactive console, and told us this was something they were working on. The thought of having real C# code evaluation sounded like a dream to me. I have struggled a lot with evaluating simple expressions in C# (up to the point of writing my own interpreter for it). Having this in the framework was huge. We had to wait a few years, and probably have to wait for after the next .NET release... But it is finally happening!


I’m talking about a new project called ‘Project Roslyn’. Microsoft made Roslyn available as a CTP about a week ago. Roslyn will bring a whole variety of new and very powerful features to the C# and VB.NET languages. I think we (including Microsoft itself) cannot envision the implications this new technology is going to have. Basically, Roslyn is a new compiler for VB.NET and C#, written in VB.NET and C# respectively. And it’s all opened up!


Compilers are typically seen as a ‘black box’. That is, if you are not a compiler developer. It works quite simple, you put source code in, and you get back something you can execute. What happens in between is something magical for most of us.


Well, most of us know some things about how a compiler works: there is the parsing of the source code, the creation of the syntax tree, creating a list of all the symbols, binding the symbols with the objects, and so on.


Roslyn opens all this up to us developers. We can now ‘interact’ with the compiler. This will open up a whole new world of possibilities like code generation, code evaluation, code analysis, meta-programming and refactoring.


Roslyn exposes the C# and VB compiler’s code analysis to us by providing an API layer that mirrors the compiler pipeline.


First, we have the parse phase. Here the source is tokenized and lexical analysis is performed. The result is called a ‘syntax tree’. This is a tree representation of the syntactic structure of the source code. Then we have the ‘declaration phase’. Here declarations are analyzed to form named symbols. Next, the identifiers in the code are matched to the symbols in the bind phase. The last phase, the emit phase, will take care of emitting all the information that has been gathered and build in the previous phases to an assembly.


If we look at the structure of the Roslyn project, each phase is represented by an object model that gives information about that phase. The parsing phase is exposed as a Syntax Tree, the declaration phase as a hierarchical symbol table, the binding phase as a model that exposes the result of the semantic analysis of the compiler. And finally the emit phase as an API that produces IL byte codes.


This is all nice and very theoretical. Now for some real world example and application. First we will need to get the Project Roslyn installed.


To get started with Roslyn you will need:


Microsoft Visual Studio 2010 SP1 (download here)
Microsoft Visual Studio 2010 SDK SP1 (download here)
Microsoft ‘Roslyn’ CTP (download here)


Once you have everything installed, you should go to the install directory. This is typically


C:\Program Files\Microsoft Codename Roslyn CTP\ for x86 systems, or
C:\Program Files (x86)\Microsoft Codename Roslyn CTP\ for x64 systems.


You should definitely checkout the ‘Readme’ directory. There is a readme html application there that will take you through some of the basics of the libraries and installed Visual Studio templates. Next is the ‘Documentation’ directory. There are 13 documents there which will be your primary source of information when working with Roslyn.


One of the first and simplest real world examples is the possibility of an interactive C# console, otherwise known as a REPL (Read Evaluate Print Loop) console. You can use the interactive C# console from Visual Studio. You will find it under View > Other Windows > C# Interactive Window.


The interactive window is a prime example of the possibilities of Roslyn. You can create variables, create expressions, etc. You can really use C# in an interactive manner.


You can also create some more complex statements, for example:




This is an even more complex example, where we create a new class called ‘Person’. It has a name and age property. It also has a Speak() function. This example will really show the ‘magic’ of Roslyn.


The real exciting thing here is  not only the (on-the-fly) evaluation of the code, but especially the automatic indentation of the class declaration and the full intellisense and auto completion. The console does not just evaluate a bunch of code (something we could do earlier with the Mono REPL), but we also have dynamic intellisense and auto completion. We had this in Visual Studio, but now it’s open to us!


I would really like to encourage you to install Roslyn. The C# interactive window only is already of great value. You will no longer have your Visual Studio projects folder filled with ConsoleApplication1 to ConsoleApplication123 because you had to check some code.


In the next blog post we will dive deeper into Roslyn, and we will start writing some code using Roslyn. We will be making use of the ScriptEngine that Roslyn provides. We are going to make a simple calculation engine for the PI system, and we will see that with a few lines of code we can already build something powerful and useful.