KAMIMUCODE
C sharp REPL 01 May 2010

Introduction

KamimuCode presents CSharpEval, a lightweight C# Eval and Read-Evaluate-Print-Loop (REPL) program.

What is this?

The CSharpEval source code can be included into any C# program. It gives you the ability to compile the contents of a text string containing some C# code and produce a method delegate. This is done at execution time. This delegate is callable from within the executing program, and the newly compiled code can access any of the public resources of the parent program, such as methods and fields and types.

Why is this useful?

How complete is this compiler?

I do not implement the full C# language in CsharpEval; however I have done my best to implement a very useful subset. In particular I have not attempted to implement any of Linq, nor yield return or exception handling. Neither can you define new classes. The standard vanilla features of C# such as statements, expressions, conditionals, loops, and the most commonly used operators, etc are supported.

Getting started

For pedagogical purposes I have written a three-part documentation. The first part introduces the C# REPL program; you can put the C# Eval through its paces and see how well it works. The second part discusses the CsharpEval API. The third part gives complete examples on how to use the CsharpEval code in your programs.

Click on the Download button to download a zipped file containing an exe file. This is an already compiled CsharpEval program that runs the REPL code. Naturally this will only work on a Windows machine. You will also need to have the .NET Framework 3.5 version installed.

So unzip and run this program (no installation is required). You will be presented with an empty edit box. To start things off, enter the universal REPL test:

2+2

Press shift return (don't forget the shift!) and it will print out 4. Or enter the following (including quotes)

"Hello World!"

and press shift return.

Let's try a more complicated expression:

"Number=" + (20.2 * (3.4 + 5.5) - "A String".Length).ToString()
  + " Date and Time=" + DateTime.Now

If there is a syntax error a separate dialog box will pop up telling you of the error. Dismiss this dialog, fix the code and try again. When the code is correct the REPL will compile the method and return with a success message. One possible source of errors is when a unicode character whose index is outside of the standard ascii range is inadvertently used. For example using En dash (code 8211) instead of the minus sign. In such cases you will have to manually edit the input to use the ascii characater.

We shall now code everybody's favourite recursive method:

public int Fib(int i)
 {
   if (i == 0) return 0;
   if (i == 1) return 1;
   return Fib(i - 1) + Fib(i - 2);
 }

Hint: a limited set of copy and paste editing operations are supported. So copy the above code into the clipboard, then click on the edit box in the REPL window, and press shift-insert. You can also use the mouse to highlight text in the REPL window. Position the mouse onto the text, hold the left button down and drag. Now doing a ctrl-shift will copy the text into the clipboard. Once the text is in the REPL window you do basic editing operations on it. The full range of (say) Visual Studio text editing abilities is not available; but you could always add them yourself:).

Now position the cursor onto any line of this method and press shift enter. It will hopefully be compiled. Enter the expression:

Fib(40)

And press shift return. Now wait...

On my machine it will print out 102334155 after about 16 seconds. Your mileage may vary. A discussion on timings is given later in this documentation.

Positioning the cursor on any entry, either just typed in or any past entry, and pressing shift return will invoke the REPL evaluation. If the REPL thinks that the code is incomplete (say unmatched brackets) it will do nothing. Otherwise it will attempt to evaluate the code.

So move the cursor back up to one of the edit entries you have just made, possibly modify it, and press shift enter again. The REPL will re-do the evaluation.

Entering a field

To add a field we type (for example)

public int NewInt ;

The trailing semicolon is important; it signals to the REPL code that this is a field definition and not something else.

CSharpEval compiler will now accept something like

public int Increment ( )
 { NewInt = NewInt + 1 ; return NewInt ; }

It pretends that NewInt is an instance field. Behind the scenes the NewInt field is an element of a list (more about this in the next article) and each reference to NewInt fetches or stores this element. In this case, since NewInt is a value type, each fetch and store will be accompanied by unboxing and boxing operations.

Test this method by entering:

NewInt = 111 ;
  Increment () ;

and pressing shift enter. Now type

NewInt

and press shift enter. The number 112 should be printed.

Note that the statement

int NewInt ;

will also be accepted but it will not appear to do anything. This is because, without the public keyword in front of it, the REPL assumes this is a statement. REPL will take this statement, place it inside a method and compile and execute the method. Since this method has only a single local variable and does nothing else, then nothing will happen.

Some implementation detail

CSharpEval can compile stand-alone methods. It can also compile a group of methods and fields and pretend that they are part of an existing class. An instance of this existing class is supplied by the calling code when calling the CSharpEval API.

In the REPL program, the class instance used is the class that implements the REPL loop itself. So the code you are entering into the REPL window can directly access the REPL internals. Some examples that do this are now given.

Timing

Enter the text

StopwatchOn

And press shift return. The word False will appear on the next line and a cursor will then move to a new edit box. This represents the current value of the StopwatchOn field in the REPL module.

Now enter

StopwatchOn = true ;

And press shift return. This will set the field to true, and so it will now print out a line stating how long in microseconds it took to evaluate. It then prints out Done to say it has done the statement just entered.

Now enter the following:

public string IsStopwatchOn ()
 {
   return StopwatchOn ? "Yes" : "No" ;
 }

and press shift return. Now type in the expression

IsStopwatchOn()

without a semicolon. On pressing shift return the text Yes should be displayed.

You could go back to the edit box with this method text in it, and alter the return statement, perhaps to

return StopwatchOn ?
   "Yes, it is on" :
   "No, you just turned it off" ;

Press shift return and the method will be updated to this new code. Now go to the IsStopwatchOn() expression and press shift return again. The new message should be printed. You can turn off the StopwatchOn flag and test again to see the expected result.

Summary so far

Saving and restoring your work

The user can save and restore the contents of the REPL window, and can also copy the contents to a .cs file. The following commands at the REPL command prompt are provided:

SaveImage ( some_file_name );

This will save the current contents of the REPL window to the given file.

RestoreImage ( some_file_name) ;

This will restore the contents from the file to the REPL window, erasing the current contents.

Note that this only restores the declared methods and fields, and the contents of the edit boxes in the REPL window. It will not fill in the field values, so they will all be either null or 0.

SaveCode ( some_file_name);

This writes the fields and methods to the filename in standard C# format.

The REPL program is mainly present as a means of testing the Eval compiler, although as I point out in the introduction it can also be used for debugging program data. It is probably too limited to be able to be used for writing much production code. However, assuming that the nature of the programming task lends itself to this pattern, you could make use of the above commands and use the REPL to write and incrementally test a C# module.

When it is working to some degree you would use SaveCode () to write the new source to a file, then copy and paste this into the final program.

How fast is this code?

In the above a Fibonacci method was used as an example. On my machine it takes about 15 seconds to do Fib(40). Just by chance there is a Fib method in the CSharpEval source code. It is in the TestProgram class of the full test version. It is a public static method, and so we can call it from the REPL code:

TestProgram.Fib(40)

So pressing Enter here will result in this statement being compiled into a method, and executed. This code will call the Fib method in the parent program. On my machine this call takes 11.4 seconds in debug mode, and 1.4 seconds in optimized mode, and, of course, it generates the same number.

Let's optimize the first Fib function by making it iterative:

public int Fib ( int n )
 {
   if (n<= 1) return n ;
   int last = 1 ;
   int lastlast = 0 ;
   for ( var i = 2 ; i <= n ; i++ ) {
     int temp = lastlast + last ;
     lastlast = last ;
     last = temp ;
   }
   return last ;
 }

Copy and paste this into a new edit box in the REPL, and run it. It should generate the same result but only take 6.4 milliseconds. Running the FibIterative method already supplied in the TestProgram class takes about 0.7 millseconds.

Finally, enter the following method:

public long TestTiming ( int n )
 {
   long total = 0 ;
   for ( var i = 0 ; i<n ; i++ ) {
     total += i * 1000 + i / 10 + 50 * i ;
   }
   return total ;
  }

and press shift enter. Then enter the statement

TestTiming ( 100000000 ) ;

It will take (on my machine) about 2.1 seconds. Running the same routine which has been placed into the TestProgram class, as in

TestProgram.TestTiming ( 100000000 ) ;

takes (on my machine with the compiler optimize flag on) about 1.6 seconds.

You can look at the compiled MIL code produced by the CSharpEval. Just enter Maker.MethodIL ( "TestTiming" ) , with no trailing semicolon, into the REPL window. This generates a listing of the Intermediate Language instructions that CSharpEval produces. You can compare this with the C# compiler generated code by using Reflector. The codes, while arranged differently (I place the loop conditional before the loop body, the C# compiler places it after) look very similar, which explains their similar timings.

So in conclusion:

Coming up next

In the next article I describe how to use CsharpEval in your program.

Article text is under the Creative Commons Zero license. Source codes are under the MIT license.