KAMIMUCODE
C sharp Eval 10 May 2010

Introduction

In this article and the next I describe how to use the C# Eval code in your programs. This article describes the API, while the next article gives detailed examples.

Some features

Of course, nothing is without cost, even if it is free:

Security considerations

Before we proceed I wish to remind all developers to never ever directly expose the C# Eval interface to anonymous users. Direct evaluation of arbitrary C# code is an excellent vector for un-authorised access to your machine! Make sure that all input from anonymous users is heavily pre-processed to prevent any such attacks.

Setting up a demonstration C# project with CSharpEval

The steps needed to get an initial stand-alone version running on your machine are:

If all is well, the program will first run through its unit tests of its compiler, and then it will bring up the Repl dialog box. If you get to this point then you have a working copy. Now we need to talk about the different configurations that are possible.

Different configurations

The CSharpEval project code can be configured in several ways. The minimal configuration provides only the compiler to turn the source code of a method into a delegate. The maximal configuration provides unit tests plus the full Repl environment together with dialogs for reporting compilation errors. The user chooses which configuration they want by the use of the conditional compilation symbols.

In the first two cases above the Dialog symbol is essential. In the second two cases this symbol is optional. If it is used then error reporting is done by the pre-supplied error dialog. If this Dialog symbol is not used then:

Each of these configurations has its own Program file, thus allowing the configuration to be a stand-alone program. One of these configurations includes the units tests; normally you would not put these into any production program. The source code for the other four configurations can be placed into a production program. You will need to remove the Program file for each such configuration; the setup code in each has to be copied across to your code and modified to suit your needs.

The Repl evaluation process

Running the CsharpEval code in the Repl configuration allows you to enter C# code and have it executed immediately. Behind the scenes the Repl program takes your code, surrounds it with some extra code and then compiles and runs it. If the code you type in is an expression, then the Repl will generate the following code:

partial class REPL {
   public string DoExpression () {
     return (Expression-goes-here).ToString() ;
   }
 }

while a statement is turned into

partial class REPL {
   public void DoStatement () {
     Statement-goes-here
   }
 }

These methods are then compiled by the CSharpEval and then executed. In the case of the expression, the return result is printed to the Repl window.

Notice the partial class REPL in the above. The basic purpose of the CSharpEval program is to turn source code into stand-alone dynamic methods. However it is very useful to allow such methods to call each other as if they were normal methods of a single .Net Type. There is no direct way of creating dynamic types in the .Net environment (except by compiling separate assemblies). So the CSharpEval program fakes it. Behind the scenes it uses an array of method references to hold the methods of a single type, and it compiles the code so it looks as if the methods are inside a pre-supplied .Net Type. It also allows fields to be added to this type, which is done by putting the field values into an array of objects.

So either of the above methods are compiled to a dynamic method which pretends it is a method of the class REPL. This class happens to be the class that implements the Repl functionality. So the compiled code can directly access the fields and methods of the Repl module. As an example, if the statement

StopwatchOn = true ;

is typed into the command line, it will turn on the StopwatchOn flag in the Repl class. When this flag is true the Repl will print the time taken to evaluate each Repl request.

The REPL class is used here since that is what implements the Repl and it is most convenient to allow code entered into the Repl window to directly access the Repl internals. When calling the CSharpEval compiler directly in production code, the user programmer can specify any class. The only requirement is that this class must have a field like the following

public List<object> Fields ;

if the programmer wants to add fields as well as methods. If the programmer does not want to add fields this is not required.

Using Eval

The main use of this system is to programmatically compose some source code, then compile it and obtain a delegate.

This sounds complicated, but don't worry. In the following article I will provide boiler-plate code for all of this. For now, however, we will continue with the documentation of the API.

TypeParser

Every real C# program needs to have a list of references specifying which assemblies are to be included into the final compilation. Each source file in a C# program can have a list of using statements, which specify which namespaces are to be used without having to explicitly specify the full namespace name.

The CSharpEval also needs this information. Rather than compile this information in from the source string each time, it is pre-computed and passed to the CSharpEval code as an input parameter.

The references and using information is stored in an instance of a class called TypeParser. A new instance of this is created for each different set of references and using statements you may require. The TypeParser instance is used by the CSharpEval code to actually evaluate types, using the assemblies and namespaces provided.

The following demonstrates the construction of a TypeParser instance:

TypeParser parser = new TypeParser(
   Assembly.GetExecutingAssembly(),
   new List<string>()
   {
     "System" ,
     "System.Collections.Generic" ,
     "System.Linq" ,
     "System.Text" ,
     "System.Windows" ,
     "System.Windows.Shapes" ,
     "System.Windows.Controls" ,
     "System.Windows.Media" ,
     "System.IO" ,
     "System.Reflection" ,
     "Kamimu"
   }
 );

The first parameter to the TypeParser constructor is an Assembly, which will normally be the currently executing assembly. The TypeParser code will use this assembly, plus all assemblies directly accessed by this assembly, during the compilation process.

The second parameter to the constructor is a list of namespaces. This corresponds to the using statements of a C# file.

The constructed parser instance can now be passed to calls to the CsharpEval API methods. It can also be assigned to a static field in the TypeParser class, as in

TypeParser.DefaultParser = parser;

This default parser is used by some of the CsharpEval methods.

Lexical analysis

The CSharpEval compiler takes the source in the form of a LexList instance. A LexList instance is a list of tokens of type LexToken. To build the final LexList the helper class LexListBuilder is used. The typical pattern of use for this builder is

LexListBuilder lb = new LexListBuilder();
 lb.Add(some_string);
 lb.Add(another_string).Add(another_string2);
 return lb.ToLexList();

where there can be any number of Add's to append more strings. There are some variations on the Add method:

lb.Add("public int 'FunctionName ( int i )" ,
 "FunctionName", the_Name)

Here you are creating the method header of the method to be compiled. Rather than use string concatenation to include the method's name, you can use a parameter name (in this case FunctionName). The single quote mark in front of it indicates to the Lexical stage that this is a parameter. The parameter value will be taken from the remaining arguments to the Add call. These arguments are inserted in pairs, the first is the name of the parameter (without the quote mark) and the second is the expansion value.

Any number of parameters may be used in a single call. A parameter name may be used more than once in the input string, all occurrences of it will be replaced with the same expansion name.

LexList MakeTheMethodHeader ( Type theReturnType )
 {
   LexListBuilder lb = new LexListBuilder () ;
   lb.Add ( "public 'Type Fn ( int i ) ",
   "Type" , theReturnType ) ;
   // and so on
   return lb.ToLexList() ;
 }

Here the programmer wants the return type of the delegate to be the same type as is in the argument theReturnType that is passed into the method. Rather than use the TypeParser instance to generate the text name of this type, then include that into the source (which the TypeParser instance will eventually parse during the compilation process and convert back to a type), the type value is simply supplied as a parameter expansion.

This is very convenient, it ensures that the type you want is what is used, without worrying about whether the TypeParser instance used in the compilation process is set up to reference this type.

lb.AddAndPromoteQuotes (
 "string s = 'contents' ; char ch = `a` ; " );

Here the string contents will be converted so as to be equivalent to the following call

lb.Add (
 "string s = \"contents\" ; char ch = 'a' ; " );

and the final string contents will therefore be

string s = "contents" ; char ch = 'a' ;

Compiling a method

To compile a single method, use the static method MakeMethod.Compile. This method has a single type parameter which specifies the type of the delegate to be returned. It has two actual parameters, the first is the TypeParser instance, the second is the list of lexical tokens which represents the source code. An example is:

Func<int,string> fn = MakeMethod<Func<int,string>>.Compile (
 parser , theLexList ) ;

The parser contains the TypeParser instance to use, and theLexList contains the lexical list of the source. The delegate is called using the standard C# syntax. For example,

String s = fn ( 234 ) ;

To compile an expression and run it immediately, use the static method MakeMethod.DoExpression (). For example

Double d = MakeMethod.DoExpression<double> ( '23.45 / 32' ) ;

This requires the type parameter to specify the return type. The parser instance is not supplied, instead the default parser instance is used. The source is specified as text, not as a LexList.

To compile a statement and run it immediately, use the static method MakeMethod.DoStatement () in the same way. For example,

MakeMethod.DoStatement ( "StopwatchOn = true ; " ) ;

Note that in this case no type parameter is required.

Compiling methods of a class

To compile one or more methods and one or more fields that are to act as if they are members of a class, use the type MakeClass. This type is created with two actual parameters: the first is the TypeParser instance, the second is the list of lexical tokens that represents the source code. This instance now contains all of the information that allows a number of method and field definitions to act as if they are members of a class.

To obtain the delegates from this instance, the GetFunc and GetAction methods in this instance are used. The example below will hopefully make this clear.

First we define some variables to hold the delegates

Func<TestClass, int> getLength;
 Action<TestClass, int[]> setArray;
 Action<TestClass> actInit;

Now we create the MakeClass instance

MakeClass mc = new MakeClass(parser, LexList.Get(@"
   partial class TestClass
   {
     public int[] LocalInt ;
     public void SetArray ( int[] input )
     {
       LocalInt = input ;
     }
     public int GetLength () { return LocalInt.Length ; }
   }")).
 GetFunc<TestClass, int>("GetLength", out getLength).
 GetAction<TestClass, int[]>("SetArray", out setArray).
 GetAction<TestClass>("FieldsInitialiser", out actInit);

Notice the three calls to GetFunc and GetAction at the end. Each such call returns the class instance, so they can be chained together, or specified separately. Each call specifies the type of the class we are compiling to (in this case it is called TestClass), the name of the method, and an out parameter that collects the delegate.

You will see a method in the above that was not explicitly specified - FieldsInitialiser. In a genuine C# class the fields are initialized automatically by the class itself. FieldsInitialiser does the equivalent for any fields you have added. If you haven't added any, then this call is not necessary. It is necessary after each time you compile in any more fields. This method can be called multiple times and it will only initialize the most recent new fields.

TestClass tc = new TestClass();
 actInit(tc);
 int[] thearray = new int[300];
 setArray(tc, thearray);
 int I = getLength(tc) ;

And this code demonstrates the delegates in use. It creates an instance of the TestClass, calls the FieldsInitialiser method on it, then uses the two compiled methods.

You need to keep in mind that the TestClass instance does not know anything about these extra methods and fields. You could define two different MakeClass instances using the exact same code, creating the delegates as above. You will now be able to call these two separate groups of delegates and their effects will be totally independent of each other. (Unless, of course, your MakeClass code explicitly changes genuine C# fields in the TestClass instance. In this situation the two different groups will be able to affect each other.)

In the next and last article in this series I present lots of examples.

Article text is under the Creative Commons Zero license. Source codes are under the MIT license.