Sunday, January 24, 2010

How to Track the RuleEngine rules in WF without starting a workflow runtime

Gents,

Recently I had a problem at hands, I wanted to run run the RuleEngine from Workflow Foundation outside WF.
It is possible and easy to run the RuleEngine outside of WF, but the possibility to track which rules were executed was skipped, as far as I found and saw in the API.

You can track executed rules when you run the workflow runtime itself and attach a TrackingService, but it came out to slow in the test we did.

The problem was that every time we called a WCF service method, the runtime was started for executing a couple of rules, sounds like overkill to me.
So I created a little ‘hack’ to execute the rules with tracking without actually starting the workflow.

Let’s jump into code :)

I created a RuleHelper<T> that executes my rules.
Below you can find the execute method of the helper class:

public void Execute(T objectToRunRulesOn, bool trackData)
{
if (trackData)
{
// Create dummy ActivityExecutionContext and see that trackData is intercepted.
// Initialize with own activity and insert IWorkflowCoreRuntime

Type activityExecutionContext = typeof(Activity).Assembly.GetType("System.Workflow.ComponentModel.ActivityExecutionContext");
var ctor = activityExecutionContext.GetConstructor(BindingFlags.Instance BindingFlags.NonPublic, null,
new[] { typeof(Activity) }, null);
var activity = new InterceptingActivity();
var context = ctor.Invoke(new object[] { activity });

_ruleEngine = new RuleEngine(_ruleSet, typeof(T));
lock (SyncRoot)
{
InterceptingActivity.Track += InterceptingActivity_Track;
_ruleEngine.Execute(objectToRunRulesOn, (ActivityExecutionContext) context);
InterceptingActivity.Track -= InterceptingActivity_Track;
}
}
else
{
Execute(objectToRunRulesOn);
}
}


The intercepting activity does actually all the work.

We first create our own (dummy) ActiviyExecutionContext, problem was, that it is an ‘internal’ class.

The argument to the context is our own InterceptingActivity.
After the instantiation we create our RuleEngine (from the WF assemblies) with a loaded or created RuleSet and the type where to run the RuleSet on.
Then some plumbing code for multiple threads (SyncRoot) and then we attach a static event to the activity.
Next we will execute the rules with the RuleEngine on the specified object that needs checking, and with the self created context which has knowledge of our own created Activity.

Let’s see what the InterceptingActivity does:

using System;
using System.Reflection;
using System.Reflection.Emit;
using System.Workflow.Activities.Rules;
using System.Workflow.ComponentModel;
using System.Workflow.ComponentModel.Compiler;
using System.Workflow.Runtime;

namespace Whizzo.Framework.Rules
{
public class InterceptingActivity : Activity
{
// Some caching variables (ExecutionEvent)
private static FieldInfo _argsFieldInfo;
// Some caching variables (InjectAllTheNeededHandlers)
private static ConstructorInfo _executorCtr;
private static PropertyInfo _currActivity;
private static Delegate _handlerDelegate;
private static FieldInfo _workflowExecutionEventFieldInfo;
private static FieldInfo _workflowCoreRuntimeFieldInfo;

// Static event for tracking rules
public static event EventHandler<TrackingEventArgs> Track;

public InterceptingActivity()
: base("InterceptingActivity")
{
InjectAllTheNeededHandlers();
}

private void InjectAllTheNeededHandlers()
{
if (_handlerDelegate == null)
{
// Get the type of the WorkflowExecutor
Type executorType =
typeof (WorkflowEventArgs).Assembly.GetType("System.Workflow.Runtime.WorkflowExecutor");
// Get eventargs type, the event and the handler type
Type eventTypeType =
typeof (WorkflowEventArgs).Assembly.GetType(
"System.Workflow.Runtime.WorkflowExecutor+WorkflowExecutionEventArgs");
EventInfo evt = executorType.GetEvent("WorkflowExecutionEvent",
BindingFlags.Instance BindingFlags.NonPublic);
Type handlerType = TypeProvider.GetEventHandlerType(evt);
// Get current activity of WorkflowExecutor
_currActivity = executorType.GetProperty("CurrentActivity",
BindingFlags.Instance BindingFlags.NonPublic);
// Get the constructor
_executorCtr = executorType.GetConstructor(BindingFlags.Instance BindingFlags.NonPublic, null,
new[] {typeof (Guid)}, null);

// Get field which has the event handler
_workflowExecutionEventFieldInfo = executorType.GetField("_workflowExecutionEvent",
BindingFlags.Instance BindingFlags.NonPublic);
// Get workflowCoreRuntime field of activity
_workflowCoreRuntimeFieldInfo = typeof (Activity).GetField("workflowCoreRuntime",
BindingFlags.Instance BindingFlags.NonPublic);

// Create dynamic method in module of workflow
Module m = typeof (WorkflowEventArgs).Assembly.GetModules()[0];
DynamicMethod dm = new DynamicMethod("MyHandler", null, new[] {typeof (object), eventTypeType}, m, true);
MethodInfo execMethod = GetType().GetMethod("ExecutionEvent");

// Generate method body
ILGenerator ilgen = dm.GetILGenerator();
ilgen.Emit(OpCodes.Nop);
ilgen.Emit(OpCodes.Ldarg_0);
ilgen.Emit(OpCodes.Ldarg_1);
ilgen.Emit(OpCodes.Call, execMethod);
ilgen.Emit(OpCodes.Nop);
ilgen.Emit(OpCodes.Ret);

// Create delegate
_handlerDelegate = dm.CreateDelegate(handlerType);
}

// Create instance of WorkflowExecutor
object executor = _executorCtr.Invoke(new object[] { Guid.NewGuid() });
// Set current activity of WorkflowExecutor
_currActivity.SetValue(executor, this, null);

// Attach delegate to event
_workflowExecutionEventFieldInfo.SetValue(executor, _handlerDelegate);

// Set executor as workflowCoreRuntime
_workflowCoreRuntimeFieldInfo.SetValue(this, executor);
}

public static void ExecutionEvent(object sender, EventArgs eventArgs)
{
if(Track != null)
{
if (_argsFieldInfo == null)
{
_argsFieldInfo = eventArgs.GetType().GetField("_args",
BindingFlags.NonPublic BindingFlags.Instance);
}
var argsValue = _argsFieldInfo.GetValue(eventArgs);
// Extract args
RuleActionTrackingEvent args = (RuleActionTrackingEvent) argsValue;
Track(sender, new TrackingEventArgs(args));
}
}
}
}

InjectAllTheNeededHandlers is the most important method,
it will create a System.Workflow.Runtime.WorkflowExecutor and it will initialize all it’s needed variables to be able to execute the rules.

It will also attach a self created DynamicMethod converted to a Delegate as an event handler in the WorkflowExecutor.

The event will be fired by the executor (internal class of Microsoft) and this way we can intercept the result of the executed rules.

The method that will get fired is ExecutionEvent, which will fire another event if attached (in the Execute method of the helper) and which will pass the RuleName and the Result of execution.

Voila, hopefully this was a little bit clear :)

There are probably other ways to do this (without starting the runtime) but this way was good enough for me :)

At a blog (http://blogs.msdn.com/moustafa/archive/2006/08/05/689776.aspx) I also found another method to track the executed rules outside the workflow,
But it involved Tracing while this method uses the tracking inside the workflow, without actually executing it. It's a lot simpler, and probably cleaner.

You will have to enable tracing for WF and write a custom TraceListener.

But I prefer to do it without diagnostics tracing put on.

Some feedback in the comments indicated that there were a couple of thing short in the code I displayed above. I'll try to complete it a little more.

This is the TrackingEventArgs class, used for triggering the 'Track' event, which we subscribe to, to track the rules.



using System;
using System.Workflow.Activities.Rules;

namespace Whizzo.Framework.Rules
{
public class TrackingEventArgs : EventArgs
{

public TrackingEventArgs(RuleActionTrackingEvent args, string rulesetName)
{
Args = args;
RulesetName = rulesetName;
}

public RuleActionTrackingEvent Args { get; private set; }

public string RulesetName { get; private set; }

}
}

This is the complete RuleHelper class, I will show below how to call it:


using System;
using System.IO;
using System.Runtime.Serialization.Formatters.Binary;
using System.Workflow.Activities.Rules;
using System.Workflow.Activities.Rules.Design;
using System.Workflow.ComponentModel;
using System.Reflection;
using System.Collections.Generic;

namespace Whizzo.Framework.Rules
{
public class RuleHelper<T>
{
private RuleSet _ruleSet;
private RuleEngine _ruleEngine;
private static readonly object SyncRoot = new object();
private List<TrackingEventArgs> _ruleMessages;

public RuleHelper()
{
_ruleMessages = new List<TrackingEventArgs>();
}

public void SetRules(RuleSet ruleSet)
{
_ruleSet = ruleSet;
}

public void Execute(T objectToRunRulesOn)
{
_ruleEngine = new RuleEngine(_ruleSet, typeof(T));
_ruleEngine.Execute(objectToRunRulesOn);
}

public void Execute(T objectToRunRulesOn, bool trackData)
{
if (trackData)
{
// Create dummy ActivityExecutionContext and see that trackData is intercepted.
// Initialize with own activity and insert IWorkflowCoreRuntime

Type activityExecutionContext = typeof(Activity).Assembly.GetType("System.Workflow.ComponentModel.ActivityExecutionContext");
var ctor = activityExecutionContext.GetConstructor(BindingFlags.Instance BindingFlags.NonPublic, null,
new[] { typeof(Activity) }, null);
var activity = new InterceptingActivity();
var context = ctor.Invoke(new object[] { activity });

_ruleEngine = new RuleEngine(_ruleSet, typeof(T));
lock (SyncRoot)
{
InterceptingActivity.Track += InterceptingActivity_Track;
_ruleEngine.Execute(objectToRunRulesOn, (ActivityExecutionContext)context);
InterceptingActivity.Track -= InterceptingActivity_Track;
}
}
else
{
Execute(objectToRunRulesOn);
}
}

public List<TrackingEventArgs> GetRuleMessages()
{
return _ruleMessages;
}

private void InterceptingActivity_Track(object sender, TrackingEventArgs e)
{
#if DEBUG
Console.WriteLine("{0} Rule result of {1} = {2}", e.RulesetName, e.Args.RuleName, e.Args.ConditionResult);
#endif
_ruleMessages.Add(e);
}
}
}

To use the RuleHelper<>, do something like this, it's pretty self-explainatory:

// Call the rules helper
RuleHelper ruleHelper = new RuleHelper();
// Set the rules
ruleHelper.SetRules(ruleSet);
// Execute the rules.
ruleHelper.Execute(rr, true);
// Get the tracked rules
var ruleMessages = ruleHelper.GetRuleMessages();

Regards,

W.

Friday, July 3, 2009

"Outlook could not create the work file. Check the TEMP environment variable"

This was the error I had when I had installed Office 2007
Word also had this error :

"Word could not create the work file. Check the TEMP environment variable"

After a while of searching and cursing, I've finally found the solution!

First I've took a look at the environment variables, but they pointed to the right directory -> %USERPROFILE%\Local Settings\Temp a lot of sites suggested that that was the problem, but the path was/is valid when I typed it in the explorer.

After a lot of configuration swimming I encountered that outlook also uses registry keys, more specificly these:

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\User Shell Folders


The 'cache' key of both registry entries.

When opening regedit, I saw that the 'User Shell Folders' key was pointing to a k: drive (which I didn't have): k:\Local\Microsoft\Windows\Temporary Internet Files

The 'Shell Folders' was pointing to C:\Users\Whizzo\AppData\Local\Microsoft\Windows\Temporary Internet Files

So I've changed the k drive to my user profile's temp folder:
%USERPROFILE%\Local Settings\Temp

And BAM, the problem was solved!!

I hope this is helpfull for someone ;)

Wednesday, September 3, 2008

Hierarchical Temporal Memory

It's been a while since I've posted a new topic, so I was thinking on what topic to post about on my blog. After a while of hard thinking and sleepless nights (just kidding) it stroked me... The idea entered my mind :)

I while ago I stumbled on an article somewhere regarding HTM (Hierarchical Temporal Memory)... 'What?' you may think? Well, it's an algorithm in A.I. invented by numenta if I'm not mistaken, and it's meant for classifying objects, or at least it can be used for it.

Imagine that you have a cat and a bowl of food in front of the cat, while the cat steps towards the bowl, the image of the bowl changes each step the cat takes.
The cat-brain must then determine whether the bowl is still the same bowl, it all happens in the sub-consciousness of the cat's brain, it's logical that it's the same bowl,
although the bowl is with every step different in size and perspective and if it’s a bummer the lighting may also be different.

Now if we want to translate it to image analysis with A.I., whether to know that an object is still in an image when we present a series of pictures (or movie) this methodology/technology becomes useful :)
It's an algorithm that functions in function of time.
You can pretty much compare it with an Self Organizing Map (SOM) network but in function of time, and with multiple layers.
Each layer is a specialization of the previous layer until the top layer has a classification answer with a certain belief that the thing in the picture is still the same thing.

The basic structure (from the algorithm document) is something like this:



I've found a particular document (paper) explaining the working of the algorithm and
for a more detailed and scientific explanation, I'll have to refer you to their amazing paper.
You can find it here.
As stubborn and curious I am, I'm trying to create something similar, but with my own twist of thought, actually I was thinking of such a mechanism for a long time before I’ve had found this algorithm, only the 'how to start on it' was still missing in my mind, this paper gave me a few ideas on how to create my own program/simulation.

I'll post my findings when my experiments with this algorithm are successful :)
Along with maybe if it's possible a basic peace of code for people who are interested in it for research or self-development.

I must mention that I've looked around on the web and didn't find a piece of code (except the numenta site) that implements this algorithm... such a bummer..
It has probably something to do with the patents obtained on this technology I guess.

Anyway, meanwhile receive my best regards and until the following post ;)

F.

Thursday, August 7, 2008

My first steps in embedded programming

Since I've always been interested in automatisation (domotics) and because I saw the introduction of a new module regarding embedded programming of GHI Electronics, I've bought a development board that makes it able to program through the .NET Micro Framework.

In the beginning I even had never heard of .NET Micro Framework, didn't know that it existed.

I got really excited because this board provides all kinds of interfaces you can think of, analog in/outputs, digital in/outputs, USB interfaces, Ethernet port, serial ports, parallel ports, etc..., even a little Piezo (=speaker) where you can buzz around with ;)

You can program managed in the micro framework, and when you deploy it, the framework uploads the code to the board, and then you can test it standalone.
It's great for making 'smart' devices, and since you can address the FAT file system on USB thumb drives/SD Cards, makes it possible that you can go really far in it.

The device also supports strong keyed assemblies, if you don't have a the right key, then the board will be flashed :) (Which is great :d, except when you've lost it...)

Here are some images of the device:

This is the development board, as you can see, all kinds of ports are present on it.



This is the master module, that contains the code (firmware).
It has also a lot of pins, where you can attach you own electronics on, if you know what you are doing :) The pins are located on the left and richt side of the PCB.
This module is located under the LCD display of the development board.



At the time of writing I've successfully succeeded at letting a LED blink on the board with a given frequency, and even fading of the LED worked ;)
I know, it's stupid but you have to start somewhere :)

Friday, June 27, 2008

Dirty check on objects

A problem that's pretty common is to determine whether an object has been changed, how do we know an object is changed?
You can use for example datasets; datasets keep track of changes, and can update only changed rows.

But in most of the projects that I had to work on, datasets weren't used, or as less as possible, they used their own domain model for storing and transmitting data from point a to point b.

Another problem they encountered was the fact that some of the objects marked as serializable are in fact not really serializable (they are, but have issues), like the ObservableCollection used in combination mostly with WPF. That class is marked serializable, but once you attach the added en removed events, then you can't serialize it anymore, because the delegates aren't marked NonSerializable.

Knowing that these were problems, I've created a solution to bypass all these problems without inheriting or wrapping or using surrogates or whatever.

I've used code that I've blogged about earlier in the IL section:
» Deep cloning objects with IL

This code has the ability to clone objects with IL, you can choose for cloning deeply or shallow, or to exclude fields from being cloned, you can also adapt it to work with other attributes for usage in WCF for example.
It can clone ObserveableCollection because it strips (ignores) the delegates from it.

The code I've written makes it possible to check for changes in an object, it doesn't really matter which object, because it uses a MD5 hash code. The only problem is: knowing what has been changed, but that's not needed in my case.
Feel free to contribute for knowing where the object has been changed, without wrapping it, or adding stuff to it :)

Here's the code that does all the work:

    /// <summary>
    /// Class for checking wether objects have been changed or not.
    /// Clones the object in order to check, for stripping events etc,
    /// but checks against the original object.
    /// Author:     Slaets Filip
    /// Date:       25/06/2008
    /// </summary>
    
public static class DirtyChecker
    {
        
/// <summary>
        /// Cache for holding hashed objects-data.
        /// </summary>
        
private static List<DirtyEntity> _hashesCache = new List<DirtyEntity>();

        
/// <summary>
        /// Retrieve a hashcode for an object.
        /// </summary>
        /// <param name="objToCheck">Object to get hashcode for.</param>
        /// <returns>hexadecimal hashcode</returns>
        
private static string GetHashRecord(object objToCheck)
        {
            
// Create our hash-record of this object.
            
MemoryStream ms = new MemoryStream();
            
BinaryFormatter bf = new BinaryFormatter();
            
bf.Serialize(ms, objToCheck);
            
MD5 md5 MD5.Create();
            
ms.Position 0;
            byte
[] hash md5.ComputeHash(ms);
            
// Convert the hashvalue to a HEX string.
            
StringBuilder sb = new StringBuilder();
            foreach 
(byte outputByte in hash)
            {
                
// Convert each byte to a Hexadecimal lower case string
                
sb.Append(outputByte.ToString("X2").ToLower());
            
}
            
return sb.ToString();
        
}

        
/// <summary>
        /// Check wether an object has changed somewhere or not.
        /// </summary>
        /// <param name="objToCheck">Object to check dirtyness of.</param>
        /// <returns>True when something has been changed in the object.</returns>
        
public static bool IsDirty(object objToCheck)
        {
            
if (objToCheck.GetType().GetCustomAttributes(typeof(SerializableAttribute), true).Length == 0)
            {
                
throw new ArgumentException("Object is not serializable, mark it with [Serializable]""objToCheck");
            
}

            
string name = string.Format("{0}.CloneHelper`1[[{1}]]"typeof(DirtyChecker).Namespace, objToCheck.GetType().AssemblyQualifiedName);
            
// Let's invoke the cloning mechanism
            
Type cloneType Type.GetType(name);
            if 
(cloneType == null)
                
throw new InvalidOperationException("Check the CloneHelper location, must be in same assembly and namespace as DirtyChecker!");
            
MethodInfo mi cloneType.GetMethod("Clone"new Type[] { objToCheck.GetType(), typeof(CloneType) });
            object 
toHashObject mi.Invoke(nullnew object[] { objToCheck, CloneType.ShallowCloning });

            string 
hashRecord GetHashRecord(toHashObject);
            
DirtyEntity entity = new DirtyEntity(objToCheck.GetHashCode(), hashRecord);
            if 
(!_hashesCache.Contains(entity))
            {
                _hashesCache.Add(entity)
;
                return false;
            
}

            DirtyEntity foundEntity 
_hashesCache[_hashesCache.IndexOf(entity)];
            if 
(foundEntity.HashRecord.Equals(entity.HashRecord, StringComparison.InvariantCulture))
            {
                
return false;
            
}
            
if (foundEntity.UpdateCascading)
            {
                foundEntity.HashRecord 
hashRecord;
            
}
            
return true;
        
}

        
/// <summary>
        /// Check wether an object has changed somewhere or not.
        /// </summary>
        /// <param name="objToCheck">Object to check dirtyness of.</param>
        /// <param name="cascaded">If the record is dirty, take the new record as base-record for checking dirtyness.</param>
        /// <returns>True when something has been changed in the object.</returns>
        
public static bool IsDirty(object objToCheck, bool cascaded)
        {
            
if (objToCheck.GetType().GetCustomAttributes(typeof(SerializableAttribute), true).Length == 0)
            {
                
throw new ArgumentException("Object is not serializable, mark it with [Serializable]""objToCheck");
            
}

            
string name = string.Format("{0}.CloneHelper`1[[{1}]]"typeof(DirtyChecker).Namespace, objToCheck.GetType().AssemblyQualifiedName);
            
// Let's invoke the cloning mechanism
            
Type cloneType Type.GetType(name)
            
MethodInfo mi cloneType.GetMethod("Clone"new Type[] { objToCheck.GetType() });
            object 
toHashObject mi.Invoke(nullnew object[] { objToCheck });

            string 
hashRecord GetHashRecord(toHashObject);
            
DirtyEntity entity = new DirtyEntity(objToCheck.GetHashCode(), hashRecord, cascaded);
            if 
(!_hashesCache.Contains(entity))
            {
                _hashesCache.Add(entity)
;
                return false;
            
}

            DirtyEntity foundEntity 
_hashesCache[_hashesCache.IndexOf(entity)];
            if 
(foundEntity.HashRecord.Equals(entity.HashRecord, StringComparison.InvariantCulture))
            {
                
return false;
            
}
            
if (foundEntity.UpdateCascading || cascaded)
            {
                foundEntity.HashRecord 
hashRecord;
            
}
            
return true;
        
}

        
/// <summary>
        /// Class for storing hashing data in the cache.
        /// </summary>
        
private class DirtyEntity
        {
            
private int _hashCodeOfObject;
            private string 
_hashRecord;
            private bool 
_updateCascading;

            public 
DirtyEntity(int hc, string hr)
            {
                _hashCodeOfObject 
hc;
                
_hashRecord hr;
                
_updateCascading = true;
            
}

            
public DirtyEntity(int hc, string hr, bool cascading)
            {
                _hashCodeOfObject 
hc;
                
_hashRecord hr;
                
_updateCascading cascading;
            
}

            
public int HashCodeOfObject
            {
                
get return _hashCodeOfObject}
                
set { _hashCodeOfObject = value; }
            }

            
public string HashRecord
            {
                
get return _hashRecord}
                
set { _hashRecord = value; }
            }

            
public bool UpdateCascading
            {
                
get return _updateCascading}
                
set { _updateCascading = value; }
            }

            
public override int GetHashCode()
            {
                
return base.GetHashCode();
            
}

            
public override bool Equals(object obj)
            {
                
if (obj is DirtyEntity)
                {
                    DirtyEntity de 
obj as DirtyEntity;
                    return 
de._hashCodeOfObject == this._hashCodeOfObject;
                
}
                
return base.Equals(obj);
            
}
        }
    }


bool IsDirty(object objToCheck) and bool IsDirty(object objToCheck, bool cascaded) the first one checks that the object is dirty against a previous version of the object, the previous version being the version when the IsDirty method is called for the first time.

The second method has the cascaded parameter, when it's set to true, then the method will compare against the previous called IsDirty result, each time it's called, the internal hashing-cache will be updated.

An example of usage:

// Create a list of 100 persons...
List<Person> list CreatePersonsList(100);

// Initialize the dirty checker to know how the object looks like
bool dirty1 DirtyChecker.IsDirty(list);

// Change something
list[0].Addresses[0].City "Jut City";

// Check again to see that something has been changed.
bool dirty2 DirtyChecker.IsDirty(list);

// Output:
// dirty1 = false;
// dirty2 = true;


Voila, if it's useful, let me know please ;)

Cheers!

Wednesday, June 25, 2008

.NET to PHP Connector

Howdy,

Wouldn't it be easy to call in .NET a PHP function, and retrieve an answer in the form of an .NET object?

I've looked around on the net and I haven't found a library that does such a thing.
Because I've made a site in PHP for managing kittens and nests (cat-site) etc, where users can add stuff as they want, the client requested to manage the clients and the nests etc, which is pretty logical, but implementing a controlpanel in PHP that does all that was requested was a little too much work, because I had a little budget to take in account.

So I've created a .NET to PHP connector (that I can re-use in other projects) for connecting to the kitten-server and retrieve information about nests etc...

I've also implemented it that way that there is a possibility to easyly change the security of the PHP-Connector.
For the moment the default security is base64. I'm sure there are better ways to send data from point a to point b, but the issue is that if you encode stuff, for sending it to the server, then you have to decode it on the other side also, and you need to decode it with PHP, while it's encoded in .NET, it's not always that easy.

The architecture is pretty easy, it's straight forward, and based on async calls to the server. The usage is also pretty straight forward:



  • Put the gateway PHP file and code file on the server;
  • Instantiate the Connector for connecting to the gateway URL;
  • Call the connector.SendQuery(involvedObject, "methodname", [arguments]);
  • Catch the receive answer event which gives you back your 'involvedObject' and an Answer object containing the server's response.


With this mechanism, I've created an administration tool, which is session-based with login system etc, you can name the gateway as you like, the mechanism only accepts POST requests, and the request are encoded... you can also define methods that may never be called from the connector for security reasons...

I know this system is not super secure, but you can adapt it to you own whishes, and extend it to your believes, it's just a primer where you can base yourselves on.

I will upload the code to a server soon when my project is ready and the code is stable enough, (it's almost ready ;))

Maye a graph would make it more clear how the system works :

Class diagram:



If you have any comments or suggestions, please feel free to leave them ;)

Regards,
F.

Friday, June 20, 2008

Object Deep Cloning using IL in C# - version 1.1

Howdy,

Meanwhile i've been busy with the cloning stuff in IL, and on demand of mr. Sam
I've adapted the code so that it can also clone LINQ entities (EntitySet etc)

The code is not yet in a final stage but it is becoming more and more stable as I refine the code, so, if you have comments or suggestions, please let me know...

I've added in this version:

  • Deep cloning until 1 level (more level's, i'm not yet sure)

  • Use of attributes for defining which field should be deepcloned, shallowcloned or not be cloned at all. You can change the 'CloneAttribute' by a system attribute if you like, when you for example transfer stuff through WCF or something like that. Just map you own attribute to the cloning attribute in the function -> 'GetCloneTypeForField' and it's normally OK.

  • The cloning code is put together in a CloneHelper static generic class, so that you can use it in combination with the ICloneable interface for example, or just in some framework as possible cloning mechanism.



Now let's take a look at the code:

using System;
using 
System.Collections.Generic;
using 
System.Text;
using 
System.Reflection;
using 
System.Reflection.Emit;
using 
System.Threading;
using 
System.Collections;

namespace 
Cloning
{
    
/// <summary>
    /// Enumeration that defines the type of cloning of a field.
    /// Used in combination with the CloneAttribute
    /// </summary>
    
public enum CloneType
    {
        None,
        ShallowCloning,
        DeepCloning
    }

    
/// <summary>
    /// CloningAttribute for specifying the cloneproperties of a field.
    /// </summary>
    
[AttributeUsage(AttributeTargets.Field, AllowMultiple = false)]
    
public class CloneAttribute : Attribute
    {
        
private CloneType _clonetype;
        public 
CloneAttribute()
        {

        }

        
public CloneType CloneType
        {
            
get return _clonetype}
            
set { _clonetype = value; }
        }
    }

    
/// <summary>
    /// Class that clones objects
    /// </summary>
    /// <remarks>
    /// Currently can deepclone to 1 level deep.
    /// Ex. Person.Addresses (Person.List<Address>) 
    /// -> Clones 'Person' deep
    /// -> Clones the objects of the 'Address' list deep
    /// -> Clones the sub-objects of the Address object shallow. (at the moment)
    /// </remarks>
    
public static class CloneHelper<T>
    where T : 
class
    
{
        
#region Declarations
        
// Dictionaries for caching the (pre)compiled generated IL code.
        
private static Dictionary<Type, Delegate> _cachedILShallow = new Dictionary<Type, Delegate>();
        private static 
Dictionary<Type, Delegate> _cachedILDeep = new Dictionary<Type, Delegate>();
        
// This is used for setting the fixed cloning, of this is null, then
        // the custom cloning should be invoked. (use Clone(T obj) for custom cloning)
        
private static CloneType? _globalCloneType CloneType.ShallowCloning;

        #endregion

        #region
 Public Methods

        
/// <summary>
        /// Clone an object with Deep Cloning or with a custom strategy 
        /// such as ShallowCloning and/or DeepCloning combined (use the CloneAttribute)
        /// </summary>
        /// <param name="obj">Object to perform cloning on.</param>
        /// <returns>Cloned object.</returns>
        
public static T Clone(T obj)
        {
            _globalCloneType 
= null;
            return 
CloneObjectWithILDeep(obj);
        
}

        
/// <summary>
        /// Clone an object with one strategy (DeepClone or ShallowClone)
        /// </summary>
        /// <param name="obj">Object to perform cloning on.</param>
        /// <param name="cloneType">Type of cloning</param>
        /// <returns>Cloned object.</returns>
        /// <exception cref="InvalidOperationException">When a wrong enum for cloningtype is passed.</exception>
        
public static T Clone(T obj, CloneType cloneType)
        {
            
if (_globalCloneType != null)
                _globalCloneType 
cloneType;
            switch 
(cloneType)
            {
                
case CloneType.None:
                    
throw new InvalidOperationException("No need to call this method?");
                case 
CloneType.ShallowCloning:
                    
return CloneObjectWithILShallow(obj);
                case 
CloneType.DeepCloning:
                    
return CloneObjectWithILDeep(obj);
                default
:
                    
break;
            
}
            
return default(T);
        
}

        
#endregion

        #region
 Private Methods

        
/// <summary>    
        /// Generic cloning method that clones an object using IL.    
        /// Only the first call of a certain type will hold back performance.    
        /// After the first call, the compiled IL is executed.    
        /// </summary>    
        /// <typeparam name="T">Type of object to clone</typeparam>    
        /// <param name="myObject">Object to clone</param>    
        /// <returns>Cloned object (shallow)</returns>    
        
private static T CloneObjectWithILShallow(T myObject)
        {
            Delegate myExec 
= null;
            if 
(!_cachedILShallow.TryGetValue(typeof(T), out myExec))
            {
                DynamicMethod dymMethod 
= new DynamicMethod("DoShallowClone"typeof(T), new Type[] { typeof(T) }, Assembly.GetExecutingAssembly().ManifestModule, true);
                
ConstructorInfo cInfo myObject.GetType().GetConstructor(new Type[] { });
                
ILGenerator generator dymMethod.GetILGenerator();
                
LocalBuilder lbf generator.DeclareLocal(typeof(T));
                
generator.Emit(OpCodes.Newobj, cInfo);
                
generator.Emit(OpCodes.Stloc_0);
                foreach 
(FieldInfo field in myObject.GetType().GetFields(System.Reflection.BindingFlags.Instance
                | System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Public))
                {
                    generator.Emit(OpCodes.Ldloc_0)
;
                    
generator.Emit(OpCodes.Ldarg_0);
                    
generator.Emit(OpCodes.Ldfld, field);
                    
generator.Emit(OpCodes.Stfld, field);
                
}
                generator.Emit(OpCodes.Ldloc_0)
;
                
generator.Emit(OpCodes.Ret);
                
myExec dymMethod.CreateDelegate(typeof(Func<T, T>));
                
_cachedILShallow.Add(typeof(T), myExec);
            
}
            
return ((Func<T, T>)myExec)(myObject);
        
}

        
/// <summary>
        /// Generic cloning method that clones an object using IL.
        /// Only the first call of a certain type will hold back performance.
        /// After the first call, the compiled IL is executed. 
        /// </summary>
        /// <param name="myObject">Type of object to clone</param>
        /// <returns>Cloned object (deeply cloned)</returns>
        
private static T CloneObjectWithILDeep(T myObject)
        {
            Delegate myExec 
= null;
            if 
(!_cachedILDeep.TryGetValue(typeof(T), out myExec))
            {
                
// Create ILGenerator            
                
DynamicMethod dymMethod = new DynamicMethod("DoDeepClone"typeof(T), new Type[] { typeof(T) }, Assembly.GetExecutingAssembly().ManifestModule, true);
                
ILGenerator generator dymMethod.GetILGenerator();
                
LocalBuilder cloneVariable generator.DeclareLocal(myObject.GetType());

                
ConstructorInfo cInfo myObject.GetType().GetConstructor(Type.EmptyTypes);
                
generator.Emit(OpCodes.Newobj, cInfo);
                
generator.Emit(OpCodes.Stloc, cloneVariable);

                foreach 
(FieldInfo field in typeof(T).GetFields(System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Public))
                {
                    
if (_globalCloneType == CloneType.DeepCloning)
                    {
                        
if (field.FieldType.IsValueType || field.FieldType == typeof(string))
                        {
                            generator.Emit(OpCodes.Ldloc, cloneVariable)
;
                            
generator.Emit(OpCodes.Ldarg_0);
                            
generator.Emit(OpCodes.Ldfld, field);
                            
generator.Emit(OpCodes.Stfld, field);
                        
}
                        
else if (field.FieldType.IsClass)
                        {
                            CopyReferenceType(generator, cloneVariable, field)
;
                        
}
                    }
                    
else
                    
{
                        
switch (GetCloneTypeForField(field))
                        {
                            
case CloneType.ShallowCloning:
                                {
                                    generator.Emit(OpCodes.Ldloc, cloneVariable)
;
                                    
generator.Emit(OpCodes.Ldarg_0);
                                    
generator.Emit(OpCodes.Ldfld, field);
                                    
generator.Emit(OpCodes.Stfld, field);
                                    break;
                                
}
                            
case CloneType.DeepCloning:
                                {
                                    
if (field.FieldType.IsValueType || field.FieldType == typeof(string))
                                    {
                                        generator.Emit(OpCodes.Ldloc, cloneVariable)
;
                                        
generator.Emit(OpCodes.Ldarg_0);
                                        
generator.Emit(OpCodes.Ldfld, field)
                                        
generator.Emit(OpCodes.Stfld, field);
                                    
}
                                    
else if (field.FieldType.IsClass)
                                        CopyReferenceType(generator, cloneVariable, field)
;
                                    break;
                                
}
                            
case CloneType.None:
                                {
                                    
// Do nothing here, field is not cloned.
                                
}
                                
break;
                        
}
                    }
                }
                generator.Emit(OpCodes.Ldloc_0)
;
                
generator.Emit(OpCodes.Ret);
                
myExec dymMethod.CreateDelegate(typeof(Func<T, T>));
                
_cachedILDeep.Add(typeof(T), myExec);
            
}
            
return ((Func<T, T>)myExec)(myObject);
        
}

        
/// <summary>
        /// Helper method to clone a reference type.
        /// This method clones IList and IEnumerables and other reference types (classes)
        /// Arrays are not yet supported (ex. string[])
        /// </summary>
        /// <param name="generator">IL generator to emit code to.</param>
        /// <param name="cloneVar">Local store wheren the clone object is located. (or child of)</param>
        /// <param name="field">Field definition of the reference type to clone.</param>
        
private static void CopyReferenceType(ILGenerator generator, LocalBuilder cloneVar, FieldInfo field)
        {
            
if (field.FieldType.IsSubclassOf(typeof(Delegate)))
            {
                
return;
            
}
            LocalBuilder lbTempVar 
generator.DeclareLocal(field.FieldType);

            if 
(field.FieldType.GetInterface("IEnumerable") != null && field.FieldType.GetInterface("IList") != null)
            {
                
if (field.FieldType.IsGenericType)
                {
                    Type argumentType 
field.FieldType.GetGenericArguments()[0];
                    
Type genericTypeEnum Type.GetType("System.Collections.Generic.IEnumerable`1[" + argumentType.FullName + "]");

                    
ConstructorInfo ci field.FieldType.GetConstructor(new Type[] { genericTypeEnum });
                    if 
(ci != null && GetCloneTypeForField(field) == CloneType.ShallowCloning)
                    {
                        generator.Emit(OpCodes.Ldarg_0)
;
                        
generator.Emit(OpCodes.Ldfld, field);
                        
generator.Emit(OpCodes.Newobj, ci);
                        
generator.Emit(OpCodes.Stloc, lbTempVar);
                        
generator.Emit(OpCodes.Ldloc, cloneVar);
                        
generator.Emit(OpCodes.Ldloc, lbTempVar);
                        
generator.Emit(OpCodes.Stfld, field);
                    
}
                    
else
                    
{
                        ci 
field.FieldType.GetConstructor(Type.EmptyTypes);
                        if 
(ci != null)
                        {
                            generator.Emit(OpCodes.Newobj, ci)
;
                            
generator.Emit(OpCodes.Stloc, lbTempVar);
                            
generator.Emit(OpCodes.Ldloc, cloneVar);
                            
generator.Emit(OpCodes.Ldloc, lbTempVar);
                            
generator.Emit(OpCodes.Stfld, field);
                            
CloneList(generator, field, argumentType, lbTempVar);
                        
}
                    }
                }
            }
            
else
            
{
                ConstructorInfo cInfo 
field.FieldType.GetConstructor(new Type[] { });
                
generator.Emit(OpCodes.Newobj, cInfo);
                
generator.Emit(OpCodes.Stloc, lbTempVar);
                
generator.Emit(OpCodes.Ldloc, cloneVar);
                
generator.Emit(OpCodes.Ldloc, lbTempVar);
                
generator.Emit(OpCodes.Stfld, field);
                foreach 
(FieldInfo fi in field.FieldType.GetFields(System.Reflection.BindingFlags.Instance
                    | System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Public))
                {
                    
if (fi.FieldType.IsValueType || fi.FieldType == typeof(string))
                    {
                        generator.Emit(OpCodes.Ldloc_1)
;
                        
generator.Emit(OpCodes.Ldarg_0);
                        
generator.Emit(OpCodes.Ldfld, field);
                        
generator.Emit(OpCodes.Ldfld, fi);
                        
generator.Emit(OpCodes.Stfld, fi);
                    
}
                }
            }
        }

        
/// <summary>
        /// Makes a deep copy of an IList of IEnumerable
        /// Creating new objects of the list and containing objects. (using default constructor)
        /// And by invoking the deepclone method defined above. (recursive)
        /// </summary>
        /// <param name="generator">IL generator to emit code to.</param>
        /// <param name="listField">Field definition of the reference type of the list to clone.</param>
        /// <param name="typeToClone">Base-type to clone (argument of List<T></param>
        /// <param name="cloneVar">Local store wheren the clone object is located. (or child of)</param>
        
private static void CloneList(ILGenerator generator, FieldInfo listField, Type typeToClone, LocalBuilder cloneVar)
        {
            Type genIEnumeratorTyp 
Type.GetType("System.Collections.Generic.IEnumerator`1[" + typeToClone.FullName + "]");
            
Type genIEnumeratorTypLocal Type.GetType(listField.FieldType.Namespace + "." + listField.FieldType.Name + "+Enumerator[[" + typeToClone.FullName + "]]");
            
LocalBuilder lbEnumObject generator.DeclareLocal(genIEnumeratorTyp);
            
LocalBuilder lbCheckStatement generator.DeclareLocal(typeof(bool));
            
Label checkOfWhile generator.DefineLabel();
            
Label startOfWhile generator.DefineLabel();
            
MethodInfo miEnumerator listField.FieldType.GetMethod("GetEnumerator");
            
generator.Emit(OpCodes.Ldarg_0);
            
generator.Emit(OpCodes.Ldfld, listField);
            
generator.Emit(OpCodes.Callvirt, miEnumerator);
            if 
(genIEnumeratorTypLocal != null)
            {
                generator.Emit(OpCodes.Box, genIEnumeratorTypLocal)
;
            
}
            generator.Emit(OpCodes.Stloc, lbEnumObject)
;
            
generator.Emit(OpCodes.Br_S, checkOfWhile);
            
generator.MarkLabel(startOfWhile);
            
generator.Emit(OpCodes.Nop);
            
generator.Emit(OpCodes.Ldloc, cloneVar);
            
generator.Emit(OpCodes.Ldloc, lbEnumObject);
            
MethodInfo miCurrent genIEnumeratorTyp.GetProperty("Current").GetGetMethod();
            
generator.Emit(OpCodes.Callvirt, miCurrent);
            
Type cloneHelper Type.GetType(typeof(CloneHelper<T>).Namespace + "." typeof(CloneHelper<T>).Name + "[" + miCurrent.ReturnType.FullName + "]");
            
MethodInfo miDeepClone cloneHelper.GetMethod("CloneObjectWithILDeep", BindingFlags.Static | BindingFlags.NonPublic);
            
generator.Emit(OpCodes.Call, miDeepClone);
            
MethodInfo miAdd listField.FieldType.GetMethod("Add");
            
generator.Emit(OpCodes.Callvirt, miAdd);
            
generator.Emit(OpCodes.Nop);
            
generator.MarkLabel(checkOfWhile);
            
generator.Emit(OpCodes.Nop);
            
generator.Emit(OpCodes.Ldloc, lbEnumObject);
            
MethodInfo miMoveNext = typeof(IEnumerator).GetMethod("MoveNext");
            
generator.Emit(OpCodes.Callvirt, miMoveNext);
            
generator.Emit(OpCodes.Stloc, lbCheckStatement);
            
generator.Emit(OpCodes.Ldloc, lbCheckStatement);
            
generator.Emit(OpCodes.Brtrue_S, startOfWhile);
        
}

        
/// <summary>
        /// Returns the type of cloning to apply on a certain field when in custom mode.
        /// Otherwise the main cloning method is returned.
        /// You can invoke custom mode by invoking the method Clone(T obj)
        /// </summary>
        /// <param name="field">Field to examine</param>
        /// <returns>Type of cloning to use for this field.</returns>
        
private static CloneType GetCloneTypeForField(FieldInfo field)
        {
            
object[] attributes field.GetCustomAttributes(typeof(CloneAttribute), true);
            if 
(attributes == null || attributes.Length == 0)
            {
                
if (!_globalCloneType.HasValue)
                    
return CloneType.ShallowCloning;
                else
                    return 
_globalCloneType.Value;
            
}
            
return (attributes[0as CloneAttribute).CloneType;
        
}

        
#endregion
    
}
}


If you have any remarks, please notify me, or when you can use this code, please notify me, you may use it as you whish :)

Regards,

F.