29 June, 2012

Quartz.net task scheduling


Enterprise business apps often require that some code is executed on regular basis as scheduled task (e.g. notifications, state polling,..). There are some common ways how to achieve that:
- writing Windows Service application with a timer
- writing external .exe application run by Windows Task Scheduler
- plumbing some timer specific code
- ...
- using Quartz.net

Quartz.net 2.0 is open source job scheduling framework that can be used from smallest apps to large scale enterprise systems. Quartz.net is a pure .NET library written in C#. It is ported from popular open source Java scheduling framework Quartz.

Below is a simple recipe how to add support for scheduled jobs under ASP.NET (MVC) app. I use this approach for polling MSMQ queue on specific intervals to receive new data packets. It is worth noting that the Quartz.net framework is very powerful and we use only basic features.

1. Quartz.net configuration can be done either programmatically or using external XML  config file. The main components of Quartz that need to be configured are:
- ThreadPool: provides a set of threads to use when executing jobs;
- JobStore: is responsible for keeping track of all work data that is given to the scheduler (jobs, triggers, calendars). There are two job store implementations (custom stores can be added by implementing IJobStore interface): RAMJobStore, AdoJobStore.
- Scheduler: scheduler itself by a given name and handled instances of a JobStore and ThreadPool.

We add the following lines in web.config (app.config) file (reference to Quartz.dll is required):

…
<configSections>
  <section name="quartz" type="System.Configuration.NameValueSectionHandler" />
</configSections>
  
<quartz>
  <!-- configure Thread Pool -->
  <add key="quartz.threadPool.type" value="Quartz.Simpl.SimpleThreadPool, Quartz" />
  <add key="quartz.threadPool.threadCount" value="10" />
  <add key="quartz.threadPool.threadPriority" value="Normal" />
  <!-- configure Job Store -->
  <add key="quartz.jobStore.misfireThreshold" value="60000" />
  <add key="quartz.jobStore.type" value="Quartz.Simpl.RAMJobStore, Quartz" />
  <!-- configure scheduler -->
  <add key="quartz.scheduler.instanceName" value="MSMQScheduler" />
  <!-- configure jobs and triggers definition -->
  <add key="quartz.plugin.xml.type" value="Quartz.Plugin.Xml.XMLSchedulingDataProcessorPlugin, Quartz" />
  <add key="quartz.plugin.xml.fileNames" value="~/quartzjobs.config" />
  <add key="quartz.plugin.xml.scanInterval" value="10" />
</quartz>
…

2. Definition of jobs & triggers can be also done programmatically or using external XML config file (quartzjobs.config). About jobs and triggers:
- Job: we make .NET class executable by the scheduler simply by making it implement the IJob interface that has only one method Execute(...). It is executed by the scheduler.

Here is our implementation of the MessageQueueJob:

namespace EE.SmsFramework.Dispatcher.Scheduler {
  
  public class MessageQueueJob: IJob {

    public void Execute(IJobExecutionContext context) {
      try {
        Logger.Info("QUARTZ: Scheduled Job started");
        // packet dispatcher
        PacketDispatcher dispatcher = new PacketDispatcher();
        dispatcher.ProcessMessages();
        Logger.Info("QUARTZ: Scheduled Job ended");
      } catch (Exception ex) {
        Logger.Error(" --> PacketDispatcher Execute Error:", ex);
      }
    }

  }
}

- Trigger: it is used to fire the execution of jobs.Most commonly used types are SimpleTrigger (single execution of a job at a given time) and CronTrigger (calendar-like schedules with CRON expressions).

Here is our definition of a single job (MSMQJob) and a single trigger (MSMQJobTrigger) inside quartzjobs.config:

<?xml version="1.0"?>
<job-scheduling-data xmlns="http://quartznet.sourceforge.net/JobSchedulingData"
                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0">

  <processing-directives>
    <overwrite-existing-data>true</overwrite-existing-data>
  </processing-directives>

  <schedule>
    <job>
      <name>MSMQJob</name>
      <group>MSMQJobs</group>
      <description>Dispatches messages from MSMQ and sends them to SMS Postar.</description>
      <job-type>EE.SmsFramework.Dispatcher.Scheduler.MessageQueueJob, SmsFramework.Dispatcher</job-type>
      <durable>true</durable>
      <recover>false</recover>
    </job>
    
    <trigger>
      <cron>
        <name>MSMQJobTrigger</name>
        <group>MSMQJobs</group>
        <description>Triggers MSMQ dispatcher job</description>
        <job-name>MSMQJob</job-name>
        <job-group>MSMQJobs</job-group>
        <!-- every 15 minutes -->
        <cron-expression>0 0/15 * * * ?</cron-expression>
      </cron>
    </trigger>
  </schedule>

</job-scheduling-data>

3. Instantiation of a Quartz.net Scheduler instance is maintained by StdSchedulerFactory which implements ISchedulerFactory. It uses a set of properties (NameValueCollection) to create and initialize a Quartz Scheduler. The properties are generally stored in and loaded from a file, but can also be created by your program and handed directly to the factory. Simply calling GetScheduler() on the factory will produce the scheduler, initialize it (and its ThreadPool and JobStore), and return a handle to its public interface.

We instantiate Scheduler instance inside the HttpApplication OnStart event (Global.asax). Web application instance is required to be always alive (IIS application pool must not be stopped). This is simply done by executing void web request inside OnEnd event.

 public class MvcApplication : HttpApplication {

    private IScheduler scheduler = null;

    protected void Application_Start(object sender, EventArgs e) {
      ...
      // quartz scheduler
        ISchedulerFactory factory = new StdSchedulerFactory();
      this.scheduler = factory.GetScheduler();
      this.scheduler.Start();
      ...
    }

    protected void Application_End(object sender, EventArgs e) {
      ...
      this.PingServer();
    }

    /// <summary>
    /// Pings the server - startup the IIS application pool.
    /// </summary>
    private void PingServer() {
      try {
        WebClient client = new WebClient();
        client.DownloadString(string.Format("http://{0}{1}", Environment.MachineName, VirtualPathUtility.ToAbsolute("~/")));
      } catch (WebException webex) {
        // possible 'Unauthorized' - do nothing
      } catch (Exception ex) {
        Logger.Error(" PingServer error:", ex);
      }
    }
    ...
  }
 
The job MSMQJob is triggered in its own thread every 15 minutes:
...
13:45:00.0133 [Thread:Worker-2] EE.SmsFramework.Common.Logger[INFO]: QUARTZ: Scheduled Job started
13:45:01.0133 [Thread:Worker-2] EE.SmsFramework.Common.Logger[INFO]: QUARTZ: Scheduled Job ended
14:00:00.0133 [Thread:Worker-1] EE.SmsFramework.Common.Logger[INFO]: QUARTZ: Scheduled Job started
14:00:01.0133 [Thread:Worker-1] EE.SmsFramework.Common.Logger[INFO]: QUARTZ: Scheduled Job ended
14:15:00.0133 [Thread:Worker-4] EE.SmsFramework.Common.Logger[INFO]: QUARTZ: Scheduled Job started
14:15:01.0133 [Thread:Worker-4] EE.SmsFramework.Common.Logger[INFO]: QUARTZ: Scheduled Job ended
...


20 February, 2012

.NET Reflection: improved

Every serious .NET developer probably already used .NET reflection mechanism to dynamically invoke methods. Recently I've written plugable framework (using IoC pattern) and I've found out that there is much faster way to invoke methods on different types/assemblies using System.Reflection.Emit.DynamicMethod than regular reflection using Type.InvokeMember(). C# 2.0 (3.5) is required.

The method in different assembly is invoked using delegate. Below is the implementation of static class FastMethodInvoker (we actually generate IL specific code):
  public delegate object FastInvokeHandler(object _target, object[] _params);
 
  /// <summary>
  /// A class to invoke methods (DynamicMethod mechanism).
  /// </summary>
  public static class FastMethodInvoker {
 
    /// <summary>
    /// Invokes method.
    /// </summary>
    /// <param name="_target">Instance of target object.</param>
    /// <param name="_sMethodName">Method name.</param>
    /// <param name="_params">Array of params.</param>
    /// <returns>Result of method.</returns>
    public static object InvokeMethod(object _target, 
                                      string _sMethodName, 
                                      object[] _params) {
      Type type = _target.GetType();
      bool bExists = type.GetMethods().Any(t => t.Name == _sMethodName);
      if (!bExists) 
        throw new ArgumentException(string.Format("Non existing method '{0}' on type '{1}'.", 
                  _sMethodName, 
                  _target.GetType().Name));
      MethodInfo mi = type.GetMethod(_sMethodName);
      FastInvokeHandler fastInvoker = GetMethodInvoker(mi);
      return fastInvoker(_target, _params) ?? null;
    }
 
    /// <summary>
    /// Returns DynamicMethod.
    /// </summary>
    /// <param name="_methodInfo">MethodInfo instance.</param>
    /// <returns>FastInvokeHandler delegate.</returns>
    public static FastInvokeHandler GetMethodInvoker(MethodInfo _methodInfo) {
      DynamicMethod dynamicMethod = new DynamicMethod(string.Empty, 
                                      typeof(object), 
                                      new Type[] {typeof(object), typeof(object[])},
                                      _methodInfo.DeclaringType.Module);
      ILGenerator il = dynamicMethod.GetILGenerator();
      ParameterInfo[] ps = _methodInfo.GetParameters();
      Type[] paramTypes = new Type[ps.Length];
      for (int ii = 0; ii < paramTypes.Length; ii++) {
        if (ps[ii].ParameterType.IsByRef)
          paramTypes[ii] = ps[ii].ParameterType.GetElementType();
        else
          paramTypes[ii] = ps[ii].ParameterType;
      }
      LocalBuilder[] locals = new LocalBuilder[paramTypes.Length];
 
      for (int ii = 0; ii < paramTypes.Length; ii++) {
        locals[ii] = il.DeclareLocal(paramTypes[ii], true);
      }
 
      for (int ii = 0; ii < paramTypes.Length; ii++) {
        il.Emit(OpCodes.Ldarg_1);
        EmitFastInt(il, ii);
        il.Emit(OpCodes.Ldelem_Ref);
        EmitCastToReference(il, paramTypes[ii]);
        il.Emit(OpCodes.Stloc, locals[ii]);
      }
      il.Emit(OpCodes.Ldarg_0);
      for (int ii = 0; ii < paramTypes.Length; ii++) {
        if (ps[ii].ParameterType.IsByRef)
          il.Emit(OpCodes.Ldloca_S, locals[ii]);
        else
          il.Emit(OpCodes.Ldloc, locals[ii]);
      }
      il.EmitCall(OpCodes.Callvirt, _methodInfo, null);
      if (_methodInfo.ReturnType == typeof(void))
        il.Emit(OpCodes.Ldnull);
      else
        EmitBoxIfNeeded(il, _methodInfo.ReturnType);
 
      for (int ii = 0; ii < paramTypes.Length; ii++) {
        if (ps[ii].ParameterType.IsByRef) {
          il.Emit(OpCodes.Ldarg_1);
          EmitFastInt(il, ii);
          il.Emit(OpCodes.Ldloc, locals[ii]);
          if (locals[ii].LocalType.IsValueType)
            il.Emit(OpCodes.Box, locals[ii].LocalType);
          il.Emit(OpCodes.Stelem_Ref);
        }
      }
 
      il.Emit(OpCodes.Ret);
      FastInvokeHandler invoker = 
        (FastInvokeHandler)dynamicMethod.CreateDelegate(typeof(FastInvokeHandler));
      return invoker;
    }
 
    /// <summary>
    /// Emits the cast to reference.
    /// </summary>
    /// <param name="il">The il.</param>
    /// <param name="type">The type.</param>
    private static void EmitCastToReference(ILGenerator il, System.Type type) {
      if (type.IsValueType) {
        il.Emit(OpCodes.Unbox_Any, type);
      } else {
        il.Emit(OpCodes.Castclass, type);
      }
    }
 
    /// <summary>
    /// Emits the box if needed.
    /// </summary>
    /// <param name="il">The il.</param>
    /// <param name="type">The type.</param>
    private static void EmitBoxIfNeeded(ILGenerator il, System.Type type) {
      if (type.IsValueType) {
        il.Emit(OpCodes.Box, type);
      }
    }
 
    /// <summary>
    /// Emits the fast int.
    /// </summary>
    /// <param name="il">The il.</param>
    /// <param name="value">The value.</param>
    private static void EmitFastInt(ILGenerator il, int value) {
      switch (value) {
        case -1:
          il.Emit(OpCodes.Ldc_I4_M1);
          return;
        case 0:
          il.Emit(OpCodes.Ldc_I4_0);
          return;
        case 1:
          il.Emit(OpCodes.Ldc_I4_1);
          return;
        case 2:
          il.Emit(OpCodes.Ldc_I4_2);
          return;
        case 3:
          il.Emit(OpCodes.Ldc_I4_3);
          return;
        case 4:
          il.Emit(OpCodes.Ldc_I4_4);
          return;
        case 5:
          il.Emit(OpCodes.Ldc_I4_5);
          return;
        case 6:
          il.Emit(OpCodes.Ldc_I4_6);
          return;
        case 7:
          il.Emit(OpCodes.Ldc_I4_7);
          return;
        case 8:
          il.Emit(OpCodes.Ldc_I4_8);
          return;
      }
 
      if (value > -129 && value < 128) {
        il.Emit(OpCodes.Ldc_I4_S, (SByte)value);
      } else {
        il.Emit(OpCodes.Ldc_I4, value);
      }
    }
  }
 The usage is simple:

a) Invoking some method on external DLL assembly:
...
// invoke method MyMethod(string) on type MyNamespace.MyClass (MyAssembly.dll)
Assembly a = Assembly.LoadFrom("MyAssembly.dll");
object instance = a.CreateInstance("MyNamespace.MyClass");
object[] args = new object[] { "some_string_argument" };
object result = FastMethodInvoker.InvokeMethod(instance, "MyMethod", args);
...

b) Invoking some method on known instance:
... 
// e.g. invoke GetXml() on DataSet instance
Type type = typeof(System.Data.DataSet);
object instance = Activator.CreateInstance(type);
object result = InvokeMethod(instance, "GetXml", null);
...
As a matter of fact this mechanism is about 5x - 10x faster than regular .NET reflection.

25 December, 2007

Winters moments

Last Sunday before Christmas it was cloudy and cold winter day. I haven't taken pictures for a while and I went to the neighbor hill where is always so many nice moments to catch. I took my old good Fujifilm S5500 camera and I took some shots.. you can see them in the Winter moments slideshow.. Friendly horses were nearby..

Link to pictures