Custom Transactions

If you haven’t used TransactionScope from the System.Transactions namespace, you don’t know what you’re missing.  This system, introduced with .NET 2.0 provides a flexible mechanism for allowing your code to take part in transactions.  Many of the built-in subsystems such as ADO.NET automatically enlist in these transactions, but the real power comes from the fact that you can allow your own custom code to also take part in ambient transactions.

Recently at work, I wanted to perform a transaction across several unrelated modules with a custom resource. Of course TransactionScope was the first solution brought up. Unfortunately, after some analysis, I realized that this wasn’t going to work for us; We have existing database code in those modules that have different conditions in which the transaction would be rolled back.

We needed a way to have multiple transaction scopes, each with different conditions of success or failure. I started thinking about how to accomplish this, and decided to write my own implementation which mimics the TransactionScope, but lets me control things a bit closer.  So I came up with a class which can be used like this:

using (Txn scope = Txn.New<MyScope>())
{
    // ... do work

    scope.Commit();
}

Notice that the usage pattern is nearly identical to the TransactionScope API … put the scope in a using statement, and call .Commit if the work was completed.  In the example above, the MyScope class is defined quite simply as:

public class MyScope : Txn
{
    protected override void OnStart()
    {
        Console.WriteLine("\tstarting");
    }

    protected override void OnCommit()
    {
        Console.WriteLine("\tcommitting");
    }

    protected override void OnRollback()
    {
        Console.WriteLine("\trolling back");
    }
}

All you have to do is inherit from the Txn class, and implement three methods: OnStart, which occurs when the transaction is first beginning; OnCommit, which is invoked only when the top-most scope exits and all sub transactions were committed successfully; And OnRollback, which as you might imagine is only called if the transaction (or a subtransaction) was not committed successfully.

One difference between this API and the regular TransactionScope is that only one instance of “MyScope” will be created when the top-most transaction is first created. As I’ve alluded to, you can nest transactions just as you can with TransactionScope. And each scope must be committed if the entire transaction is to be completed.

The Txn class can be found below:

public abstract class Txn : IDisposable
{
    private Queue<bool> committed = new Queue<bool>();

    public Txn()
    {
    }

    [ThreadStatic]
    public static bool committable = true;

    [ThreadStatic]
    public static int depth = 0;

    [ThreadStatic]
    public static Txn current;

    protected abstract void OnStart();
    protected abstract void OnCommit();
    protected abstract void OnRollback();

    public void Commit()
    {
        this.committed.Enqueue(true);
    }

    void IDisposable.Dispose()
    {
        if (committed.Count == 0 || !committed.Dequeue())
        {
            committable = false;
        }

        depth--;
        if (depth == 0)
        {
            if (committable)
            {
                this.OnCommit();
            }
            else
            {
                this.OnRollback();
            }

            current = null;
        }
    }

    public static T New<T>() where T : Txn, new()
    {
        depth++;

        if (current == null)
        {
            current = new T();
        }

        if (depth == 1)
        {
            // first transaction, assume committable
            committable = true;
            current.OnStart();
        }

        return current as T;
    }

    #region IDisposable Members

    #endregion
}

There is one caveat to mention. My requirement was to run this code in a windows service. The entire scope of the transaction would be single threaded, but there would be multiple ongoing transactions at once. To support this scenario, notice that some of the internal state of the Txn class uses the [ThreadStatic] attribute. This means that the API can be used from multiple threads at once and each thread would have its own state.

Of course, this might be a problem if you want to use this in an ASP.NET project. I’ve written about this issue before. There is probably a way to make this work using the techniques I outlined in that article, but I haven’t given it a lot of thought (because I didn’t need to). But I thought I’d share the work I did in case it is useful for anyone else.

Comments

Scurvy.Test v1.2 Released

Quick on the heals of yesterday’s post, I’ve released v1.2 of Scurvy.Test. This is the first official release of the framework and improvements over the initial announcement are mainly centered around the status reporting of test pass/failure.  I also upgraded the solution to vs 2k8 and xnags 3.1.

Here’s the changelog if you’re interested:

  • Introduced TestStatusReporter to make reporting test status easier. DefaultReporter writes to the debug output, while the test console app implements a custom ConsoleReporter that writes it out to the stdout instead.
  • Added additional methods and overloads to the Assert class for more assert options and the ability to replace your own custom TestStatusReporter instance.
  • Added XBox version of Scurvy.Test assembly
  • Upgraded solution to visual studio 2008
  • now using SVN bindings instead of TFS Explorer
  • Upgraded sample xna project to GS 3.1
  • Added custom XNA test status reporter to sample project

Comments

ScurvyTest v.next under way

I finally had a chance to do some long planned work on ScurvyTest, my flexible unit testing framework that was design from the ground up to be friendly to XNA project unit testing.

Specifically, one of the drawbacks to the previous version was that unit test status was only communicated via an easy to miss write to the debug output.  This next release will improve that by introducing an official status reporting mechanism. You will be able to replace this reporting channel with your own implementation to let you integrate with any custom game or engine.

I’m also adding more methods which will be available to you on the Assert class, and fixing a few bugs that have been reported in the exit criteria feature.

Check out the latest change set if you’re curious as to the latest, otherwise you can wait for the official release here in the next weeks :-)

http://scurvytest.codeplex.com/SourceControl/list/changesets

Comments

Resource Pool in F#

Erik Schulz, who has written a bunch of articles on how to make XNA games with F# has ported the Resource Pool class I wrote about in F#. Cool!

type FPool<'a>(newRoutine, capacity : int) =
    let queue = Queue<'a> capacity

    member this.Count = queue.Count

    member this.New() =
        if queue.Count > 0 then queue.Dequeue() else newRoutine()

    member this.Return(item) =
        queue.Enqueue(item)

I’ve been toying around with F# recently, it’s good to see an example that you can easily compare and contrast with the C# version. Has anyone else tried out F#?

Comments

XNA Resource Pool

This is likely an old topic at this point, considering the XNA framework has been out for several years. However, I meant to follow up on the topic of Resource Pooling after I posted the presentation materials from my ONETUG presentation on XNA last February. This is a simple resource pool that you can use to mitigate memory allocation issues in the XNA framework.

The canonical example of where a pool is useful, is a game entity firing bullets.  If you simply create a new instance of a bullet each time you fire, you will soon run out of memory. To avoid allocating new memory each time, you can reuse bullets that have moved off-screen or have collided with something.  There are several implementations that have been posted online such as:

These are great, they definitely do the job.  However, they have a common issue in that they want to take on too much responsibility. In both cases, the resource pool acts as an array to hold all known instances in addition to being concerned with object construction logic.

I wanted a simple resource pool which provided the functionality of providing a “new” instance of an object when I needed one. So I created my own; The basic design is that there are only two methods: “New”, and “Return”.  The new method returns a new instance that is ready to use, while the return method adds a previously obtained object back into the pool.

using System;
using System.Collections.Generic;

namespace Scurvy
{
    public class Pool<T> where T : class, new()
    {
        private Queue<T> queue;
        private Action<T> newRoutine;

        public Pool(int capacity)
        {
            this.queue = new Queue<T>(capacity);
        }

        public Pool()
        {
            this.queue = new Queue<T>();
        }

        public Pool(Action<T> newRoutine)
            : this()
        {
            this.newRoutine = newRoutine;
        }

        public int Count { get { return this.queue.Count; } }

        public T New()
        {
            T item;

            item = queue.Count > 0 ? queue.Dequeue() : new T();
            if (this.newRoutine != null) this.newRoutine(item);

            return item;
        }

        public void Return(T item)
        {
            queue.Enqueue(item);
        }
    }
}

An interesting feature of this pool is that you can control the object initialization logic by providing a simple lambda to the constructor. Any initialization logic (such as resetting “isdead” fields, or resetting positions) can be done there and it will be executed for each new instance.

I’d love to get feedback on this class if you end up using it in a project. Thanks!

Comments (2)

The Next Decade in Software

The scientists are already hard at work at coming up with predictions for the next 10 years. I thought it would be interesting to extract from the list, the predictions that are directly related to software … along with a few recommendations on how to get started today.

  • Augmented Reality
    This one is almost already a reality. A number of iPhone apps use the compass, gps, and camera as a viewport to overlay information on top of the world. A few years ago, Johnny Lee posted some amazing videos on how to use cheap consumer hardware to meld real and virtual worlds. It is clear that 3D graphics programming will be very helpful in this field … why not start learning now?
  • True artificially intelligent computer programs
    This one seems to be the most far fetched. Not because the field is not progressing, because it is, but because it’s been promised for so long. The biggest problem I think is lack of a clear goal; you can say you want something that learns, but learns what? There needs to be a catalyst that makes it clear what people really want/need from an intelligence. In my opinion, the game industry is pushing the state of the art here.
  • Cell phone apps that will act as a health monitor … Sort of an OnStar system for the body
    Since the iPhone proved to everyone that you can have a powerful computer in your pocket, new devices have started appearing that try to piggyback off of the success. I’m looking forward to more people making symbiotic peripherals for phones like the iphone, android based devices, and hopefully WinMo if they ever get their act together.  Mobile programming will surely be a skill to watch in the next decade.
  • Cloud computing
    I was a little surprised to see a reference to cloud computing in a science article. But it makes sense when you think about it, because scientists need to make increasingly complex models to prove their theories and solve problems, they will need more and more processing power to do it.

    I don’t think that it will be worth it for the average “you and me” to invest time into learning how to make cloud computing infrastructures. The big 3 (Microsoft, Google, and Amazon) will commoditize these infrastructures and make them easily available … for a price of course. Learning how to make programs that thrive in the clouds however, will prove to be useful. I’ve written some about this topic recently. You can start programming in functional languages like F#, or learn how to program shaders using HLSL to teach your brain how to think in highly parallelizable terms.

Time is always the ultimate author of history, so we will have to wait and see how many of these predictions turn out to be true.  But one thing is clear, if you learn how to make games, there is a good chance you will be successful in the next decade ;-)

Comments (1)

Cleaning up after MSTest

For those of you that use MSTest, you’ve no doubt noticed a folder called “TestResults” which can get quite large.  This is because each test run copies all of the project output into that folder to store the results of the run.

Well, I updated my little open source tool to take care of it: http://vsclean.codeplex.com

As it’s traversing the directories, it will delete any of those “TestResults” folders that it comes across.  Download now!

Comments

Snap-Circuits: Review

Completed Snap Circuit

In the months leading up to Christmas ‘09, I was pondering what to buy for my kids. Instead of the normal brainless and easy gifts like transformers, gi-joes, and barbies, I wanted to get them something that matters, something that will benefit them. As I browsed around the internet, I came across microscopes, chemistry kits, telescopes, bird watching kits. Those were all fine enough gifts … but they didn’t really jump out at me for some reason.  Until I found …

Snap-Circuits

“Elenco’s Snap Circuits makes learning electronics easy and fun! Just follow the colorful pictures in your manual and build exciting projects such as AM radios, burglar alarms, doorbells and much more! You can even play electronic games with your friends. All parts are mounted on plastic modules and snap together with ease. Enjoy hours of educational fun while learning about electronics. No tools required. Uses “AA” batteries. Not included.”

Wow, the thought that electronics can be packaged up and easily explained to an eight year old was fascinating. I found one at my local Radio Shack, picked it up, and wrapped it. So far I have to say that I am very impressed with the product and my kids both seem genuinely interested. I thought I was going to have to carefully walk them through it, but they have picked it up quickly and have started doing projects on their own.

The manual is fantastic, very easy to follow instructions, and great descriptions of what’s going on in the circuit.

The manual

Of course, the only drawback so far is that the batteries drain pretty quickly … I’ve already had to replace one set. However, the kids like it so much that I’m considering picking up one of the larger sets along with this awesome battery eliminator that lets you draw power from a wall plug.

I will definitely rate this a buy if you’ve got kids … or heck, even if you want to learn more about electronics yourself and don’t already have the background :-)

Comments

Semi-Literate Programming with C#

Recently, I’ve been reading the book Coders at Work, where author Peter Seibel interviews lots of well known developers.  One of the questions that he often asks is whether they have tried Literate Programming, an idea introduced by Donald Knuth in the 70s.  Although most of them say no, some of them have tried it briefly.  They usually come to the conclusion that it’s an interesting idea, but a lot of the tooling doesn’t really make it a realistic solution.

The author’s insistence at asking them about this got me thinking about some of the problems that I’ve encountered during my career.  Could literate programming be applied in this modern day and age to help solve some of the problems that we all face?

A Use Case

I started thinking about the types of code where these questions tend to come up with. It’s usually around code where business analysts make up some business rules.  Let’s use an example.  Say you work at a company that needs to creating billing statements for clients.  When you process the statements, there are a series of fees which you must attach to the bill based on certain conditions.

The rules may have been explained to the developer in a face to face meeting like this:

  • Apply a 7% tax on the principal when the client is in Florida
  • A $50 flat fee will be applied when a New York based client maintains a principal of more than $1000

So I, as a developer will scuttle away and create the program per specifications. The application goes to production and everyone is happy. Two years pass by, most of the original staff that was working when the system was originally deployed has turned over, and the new staff has a question about why some fees are being charged for a given client.

In an ideal world, the business will refer back to documentation that they created when wanting to know about the behavior of some system that you programmed. However, in the real world, a more likely scenario is that they will end up asking you over the phone about some obscure section of the code and you end up having to crack open the source to figure out what the code is doing in that piece.

The Solution?

I want a solution that lets me write code, and without manual intervention, allow other developers and the end user to understand what the business logic is doing.  Of course, most people will point to the XML documentation feature of C# along with auto-documentation products like SandCastle, and suggest that this is enough.  However, maintaining XML comments violates the “without manual intervention” part of my own requirements. It also creates output that is not really consumable by end users.

There is also another issue that most people probably don’t really think about. A lot of the code in today’s projects is not really … useful, to document. Serialization code, parsing code, data access code … most of that is pretty standard.  Developers will easily understand it assuming they already know how to use the APIs like ADO.NET, and WCF.  And users won’t care about it.  So that really just leaves the fundamental logic that is the raison d’être for your application in the first place. This is what I am interested in making easily available for a human to read.

For the solution, I wrote a simple Rule class:

public class Rule<T>
{
    private Expression<Action<T>> expression;
    private Expression<Func<T, bool>> evalExpression;
    private Action<T> compiled;
    private Func<T, bool> evalCompiled;

    public void Execute(T context)
    {
        if (this.evalCompiled(context))
        {
            this.compiled(context);
        }
    }

    public Expression<Func<T, bool>> Evaluation
    {
        get { return this.evalExpression; }
        set
        {
            this.evalExpression = value;
            this.evalCompiled = value.Compile();
        }
    }

    public Expression<Action<T>> Action
    {
        get { return this.expression; }
        set
        {
            this.expression = value;
            this.compiled = value.Compile();
        }
    }

    public override string ToString()
    {
        return string.Format("if {0} then {1}",
                                    this.Evaluation.Body,
                                    this.Action.Body);
    }
}

This class takes, as a generic parameter a context which represents one item that needs to be processed.  You will set two lambda expressions: the Evaluation, and the Action.  The evaluation will return true if the action is to be applied.  An example can be seen below:

List<Rule<BizContext>> rules = new List<Rule<BizContext>>();

rules.Add(new Rule<BizContext>()
{
    Evaluation = c => c.State == Florida,
    Action = c => c.Fees.Add(c.Principal * .07M)
});

rules.Add(new Rule<BizContext>()
{
    Evaluation = c => c.State == NewYork && c.Principal > 1000.00M,
    Action = c => c.Fees.Add(50.00M)
});

“BizContext” in the above code can contain anything that pertains to the item that needs to be processed.  In our case, the analyst’s rules say that we need to operate based on the principal and client’s state, and add fees.  So those are the properties that the context contains. Because the rules were added to a list, you can iterate through the list and call the rule class’ “Execute” method.

foreach (var rule in rules)
{
    rule.Execute(context);
}

So far, there’s nothing groundbreaking about the Rule class. I’m sure many of you have written something similar time and time again. But here’s where the literate programming comes into play. Because the “Evaluation” and “Action” methods are actually Expressions … we have access to the textual representations of the code, in addition to having the ability to execute it.

The overridden .ToString method on the class will output an easy to understand string of the business rules using actual code that will execute when it’s run. So for the two rules defined above, you can get a printout like this:

if (c.State = Florida) then c.Fees.Add((c.Principal * 0.32))
if ((c.State = NewYork) && (c.Principal > 1000.00)) then c.Fees.Add(50.00)

The end user gets a realistic printout of the actual business logic in the system on-demand; And the developer doesn’t have to do anything to update this when the business logic changes.

So there you have it; I wouldn’t exactly call it full fledged literate programming with C# in the way that it was described originally.  But I think that it embodies the qualities of literate programming, where documentation and code are one and the same. It’s a compromise, and it would be interesting to see if this approach can be implemented in a real-world scenario.

Any takers? :-)

Comments (1)

Qizmt: MapReduce Framework in C#

I was recently surprised to find that MySpace had open sourced a distributed “MapReduce Framework” called Qizmt (http://qizmt.myspace.com/).  From the site’s description:

MySpace Qizmt [kiz-mit] is a mapreduce framework for both developing and executing distributed computing applications on large clusters of Windows servers.

This has been a topic that I’ve been interested in for a while so I’m glad to see that someone has been making progress in this space for the .NET world.  Dryad remains an interesting prospect which is apparently even seen production use in Microsoft’s ad service, however it’s clear that it hasn’t been “productionized” yet.

One interesting aspect of Qizmt is that it seems they paid a lot of attention to making it easy to deploy.  In my opinion, the ability for a developer to easily get started with a one machine install is a key enabler for the uptake of any new technology.  Hopefully with more competition for technologies such as this, we will see some cool options become available to us as developers for high performance computing.

Comments