While preparing code for my talk at the upcoming Dallas TechFest 2011, and writing code samples to demonstrate how exceptions work with async and await, I ran across some nastiness that I really don’t think should be working, or in this case… breaking, the way it is in the refresh of the Async CTP. Since I could not find any mention of this issue on any website or forum (I’m not saying they don’t exist), I am going to show you it here.

Without any further ado, exceptions… inside an async method. Exceptions are supposed to propagate to the caller who is awaiting the completion of an async call. But… take this code for example:

public void Run()
{
	try
	{
		MyVoidAsyncMethod(null);

		Console.WriteLine("Press a key...");
		Console.ReadLine(); // <--- BIG problem!
	}
	catch (Exception ex_)
	{
		Console.WriteLine(ex_.Message);
	}
	Console.WriteLine("You'll never see this!");
}

private async void MyVoidAsyncMethod(string betterNotBeNull)
{
	if (string.IsNullOrEmpty(betterNotBeNull))
		throw new ArgumentNullException("That's why we can't have nice things!");

	await TaskEx.Yield();
}

Here’s what you get if you execute it:

Press a key…

Exception rethrown at [0]:
at System.Runtime.CompilerServices.AsyncVoidMethodBuilder.b__1(Object state)
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()
Press any key to continue . . .

The UI thread is blocked by the Console.ReadLine() and that totally hoses up the entire system… and it crashes the application immediately. This is an edge case and is definitely one of those, “Doctor, it hurts when I do this. Well, don’t do that!” situations. You probably shouldn’t be blocking the UI thread that called the async method anyway… so say’th the peanut gallery… and I agree.

However, it is inside a try/catch which “appears” to be completely ignored. This all has to do with the way the compiler generates the code for an async method. The first thing it is contractually obligated to do is to create an AsynVoidMethodBuilder object… and it does as you can see from the exception above. But since this is an ‘async voic’ method, it sort of freaks out… which I believe (ultimately) it shouldn’t.

Now, if you replace the async method with one that returns a Task<> like this:

private async Task<int> MyTaskAsyncMethod(string betterNotBeNull)
{
	if (string.IsNullOrEmpty(betterNotBeNull))
		throw new ArgumentNullException("That's why we can't have nice things!");

	await TaskEx.Yield();

	return 42;
}

Then… all you get for output is:

Press a key…

Moral? Not sure… maybe it’s don’t block your UI?! Maybe it’s just a hole that will be filled shortly by the C# compiler team before C# 5 ships. It’s worth noting at least. You’ve been warned!

Ah, that’s just the way it goes sometimes!

Vyew is 100% free as long as you’re ok with advertisements and a somewhat limited account. Still, the free account seems to have more power than the average Joe normally needs. Should you find yourself not one of the average Joes, you can opt to pay for a “bigger” account with no advertising. If you take this route, you’ll still pay 80% less than WebEx or GoToMeeting.

The only caveat is that it is not “live” like WebEx or GoToMeeting… meaning you can share but you can’t see the presenter’s screen in real-time fashion. That may be a deal breaker for you.

Give them a shot and let me know what you think!

	// Call the AddException method on the _importer object.
	// The 2nd parameter to 'Call' is the parameter list to pass to AddException
	// The 3rd parameter to 'Call' is the types of the parameters expected by the AddException method
	_importer.Call("AddException", new object[] { new Exception("blah") }, new [] { typeof(Exception) });

You can see that the parameter list is an object[] so the Type.GetMethod method has no way of differentiating between two methods with the same name solely by looking at the parameters passed in. You have to specifically tell it, “Hey, I want you to call the ‘AddException’ method that expects an ‘Exception’ parameter.

Some might say, why not just create a phat testing base class instead of creating these extension methods. You could do that. However, I personally feel that this is slightly cleaner and a bit more readable. There are times when a testing base class comes in quite handy and I pretty much always use one… for a completely different reason though.

Let’s take a traditional look at how to write a test where you expect an exception to be thrown.

[Test, ExpectedException(typeof(ApplicationException))]
public void Demonstrate_NUnitWayOfExpectingExceptions()
{
	// This 'throw' will cause the test to pass because this test is
	// decorated above with the 'ExpectedException attribute.
	throw new ApplicationException(exceptionMessage);
}

If you do it this way, there’s no (decent) way to look at the thrown exception or do anything beyond where the exception is thrown. That’s not exactly ‘ideal’ when more than one type of exception could be thrown or if you want to do multiple things in the same method after the exception is thrown. Now let’s look at a slightly better way to do just that.

If you downloaded the source code in Part I, you’ll find a class named TestingBase.

[Test]
public void Demonstrate_ExpectException()
{
	// ExpectException (on base class) is great when you expect an exception but
	// ... the ExpectedException attribute just doesn't give you enough flexibility
	string exMessage = ExpectException<ApplicationException>(() => { throw new ApplicationException(exceptionMessage); });
	Assert.IsNotNull(exMessage);
	Assert.AreEqual(exMessage, exceptionMessage);

	// We're also free to continue more testing and still be in the same 'state'
	// Here we can write several lines of code 'inline'... no problem.
	exMessage = ExpectException<FileNotFoundException>
		(() =>{
				using (var reader = new StreamReader("this is not a valid file path"))
					reader.ReadToEnd();
			});
	Assert.IsNotNull(exMessage);
	Assert.IsTrue(exMessage.Contains("Could not find file"));
}

In the test above, we are calling a base class method called ExpectException which, as you can see, expects an exception of type ApplicationException to be thrown. ExpectException takes an Action and will execute the action and swallow any thrown exceptions, validate an exception is actually thrown, and ensure that the thrown exception is of the specified type. It also returns the message inside the exception so just in case you have more than one code branch that throws the same type of exception but has a different message… you’re able to validate that as well as is done in the code above.

Using the ExpectException method gives you more flexibility and to continue executing code in the test. There are times when you may need to write a test that calls a method twice in order to have it throw an exception. The ExtractId method in the Part I post has logic to ensure that the Person Id is only added once so that no two people can have the same Id. That is an example of a case where you would need to call that method twice. Now, ExtractId doesn’t throw an exception on the first call but it’s just an example and if it did, you could use ExpectException in that situation to make quick work of that test.

If you don’t want to bother with checking the returned exception string, you can use the other method on that base class called ExceptionWithMsg. Here’s an example of how to use it:

// ExceptionWithMsg is "implemented in terms of" ExpectException... meaning it
// just calls ExpectException and then verifies that the exception message contains
// the textual fragment passed in.
ExceptionWithMsg<FileNotFoundException>
	(() =>{
			using (var reader = new StreamReader("this is not a valid file path"))
				reader.ReadToEnd();
		},
		"Could not find file"
	);

ExceptionWithMsg just calls ExpectException and tests the message for you… as you can see from the implementation:

protected void ExceptionWithMsg<ExceptionType>(Action action_, string exceptionMessageFragment_)
{
	Assert.IsTrue(ExpectException<ExceptionType>(action_).Contains(exceptionMessageFragment_));
}

Enjoy!

That said, there are many ways to judge code quality. The one I would like to single out today is “Code Coverage”. That is defined as the percentage of a given source code base that has been executed. For example, if I have a source code base with 100,000 lines of source code in it and 95,000 of them are verified (by a tool) to have been called, then we have 95% code coverage. If you can’t wait and want to know right this second, in this post I will be using NUnit for unit testing, NCover for code coverage, and Hudson for continuous integration. If you want to learn more about those technologies, I highly recommend DimeCasts.Net, Rob Connery’s TekPub, and Channel 9.

Now, even the best developers in the world miss things. We write a few unit tests, or better yet we’re doing test-driven development (TDD), and we’re writing our test first. We feel good about what we’re doing and we feel confident our code is being sufficiently test. That is, of course, until we run a code coverage tool through our source code’s proverbial hair.

Let’s talk about “doing things right” for a second before we dive into testing and code coverage. Organization of a software project is vitally important and is best to setup the right way the first time instead of having to go back and try to fix it later. For a while, I’ve been laying out (roughly) our Visual Studio solutions in this fashion:

“So what!?” you might be saying. I hear ya but stay with me as I’ll get to why I even bring this up. If you use this, or a similar, layout, no matter what size project you’re working on, it will almost always have the same organizational flow. The real reason I bring this up is because this kind of organization lends itself to a little thing called “loose coupling”. Loose coupling is where the components (or anything for that matter) are physically decoupled. In this context, it means that we have several (or many) different class libraries (DLLs) that contain code specific to a given development context. What this means in lay terms is that your TCP communications code is in a separate DLL than your logging code… i.e. they are physically “decoupled”.

Again with the “so what!?”. Well, there is an important distinction when it comes to “visibility” and when a source code base lives in many different assemblies/modules/DLLs/et cetera. What this means is that, assuming you’re a “good” programmer and you’re using private and protected accessibility modifiers instead of blindly marking everything as public, then those private and protected methods and data members are not (normally) visible outside of the assembly within which they reside. Since we’ve chosen the (good) path of loose coupling, and we want to have the highest percentage of code coverage as possible, then we are going to have to test (using unit tests) as many of the source code branches/paths as possible.

Last week, I had a developer tell me, “I put all my tests in the same assembly as the code it’s testing, that way I can use the ‘internal’ access modifier and test everything from there.” As my friend and muse Jef Newsom would say, “Well, that’s one way to do it.” Of course, the rest of us who are tact-challenged would just say you’re an idiot and we’d really like to slap you for saying something ridiculous like that. Let’s just leave it at “that’s a horrible idea and just don’t do it that way” so we can continue.

Now that we’ve got tests in a completely different assembly than the source code those tests are testing, how do we ensure we hit every single code path? That’s tough if we’re doing interface-based programming and we’re attempting to test solely through an interface. However, with some “cool kid” testing extension methods, we can very easily hit our mark.

Before we continue, grab ALL the source code and code coverage files we’ll be discussing by clicking this line.
NOTE: This is a Visual Studio 2010 solution and uses C# 4 and thus the .Net framework 4.0.

The demo source code is complete and is not some frivolous “test code” that is not reusable. The business problem this (demo) code is supposed to solve is to be able to parse a data file with fixed-length records (like old COBOL record) and create domain objects from the data in the file. The code should keep track of when it runs into “bad” data but continue to process the entire data file.

I wrote most of this code using TDD and somewhat decent “best practices” all around. When I fired up NCover to see how much code had actually been tested by my unit tests, I was woefully disappointed to see the following:

View the first NCover Report HTML by clicking this link

What?! Are you kidding me? But I tested the entire interface! Or did I? Yes, I had been working for 14+ hours at the time so what could I have missed?! What I had done was test all the edge cases to ensure I got all those tricky cases… but I complete forgot to write a test with valid data (doh!). After I added the normal tests, it improved somewhat to the following:

View the second NCover Report HTML by clicking this link

That is certainly better but I’m still missing 10.44% of the code branches. NCover will actually show you exactly what lines of your code are not being called. Here’s a view of the NCover report at the source code level (red = not covered, green = covered):

As you can see, some private members’ code paths are not being called. So, how do we ensure those code paths are called? We could modify the data file to force those branches to be called. Or, we could call those methods directly. “The hell you say!” That other developer we heard from earlier clearly told us it is not possible to call protected and private methods and such… ah, but he really just wasn’t thinking deeply enough… which (FINALLY!) brings us to what I wanted to talk about in the first place… better testing through some really cool extension methods!

/* MOST OF THE METHODS LEFT OUT IN AN UNSUCCESSFUL ATTEMPT AT BREVITY FOR THIS BLOG POST */

public static object Call<T>(this T this_, string methodName_, object[] parameters_, Type[] paramTypes_ = null)
{
	const BindingFlags eFlags = BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic;

	MethodInfo methodInfo = GetMethodInfo(this_, methodName_, eFlags, paramTypes_);

	return InvokeMethod(this_, methodInfo, parameters_);
}

private static object InvokeMethod<T>(T this_, MethodInfo methodInfo, object[] parameters_)
{
	// There is currently a "defect" in .NET 4 where a TargetInvocationException is thrown
	// ... when we don't want it to be throw.  Easiest fix to this problem (for now) is to
	// just catch the exception and rethrow the inner exception which is the exception we
	// are interested in anyway.
	try
	{
		return methodInfo.Invoke(this_, parameters_);
	}
	catch (TargetInvocationException ex)
	{
		throw ex.InnerException;
	}
}

private static MethodInfo GetMethodInfo<T>(T this_, string methodName_, BindingFlags flags_, Type[] paramTypes_ = null)
{
	MethodInfo methodInfo;
	if (paramTypes_ == null)
		methodInfo = this_.GetType().GetMethod(methodName_, flags_);
	else
		methodInfo = this_.GetType().GetMethod(methodName_, flags_, Type.DefaultBinder, CallingConventions.Any, paramTypes_, null);

	if (methodInfo == null)
		throw new MethodNotFoundException(methodName_);

	return methodInfo;
}

These methods allow us to call private methods, set private fields, call private static methods, on so on. Armed with these new tools, we can now knock out those missing tested branches. Here’s a sample of how to use these methods:

[SetUp]
public void SetUp()
{
	_importer = new PersonDataFileImporter();
	_person = new Person();
}

[Test]
public void AddingTheSameIdMoreThanOnce_ExtractIdAddsExceptionToList()
{
	// The ExtractId method needs _currentLine to already be initialized.
	_importer.Set("_currentLine", 1);

	var exceptions = _importer.Get("_exceptions") as IList<Exception>;
	Assert.IsNotNull(exceptions);
	Assert.AreEqual(0, exceptions.Count);

	// Pass a negative value in for the Id
	_importer.Call("ExtractId", new object[] { "1234", _person });
	Assert.AreEqual(0, exceptions.Count);

	// Now pass the same data to ExtractId again...
	_importer.Call("ExtractId", new object[] { "1234", _person });
	Assert.AreEqual(1, exceptions.Count);
	Assert.IsTrue(exceptions[0].Message.Contains("already been used"));
}

The first thing this test does is set a private data member called _currentLine on the _importer instance to the value of 1. Then it gets a reference to the private data member called _exceptions on the same _importer instance. We then make back-to-back calls to the private method called ExtractId with the same Id value of “1234″. This ensures an exception is added because we can’t two people with the same Id in the same import run. Nifty huh!

And after creating more tests to fully test all the branches of our code, we can see the final outcome of our code coverage report:

View the final NCover Report HTML by clicking this link

But wait, why are we only seeing 96.55% branch coverage now?

If you drill down to the offending methods in the NCover report you will see that all of the branches are indeed covered… so this is a mystery.

Well, I contacted NCover and had a nice long “Go To Meeting” conference call with Bryn Brutosky and David Williams (great guys who were extremely helpful and wanted nothing more than to get these problems resolved). For now, they are looking into it and are going to get back with me soon. For now, we’ll just chalk it up and a minor defect that will be fixed soon. It could be NUnit or .Net 4 that is throwing them off. It’s all good though.

There is a TestingBase class in the solution and I’ll save that and the Hudson discussion for our next blog post as I have drug you through the mud enough for today!

We all “half ass it” when we are working on pet projects of our own. You know you do it! You cut corners, write code you know you shouldn’t write just to “test something out”. No worries. I’m not saying you’re a bad person. I’ve done it and I still do it at times. However, the older and (hopefully) wiser I get, the more I come to find value and comfort in doing it right the first time instead of having to “fix it later”.

Subversion and Git (currently) are two of the major “revision control systems” in use by software developers. This is not about which is better as that is (usually) a very subjective dialog. This is more about how to get your precious source code to a safe place so you can rest soundly.

Project Locker (http://www.projectlocker.com/) is a great place to start (and possibly stay). They offer hosting for both Subversion and Git and you can use either one you choose on a project by project basis. And if you’re really into being one of the geeky cool kids, they also offer continuous integration, defect tracking, team collaboration, and document management. For FREE, you can get an account with three users, 500MB of space, and three projects. Not bad for free huh!

I’ve been using them for a few years and have absolutely nothing but great things to say about them. Check them out. You won’t regret it.