jordan.terrell
Just trying to make sense of things...

A New Testing Framework

Friday, 7 November 2008 10:23 by jordan.terrell

A new friend of mine wants to collaborate on building a new .NET based testing framework.  I know, I know, there are plenty of testing frameworks out there.  Just to name a few: NUnit, MbUnit, xUnit.NET, nSpec, etc.  Each of these has their own strengths, weaknesses, and implicit/explicit testing methodologies associated with them.  So why would I bother saturating the market further with another testing framework?

I think I have a semi-unique idea.  I say “semi”, because the underlying design idea is applied in other technology areas, but it would be unique in its application to testing.  I think it’s has enough of a fundamental difference from all of the existing testing frameworks to justify building it.  A by-product of the design would mean that it would be mostly compatible with existing testing frameworks (and in fact I would try hard to maintain that quality, but not in all scenarios), but would also open up avenues for experimenting with new ways to represent tests.

So, the reason for this post (besides encouraging a barrage of people telling me not to put yet-another-testing-framework out there) – what do you think is missing from testing frameworks today that you would like to see in the future?

Comments (4) -

November 7. 2008 10:53

Bryan

Instead of saying, "we don't need ANOTHER test framework", I'll ask, why not try to contribute the ideas to an existing framework?  It's hard to say, since for obvious reasons you are keeping quiet your secret weapon.

Maybe what you should ask is, along with your new functionality, which features should be ripped off whole-sale from the other frameworks.  I like Nunits Assert model, but really like MbUnits generative testing (RowTest and TypeFixture specifically).

I personally probably barely scratch the surface of what these frameworks do.  And MbUnit 3 looks pretty crazy with the features they are adding.  So, I can't really confidently say that these other frameworks are missing features, when I only have experience with two of them, and I tend to use the same features over and over.

It will probably be hard to pry me away from my MbUnit, but I welcome you to try ;)

Bryan

November 7. 2008 11:54

Jordan

Normally, I would agree with you Bryan - to contribute to an existing framework.  But the idea I have would equate to a complete rewrite for all of the existing frameworks - something I seriously doubt that the communities around these frameworks would appreciate!

As for the NUnit and MbUnit features you mentioned, if this framework actually happens, I'm planning on supporting those features, along with some addtional ones that I've not seen in any framework.  I want to maintain an opt-in approach to all of the features, so that developers from all the different frameworks can use what they want.

Good suggestion though - I would like to hear of any features that people DO NOT want to use.  That way I can make sure that the opt-in approach applies to those features as well.

Jordan

November 7. 2008 14:36

Dilip

I have an opposite question -- what is missing from the current testing frameworks that is driving you to build a new one?  Is there going to be some major philosophical break-away from the way unit tests are being conducted today?

Dilip

November 7. 2008 15:34

Jordan

Answering that would give away too much of my "secret weapon", as Bryan called it.

Jordan

Comments are closed