thinking-out-loud

Continuous Improvement

One of our instructors is pursuing a PHD in learning theory.  Recently we talked about the process that people and organizations go through when learning new things or addopting new technology.  It basically looks like this:

It seems to me that the trick to avoiding a deep and sometime insurmountable productivity decline is to keep up-to-date – be always improving.  Then your steps to learning new technology are smaller and less risky. When an organization goes too long without improving, that trough can get big enough to cause frustration, project failure, even loss of staff.

At Improving, we help minimize the down side and maximize the upside by leading people through the adoption of technology and techniques – teaching them quickly, avoiding common pitfalls, and giving lessons from our experience so they don’t have to learn from their own trial and error.  And finally, by fitting all of the technologies and best practices together into an integrated solution delivery capability, we make individuals and organizations higher yield than ever before.

More: Agile vs. Formal

Greg Vaughn piped in on the Agile vs. Formal question I had yesterday, suggesting 'delayed validation' as a contrast to agile (borrowed from Alan Francis.) Alan links to an image on wikpedia showing The SLDC (as opposed to a SLDC), and how things get validated -- and it is a beautiful, beautiful diagram.

Formal vs Agile?

I have been reviewing the MSF lately, and it would seem that the MSF distinguishes non-Agile processes as "formal" processes. I am wondering out loud whether that is a good choice of terms? It seems to imply, by comparison that an Agile process is an informal approach to software development. If that is an intended distinction, then I think that the MSF is missing the point.

Stealing the following definitions from answers.com (via google, tyvm),

1. 1. Relating to or involving outward form or structure.
   2. Being or relating to essential form or constitution: a formal principle.
2. 1. Following or being in accord with accepted forms, conventions, or regulations: had little formal education; went to a formal party.
   2. Executed, carried out, or done in proper or regular form: a formal reprimand; a formal document.
3. 1. Characterized by strict or meticulous observation of forms; methodical: very formal in their business transactions.
   2. Stiffly ceremonious: a formal manner; a formal greeting; a formal bow to the monarch.
4. Having the outward appearance but lacking in substance: a formal requirement that is usually ignored

I think that 1, 2, and 3a can all apply equally well to Agile methods. You might agree if you have ever:

• had a scrum master that calls you into line for straying from the daily scrum format
• executed (or tried to execute) test-driven design
• refactored in the way that Martin Fowler suggests -- baby steps, always getting back to a working system as soon as possible.

I cynically think of waterfall as 3b or 4.

If I were trying to distinguish, I would choose agile vs. rigid.

What's the difference with regression testing?

Many winters ago I was a tester, and then a maintenance developer doing regression testing.  While maintaining a printer driver which produced binary printer data, we used a crafty technique (at least it seemed so then) to test when some fix or enhancement we made to the driver caused adverse effects to the output.  That technique I’ll call difference testing.  Here’s how it worked:

• Run a bunch of documents through the printer driver and capture the output
• Make a change to the driver
• Run the same bunch of documents through the printer driver and test each output against the previous output from the same document
• When a difference is detected, visually compare the printed output to determine if it’s a good change or a bad change
• And, (this is important) update the baseline output when the difference is good

I’ve recently had the opportunity to use this technique again, regaining an appreciation for its effectiveness.  I’m just wrapping up the development of a pricing engine that can accept product orders in an XML format and output the same format with prices attached to each line item and sub-item.    The testing challenge here is to check the dazzling number of combinations of options and the interplay of business rules applied to those combinations via the prices they produce.  So, here’s what we did:

• Order a bunch of products with specific combinations of options designed to test edge cases of pricing rules, capturing the XML containing product orders and prices as baseline data
• Make a change to the pricing engine
• Iterate over the previously captured XML product orders and generate pricing again, comparing each file output to the corresponding baseline output
• When a difference is detected, visually or programatically compare the XML elements
• And, (this is important) update the baseline output when the difference is good
• Also, as with any regression suite, add new test cases to the suite when a previously untested error condition is discovered

I originally planned to parse XML elements and spit out some specific error statements about the differences.  But, my client suggested just comparing the files with a visual diff, thus tossing the data in question up in front of the user with all the context required to determine the real error.  Way to leverage the readability of XML!

Feel free to use this technique anytime – no charge.  Oh, and make sure the XML is generated with <new lines> or your diff will just show one long line in error.

Testing VB6 with NUnit

We at Improving are huge fans of test-driven development (TDD).  I just started a short project for a client who requires the development (for very valid reasons) to be done in VB6.  I did a little research around TDD in VB6 and found vbUnit as well as some other tools.  However, months ago I'd heard about the rare and mysterious practice of unit testing VB6 code from NUnit.  Only, I couldn't find a single reference to it with Google.  So, here's what I learned from one of my wife's coworkers....

1. Create your VB6 code in an ActiveX DLL project (e.g. Project).  Add and develop classes as normal in this project.

2. Use menu item File|Make DLL to copile the DLL (e.g. Project.dll) and automatically register it for use.

3. In Microsoft Visual Studio .Net (or 2003), create a DLL solution and add references to the .Net assembly NUnit.Framework and the COM library produced by VB6 (e.g. Project).  VS.Net auto-gens a COM wrapper for you (e.g. Interop.Project.dll) in the bin directory.

4. Write .Net test cases to invoke member functions of classes in Project. You can make a using Project; statement if desired.  Build your DLL as normal.

5. Execute tests by FIRST running the ActiveX DLL from the VB6 environment.  This puts VB6 in debug mode.  Set a breakpoint somewhere to prove that it's working.  SECOND: launch NUnit, open your .Net assembly, and run some tests. 

NOTE: If you don't run the VB DLL first, NUnit will report a System.Runtime.InteropServices.COMException. 

Of course, once you're up and running, you'll want to change the order of these steps so you're actually writing your tests first. ;)

I hope to have NO other words of wisdom about hard to find answers to VB6 development questions from this project.  But, it is worth mentioning that www.connectionstrings.com had the answer I needed for connecting to a password-protected database with ADO.

.NET TDD Resources

I was doing a little browsing and ran across several items of interest.

First, let me join the voice of many who love Lutz Roeder's .NET Reflector. This tool absolutely rocks. I wish it were a visual studio add-in replacement for the "Go To Definition" and "Go To Reference" options, because it is sooo much nicer.

There is an add-in (yes, Reflector has an add-in api!) by Jonathan de Halleux called Reflector.Graph. You can see some snapshots of what it can do on his blog. His blog is excellent reading material. Apparently, Microsoft took notice, because they recently hired Jonathan.

Advanced Unit Testing is a collection of tools, practices, and patterns for unit testing, especially focused on .NET. Core to AUT is the MBUnit testing framework by (again) Jonathan de Halleux. Jonathan also has a production grammar framework, some graphing libraries (implemented on top of AT&T Research's GraphViz), unit test generation tools, and more. Very, very good stuff.

Marc Clifton created AUT, and he has a number of articles on Unit Testing over at Code Project. Good stuff as well.

Finally, Roy Osherove has an interesting article that extends some ideas ideas presented in James Newkirk, et al.'s Testing in Microsoft.NET to leverage Enterprise Services in database testing to rollback any changes that are the result of the test.

In my experience, even in organizations that are aware or actively trying to adopt test-driven development or *at* *least* implement decent regression test suites for their applications, there is a constant struggle to develop the tests. In my experience, the amount of test code necessary to test a component well is 2-3 times the amount of code necessary to implement the functionality. it is good to see so much activity in this area in terms of helping people understand best practices and providing tools to automate some of the more redundant bits can help narrow the gap. Of course, tools alone won't necessarily make the job easier. You must properly apply them. But all of the infrastructure certainly lowers the barrier to entry to increasing the quality of code.

I was talking to a one-time cohort about TDD a while back, and he influenced how I think about it. He basically said (paraphrased) that TDD is not so much a testing method or a coding method as it is a *design* method.

I hope you had a nice weekend.

Obtaining Consensus

Jared M. Spool of User Interface Engineering Fame published a new article today on the K-J technique, a prioritization/consensus technique.

Some of the biggest hurdles to enterprise architecture in general and service-oriented architecture in particular are gaining buy-in and establishing priorities to scope incremental releases (Down with Big Bang). As you adopt a service-oriented architecture, if you start to recognize that you are struggling with priorities or consensus, try it out. Heck, try it out even if you don't recognize that you are struggling.