AppSettings and Visual Studio Test System

Apparently I'm in tester mode these days.  Today I spent the entire day figuring out how to do proper unit testing with Visual Studio 2005's new Test System.  It's integrated with the Visual Studio Team Edition, which my client is using for a variety of small teams where every developer plays just about every role.  Here's a quick overview of the unit testing features and a webcast of some MS dudes pitching test-driven development (TDD) in VS 2005 (more on that soon).

The big buggaboo that consumed my day is the little fact that when a test is executed, the app doesn't run in the bin\Debug directory like normal, but rather in the ~\Local Settings\Application Data\VSEqtDeploymentRoot where ~ is the user's home directory, generally under C:\Documents and Settings\<user name>.  You can change that test run path in VS, but you won't want to point it to your solution folder because it copies all the runtime assemblies into a temp folder there.  Sure, it sounds like a good thing to isolate your tests, but what about config files???

Now, I already had unit tests built for NUnit and found this crafty tool to do  conversions.  But I didn't use it since there were just a few tests so far and I wanted to ingrain all the differences in my fragile little mind.  However, right off the bat I realized that you can't just convert any ole' class library into a VS test library.  First, you have to create a test project from scratch, and then write tests or copy your existing ones in.  (The instructions on the tool just mentioned indicated the same thing.)  I spent a good while trying to hack the solution and project files to no avail. 

Also, in my June or July release of VS 2005, you can't just select Add > New Test from the solution's context menu, either, because it will create a VB.Net project <blech> (unless that's what you really want, of course).  Instead you have to choose Test > New Test from the main menu to get a choice of languages and the option to use the Unit Test Wizard.  The wizard works after you get past it's penchant for reverting back to VB.  I accidentally create a VB test project despite my initial choice of C# and had to delete and re-create, so doulbe check!  It would have been a pain if under source control instead of in my SandBox solution.

So, I finally got a C# test project, copied my NUnit test into it, and made the minor syntax changes to attribution (TestFixture becomes TestClass, Setup becomes TestInitialize, TearDown becomes TestCleanup, and Test become TestMethod - I like these better).  I ran the tests using TestDriven.net's handy context menu option and everything worked.  Then I got ambitious.  Let's use the built in test execution method like they did in the wabcast!

It's actually pretty simple: you just set the test project as the startup project, and run it (F5).  Hmmmm....but every test broke.  And I just ran them with TestDriven.net!  What gives? 

The cool thing about running tests with the Test Manager built into the IDE is that they run in Debug mode (or whatever your configuration setting is).  That means you can click a break-point, run, and inspect your variables.  For me, that inspection showed that no AppSettings were being retrieved properly.  I soon discovered that tests were running in a whole new place, as previously stated.  The DLLs were being copied, but .config files weren't.  Hmmm...what to do?  To shorten an already long story.... I eventual found this one single beautiful blog post about copying config files in regard to NUnit and TestDriven.net.  Thank you Loren Halvorson!

So here's the deal:

1. In each test project, create an App.config file with your AppSettings you'd normally put in Web.config or your main executable's config file.
2. In your test project's properties pane, select Build Event and add this line to your post-build command line:  copy /Y $(ProjectDir)app.config $(TargetPath).config
3. Run your tests (F5).
4. See the pretty status list and the red and green result indicators. Yay!

My colleauge and I spent quite a bit of time trying to make this config-copying work for other class libraries.  No dice.  The build-event will run on other projects, but the VS Test System will only copy the contents of the bin\Debug directory for the selected test project to the test environment, not any config files from pre-requisite projects. 

The down-side having an App.config for each test project is that AppSettings now have to be maintained in multiple config files.  While hunting for solutions we came across a post by Paul Wilson about including AppSettings from one config file to another.  However, since other config files from other projects don't get copied to the test path, you lose your relative path to them and the inclusion breaks.  Also, only <appSettings/> has a "file" attribute, not <connectionStrings/> as is now en vogue.  You have to put everything in every config file, and you get to use the new ConfigurationManager in all of it's glory.  Be sure to add the System.Configuration.dll reference to your project!

Comments

[Jason C]

In your #2 item above, the post-build command text should be this: copy /Y $(ProjectDir)app.config $(TargetPath).config

[Todd Girvin]

Thanks, Jason. I don't know how my original post got so non-functional. But I've corrected it with your feedback.

[Todd Girvin]

I found another related blog entry that says you just have to name your config file properly. I'll try it out when I have a chance and confirm. http://blog.u2u.info/DottextWeb/peter/archive/2005/07/13/6179.aspx

[Gerard]

This is all good stuff but what is needed is on top of this is: 1. The ability to add build events to web based projects in VS2005. There is no concept of a project any more for VS 2005 web projects (both web clients and web services). 2. The ability to generate the config file for each user from an XML document (for example). This is a simple find/replace dependent on environment. Some inheritance in the data would be good as different users may have the same environemnt topology for some solutions. I wish a SourceForge or GotDotNet project would handle this. Maybe there already is one. Anyone know ???

[Todd Girvin]

Gerard, there is already a Web Application Project plug-in being developed here: http://webproject.scottgu.com/

[Jim Lesch]

Problem: Unit tests fail when they depend upon a config file for another project in your solution. Proposed Solution #1: In each test project, create an App.config file with your AppSettings you'd normally put in Web.config or your main executable's config file. In your test project's properties pane, select Build Event and add this line to your post-build command line: copy /Y $(ProjectDir)app.config $(TargetPath).config Problem with solution #1: Not intuitive when you have a web.config format and transferring it to an app.config format. You also have 2 files to maintain when there is a change. Proposed Solution #2: Copy the config file into your test project output (bin) directory and rename it to YourTestAssembly.dll.config. Now you get all the config settings and you also get it copied automatically for you. Problem with solution #2: You still have to maintain 2 config files! Solution #3: Each time you build a Pre-Build event copies the master config file to the Test ouput dir and renames it using the .dll.config convention. On the deploy, you get an up to date config file but you only have to change it in one place – the source project. This is the full line in the Pre-Build event command line: copy $(SolutionDir)\Web16\web.config $(TargetDir)\$(TargetFileName).config Problem with this solution: The source project and config file name are hardcoded. You could use some kind of naming convention to make it work. it also assumes you have a 1-1 relationship between projects and test projects.

[Brandon]

For paths that contain spaces, the post-build command should be adjusted to copy /Y "$(ProjectDir)app.config" "$(TargetPath).config" Enjoy!

[Mordy]

I was just about to give up and hard code it within a #IF (DEBUG), thank god I found this post in time :)