Visual Studio 2010 Lab Management Overview

In PDC 2008 it was announced that Visual Studio Team System 2010 would contain functionality for managing test labs. Most of us don’t have one, but should we care?

What is a test lab?

Test labs are groups of computers dedicated for the sole task of running test copies of your applications. In more advanced scenarios, the operation of the test lab is highly automated – in practice machines should be scripted to initialize themselves, run a set of tests and report the results.

For simple applications, such labs are not usually necessary. For complex applications involving lots of OS version sensitive parts and localization issues, testing the app on all the platform combinations by hand would be impossible, so lots of machines and some sort of automation is called for.

Visual Studio 2010 now offers to put all this under your command with an easy interface. It also throws in SCVMM/Hyper-V virtualization to balance the testing load and save on hardware resources.

How does VS2010 manage test labs?

Essentially, the lab-driven test workflow goes like this:

  1. IT administrators allocate machine pools consisting of virtual server hosts that can be used for the test lab purpose.
  2. IT administrators or testers configure a set of virtual machines that can be used as testing environments. Such VMs would then be saved into a VM library for further use. Typical virtual machines might include configurations such as “Windows XP 64-bit Finnish” or “Windows Server 2008 with SQL Server 2005”.
  3. Testers configure “test environments” into VSTS 2010, forming a group of machines used for the purpose. A test environment might be called “FooSoft 1.3 integration test” or “Jouni’s FooSoft 1.4 beta test”.
  4. As the tester picks machines to add to the environment, VSTS forwards a request to a System Center Virtual Machine Manager instance to spin up new VMs based on the VM images created earlier on. When the new VMs are up, the test environment is ready.
  5. The tester runs the tests, automated or manual, on the testing environment.
  6. The testing machines can then be torn down and the machine capacity reused.

What you’re essentially gaining is higher usage of test-dedicated hardware, combined with reduced complexity. Your testers don’t have to work with test environment setup which improves efficiency, of course. For bonus points, hard bugs are no more flagged as non-repro, as the tester can snapshot the virtual machine at the time of the bug’s occurrence and then deliver the VM to the developer, giving him an instant repro case.

Do I need one of these?

If you’re developing a well-behaving web application, chances are you don’t need one. Installation of ASP.NET web apps is generally pretty easy to automate (by, for example, PowerShell), and often even multiple simultaneous test runs can be executed in the same logical OS using separate IIS sites and databases.

If you’re writing WinForms, WPF or Windows Service applications, the separate lab installation is more appealing, as there are often components that would benefit from getting installed onto a clean server every time, perhaps making even installer testing worthwhile. But remember, even in these cases you can also do without VS2010 – creating a simple Hyper-V –based virtual test environment isn’t really that hard. If you can read Finnish, check out some of Jani Järvinen’s notes from one of our ITpro.fi sessions.

On the other hand, if you have Visual Studio Team System + SCVMM running already, have extra Hyper-V –capable hardware and actually can use the testing environment (i.e. you have sufficient manual testing power and/or unit test automation), why not? A test lab need to be tens of computers. It can even be one relatively modern box, 64-bitness and hardware virtualization support being the key requirements. For most of the time, even desktop hardware will do quite nicely.

More info

Share with your friends:
  • Digg
  • del.icio.us
  • Facebook
  • Twitter

July 3, 2009  Tags: , , ,   Posted in: .NET

Leave a Reply