Monday, April 27, 2009

Testing Without a QA Team

Yikes! That sounds like heresy! Regardless, it's the position that many developers find themselves in. Typically, this is for internal applications or small-ish products. There is no easy justification to hire a dedicated person (let alone a whole team) to do the QA. That doesn't mean that testing can't be done.

Here are some tips from a book "Beta Testing for Better Software" by Michael R. Fine on approaching the testing effort when there are no full-time QA assigned to the product.

  • Start with a usable test product
    Even if you have a list of known issues, if you can describe them clearly you can avoid unnecessary churn.
  • Target actual users
    Find people that not only know the domain for which the product is intended for, but also someone that is able to think clearly and provide good feedback.
  • Be sure you are mentally ready for the feedback
    You may have to ask for clarification for bugs that are reported. Users are not experienced in writing bug reports and likely cannot do any sort of analysis on what the root cause is.
  • Be sure your schedule is ready for the feedback
    The users that are helping with the feedback will be more likely to provide feedback if they feel that their issues are addressed quickly. If there is a server portion that is offline for several hours, users may loose confidence in the product. Knowing that it's down and reacting to that quickly is good. If there is a simple fix you can provide, be ready to send that out as appropriate. Even better, plan on updates to the software as a result of the beta testing so that the users can see the improvement (and have some pride knowing that they contributed to this)
  • Have a clear plan
    Knowing how many users is one thing. Clearly guiding them on what new features they should be focusing on or bug fixes to verify is important. You should also have a clear timeframe in which to ask for this feedback.
  • Identify what success looks like
    How do you know if the beta test was successful? How do you know that the new features have been sufficiently exercised? How do you know that your participants provided some value? How do you plan on addressing these issues before you leave 'Beta'? How does this beta compare to previous betas?
The book goes into more details and I encourage you to read it. Even if you have a full-time QA staff, this book provides some insight into how Beta testing can be used effectively within your current processes.

No comments: