Here are some tips from a book "Beta Testing for Better Software" by Michael R. Fine on approaching the testing effort when there are no full-time QA assigned to the product.
- Start with a usable test product
Even if you have a list of known issues, if you can describe them clearly you can avoid unnecessary churn. - Target actual users
Find people that not only know the domain for which the product is intended for, but also someone that is able to think clearly and provide good feedback. - Be sure you are mentally ready for the feedback
You may have to ask for clarification for bugs that are reported. Users are not experienced in writing bug reports and likely cannot do any sort of analysis on what the root cause is. - Be sure your schedule is ready for the feedback
The users that are helping with the feedback will be more likely to provide feedback if they feel that their issues are addressed quickly. If there is a server portion that is offline for several hours, users may loose confidence in the product. Knowing that it's down and reacting to that quickly is good. If there is a simple fix you can provide, be ready to send that out as appropriate. Even better, plan on updates to the software as a result of the beta testing so that the users can see the improvement (and have some pride knowing that they contributed to this) - Have a clear plan
Knowing how many users is one thing. Clearly guiding them on what new features they should be focusing on or bug fixes to verify is important. You should also have a clear timeframe in which to ask for this feedback. - Identify what success looks like
How do you know if the beta test was successful? How do you know that the new features have been sufficiently exercised? How do you know that your participants provided some value? How do you plan on addressing these issues before you leave 'Beta'? How does this beta compare to previous betas?
No comments:
Post a Comment