How does User Experience and Agile interact?

The whole idea of 'User Experience' has always meant a relatively long process to describe the requirements and validate them, meaning there didn't seem to be a good fit in an Agile team. Alas, I find myself re-thinking my ideas when confronted with new information.

Here is an article that describes one project's efforts to integrate UX tasks into an agile project.

http://johnnyholland.org/2010/10/21/beyond-staggered-sprints-how-theladders-com-integrated-ux-into-agile/

New scripting environment for UI testing - Sikuli

The one thing that makes this different from other lightweight scripting tools is that you take screenshots as markers for the UI components to interact with. (Thanks to John Z. for sending this to me)

http://groups.csail.mit.edu/uid/sikuli/

Free StarWest Virtual Access!

I just got an email that the StarWest Virtual conference is available for free! I attended that day online and the presentations were very good. I hope they continue to offer this!

https://event.on24.com/eventRegistration/EventLobbyServlet?target=registration.jsp&eventid=239875&sessionid=1&key=7C4552EDC5D391DB815FD6223C1C630A&sourcepage=register

Worse Than Failure

If you aren't a fan of 'The Daily WTF', you should be. They post all our dirty laundry from a software perspective in the hopes that we don't commit the same crimes.

To make this easier, they've created a Visual Studio Plugin that allows you to easily submit code samples that have caused you to exclaim 'WTF?'

http://thedailywtf.com/Articles/Submit-WTF-Code-Directly-From-Your-IDE.aspx

Microsoft Web Application Configuration Analyzer

Here is a new tool from Microsoft to check the security settings of your production web servers to ensure they are configured with the proper level of security. (Thanks to Bill V. for sending this to me)

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=60585590-57DF-4FC1-8F0C-05A286059406

Stupid DNS tricks - Saving/Loading Data

http://byteworm.com/2010/10/27/free-content-delivery-network-using-dns-cache/

This is one more way to keep data available without using local storage. From a security standpoint, it could be used for a 'scratchpad' during intrusions. Maybe you DNS admins can come up with some way to track this? :)

No Good Deed Goes Unpunished

Every evening before my son goes to bed, we spend half an hour reading a book. We've read Treasure Island, Percy Jackson, Raggedy Ann and many, many more. Last night, we were 'between books' and we picked up a copy of poems by Elizabeth Barrett Browning to see how that would would work out. My son randomly picked the poem below.

The real background to this story is interesting, but see if you see anything familiar.

Here is a preview of some quotes

• 'For where's our blessed " status quo," '
• 'A just Deed in the world ? — call out the rifles !'

TALE OF VILLAFRANCA - Elizabeth Barrett Browning

TOLD IN TUSCANY

First printed in the Athenceum, September 24,1859.

My little son, my Florentine,

Sit down beside my knee,

And I will tell you why the sign

Of joy which flushed our Italy

Has faded since but yesternight;

And why your Florence of delight

Is mourning as you see.

A great man (who was crowned one day)

Imagined a great Deed:

He shaped it out of cloud and clay,

He touched it finely till the seed

Possessed the flower: from heart and brain

He fed it with large thoughts humane,

To help a people's need.

He brought it out into the sun —

They blessed it to his face:

'O great pure Deed, that hast undone

So many bad and base !

O generous Deed, heroic Deed,

Come forth, be perfected, succeed,

Deliver by God's grace.'

Then sovereigns, statesmen, north and south,

Rose up in wrath and fear,

And cried, protesting by one mouth,

' What monster have we here ?

A great Deed at this hour of day ?

A great just Deed — and not for pay?

Absurd, — or insincere.'

'And if sincere, the heavier blow

In that case we shall bear,

For where's our blessed " status quo,"

Our holy treaties, where, —

Our rights to sell a race, or buy,

Protect and pillage, occupy,

And civilize despair ?'

Some muttered that the great Deed meant

A great pretext to sin;

And others, the pretext, so lent,

Was heinous (to begin).

Volcanic terms of ' great' and ' just' ?

Admit such tongues of flame, the crust

Of time and law falls in.

A great Deed in this world of ours ?

Unheard of the pretence is:

It threatens plainly the great Powers;

Is fatal in all senses.

A just Deed in the world ? — call out

The rifles ! be not slack about

The national defences.

And many murmured, ' From this source

What red blood must be poured !'

And some rejoined, ' 'T is even worse;

What red tape is ignored !'

All cursed the Doer for an evil

Called here, enlarging on the Devil, —

There, monkeying the Lord I

Some said it could not be explained,

Some, could not be excused;

And others, ' Leave it unrestrained,

Gehenna's self is loosed.'

And all cried, ' Crush it, maim it, gag it!

Set dog-toothed lies to tear it ragged,

Truncated and traduced !'

But He stood sad before the sun

(The peoples felt their fate).

'The world is many, — I am one;

My great Deed was too great.

God's fruit of justice ripens slow:

Men's souls are narrow, let them grow.

My brothers, we must wait.'

The tale is ended, child of mine,

Turned graver at my knee.

They say your eyes, my Florentine,

Are English: it may be.

And yet I've marked as blue a pair

Following the doves across the square

At Venice by the sea.

Ah child I ah child ! I cannot say

A word more. You conceive

The reason now, why just to-day

We see our Florence grieve.

Ah child, look up into the sky !

In this low world, where great Deeds die,

What matter if we live ?

Book Review - Software Engineering Best Practices

Read my latest book review on Sticky Minds

http://www.stickyminds.com/s.asp?F=S1243_Book_4

Free ebook on Computer Security

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=87583728-ef14-4703-a649-0fd34bd19d13&displayLang=en

This book is for end users primarily, but it does give testers ideas for what kinds of security testing to consider for your applications.

Google Code University

I saw a link to this online today. Check it out and build up your coding chops!

http://code.google.com/edu/courses.html


Here is a sampling of the courses offered
Programming Languages Web Programming Web Security Algorithms

Enjoy!

New Email Address

We're going to be using redearthqa@gmail.com . Please update your contacts.

That is all :)

Software Test Automation Workshop - September 14-15

http://www.mysoftwaretesting.com/product_p/okcauto.htm

This 2-day hands-on course focuses on the basics of software test automation and expands on those topics to learn some of the deeper issues of test automation. This course is not specific to any particular tool set but does include hands-on exercises using free and cheap test tools. Bring your own notebook computer!

The main objective of this course is to help you understand the landscape of software test automation and how to make test automation a reality in your organization. You will learn the top challenges of test automation and which approaches are the best ones for your situation, how to establish your own test automation organization, and how to design software with test automation in mind. You will also learn many of the lessons of test automation by performing exercises using sample test automation tools on sample applications.

You will leave the course with your own test automation strategy and plan for implementing it.

GSA discount available. Contact us for information.

This workshop will be held at:

Hampton Inn, Airport South
1905 South Meridian Avenue
Oklahoma City, Oklahoma,
USA, 73108-1719
1-405-682-2080

Randy Rice is back in OKC! I hope you are able to make it!

Webinar: Fundamentals of Complete Crash and Hang Memory Dump Analysis (Second Session)

Here is an excellent opportunity to look at analyzing memory dumps by an expert. This is the second session and I'm sorry I missed the first)

Information and Registration for the Webinar

Replica of an early computer

This video shows how the Atanasoff-Berry Computer works. It's an interesting mix of mechanical and electrical components to solve problems.

http://www.wimp.com/importantcomputer/

Coding skills

Here is a list of sites to test your coding skills.

http://sixrevisions.com/resources/10-puzzle-websites-to-sharpen-your-programming-skills/

MD5 checksum is good, but not foolproof

As with anything security-related, if something is touted as tough to crack, that's taken up as a challenge. I recently stumbled on an article that not only gives examples of two sets of data with the same MD5 sum, but a method to exploit that.

http://www.mscs.dal.ca/~selinger/md5collision/

But I already learned that...

I was re-reading a technical book the other day and was starting to get annoyed at having to re-learn some things I had learned previously, but have since forgotten. Then I began to ponder the idea of why we forget. There are some people that have a photographic memory and never forget, but they are rare. Consider for a moment what life could be like if you never forgot anything. Suppose that every time you got into a car, you could remember the horror of the car accident you had ten years ago. Maybe you would be flooded with the pain of being lost every time you went to that same grocery store you were lost in when you were five. It may be so severe, you would need to find a different place to shop. So forgetting is not such a bad thing, but it can be inconvenient at times. So like Monk says "It's a gift.. and a curse".

Don't beat yourself up if you have to 'sharpen the saw' (a la Steven Covey) and re-learn things. Take the opportunity to re-think the information and hopefully gain from the experience.

Memory Dump Analysis Book

Dmitry Vostokov has released a book on Safari Online books called "Memory Dump Analysis Anthology, Volume 1"

http://my.safaribooksonline.com/9780955832802

Those of you with ACM (and possibly IEEE) memberships may already have access to this online.

How to be a programmer - One person's perspective

http://samizdat.mines.edu/howto/HowToBeAProgrammer.html

I ran across this the other day. It's old, but it does describe some skills that are not often discussed.

More ways to learn about computer security

I ran across "Damn Vulnerable Linux" as an environment to understand security vulnerabilities. It's offered through a company that provides training and certification for computer security.

I haven't tried to download or use the image yet, though I hope to soon.

http://www.damnvulnerablelinux.org/

Test your code online!

I've seen this posted elsewhere and it's worth a look! While this is not a full test, but you can validate your C# code online or you can download Pex and use it locally.

Interesting debugging technique for windows

I've been reading "Windows Internals, Fifth Edition" and ran across an interesting way to debug not only windows drivers, but any user application.

If you have a MSDN license, you can get what's called the 'checked build'. This is a build with debug messages enabled and optimizations turned off. It's most useful for debugging device drivers, but it can help replicate timing issues with the kernel since the timing will be different than for the retail version. Rather than having to install each component from the checked build, you can limit it to a couple files files. The instructions below show how to install and set up your system to have a boot option for this.

Obtain the checked build
http://msdn.microsoft.com/en-us/library/ff549603(v=VS.85).aspx

Install minimal comonents from the checked build
http://msdn.microsoft.com/en-us/library/ff547196(VS.85).aspx

Especially if you are tracking down timing issues, this may expose the issues more clearly.

Barriers to Automation

I've seen several attempts at automation from several companies. Many getting started and then dropping off and eventually rendered unusable. Keeping in mind the definition of insanity being "Doing the same thing over and over and expecting different results", it may be time to look at why these have failed.

Here are some of the scenarios that have come to mind based on what I've seen:

• Someone takes the inititative to create a set of tests, but other priorities take them away and the tests become obsolete, making them nearly useless.


• We bring in a contractor to build tests, but when the contract is over, there is nobody given the responsibility and the time to keep them up.


• We start some UI automation testing and find that the scripts are fragile, making upkeep difficult and ultimately are left to become obsolete.

This is not to say that we don't have some successes with automation:

• There are many experiences of using throw-away scripts to perform some focused and repetative task.


• There are internal tools built to assist with generating data.
  
• Development teams have their own scripts / applications for performing installation/configuration/cleanup tasks.

The trick is to see the pattern with the successes and failures.

Successful attempts at automation seem to have these common qualities. They typically are either grassroots efforts where time is found to work on them or they are given priority by management to spend on them. Grassroots efforts typically have modest upkeep costs and time can be found for upkeep. Management-directed efforts have had continued priority set for them since they require much more upkeep. Grassroots projects are typically used heavily by internal staff and Management priorities are typically used outside the development teams (including other internal teams as well as customers). Your experiences may differ from these as these are based on my own observations.

Unsuccessful attempts at automation appear to have these in common. There was no call to maintain the time needed to maintain these scripts either from the grassroots level or from management. The scripts were succeptible to changes in code, operating system, 3rd party components such as browsers, Java, .NET, Application Server Versions, etc.

So how do we take advantage of the things that make these efforts successful and mitigate the things that make them unsuccessful?

Auutomation has to be something that is used regularly. Whether it's an expectation of your development process or a commitment made to have time spent on upkeep during a project, it can't be an afterthought.

The benefits of automation must be valued both at the grassroots level and by management. I see that in both cases, I generally see agreement that automation is helpful, but I think there may be different ideas on what that looks like. Having this be very visible and openly discussed will contribute to it's long-term success.

Environment and code changes that affect scripting should be mitigated. Managing unit tests over time is a difficult process when the library of unit tests becomes large. Not only do they take time to run, they need to be managed and 'sunset' just as we would do for any other piece of code. There needs to be a lifecycle for these tests that address 1)When they should be built 2)How long they should be maintained 3)When they should be removed from use. UI tests are much more succeptible to these environmental changes. For example, different operating systems render web pages as well as applications differently, making some UI tests suitable for cross-platform execution difficult or impractical. Using 3rd-party UI components require customized tools to use for automation, often at additional cost. These additional tools are not absolutely required, but they do help with not only automating the tests, but also in validating the results. These UI tests also need to have a lifecycle with the same requirements as for Unit Tests. While UI tests are helpful, there needs to be more scrutiny applied to what tests get automated and in what environment(s).

Now what?

"It depends". Much of what needs to happen must be based on your circumstances. What is the will internally to make changes? How far does this will go to ensure that these changes are implemented for the long-term? What resources are available to impelement these? What training is needed? Once you start to answer these, the answers will become more clear.

Book Review - Little Brother

Well, it's not a book review in the truest sense. I've just gotten about a third of the way through "Little Brother" (free download) and I just had to post a review. You can read a summary from the website for the book.

Essentially, this is a fictional account based in many topics related to security (physical, computer, privacy, etc.) and a series of events that affect these. A seventeen year old boy and his friends are caught up in a terrorist plot by being in the wrong place at the wrong time. What follows is a drastic shift in what various governments and other groups consider 'acceptable levels of monitoring' and what it means to those being monitored and those doing the monitoring.

What strikes me most is the main character's internal monologue on the effectiveness of different security measures. In some cases, they just make everyone feel more secure, but do little to address true risks. In other cases, the data gathered starts to be misused, prompting the question "Who is watching the watchers?".

This book is an easy and excellent read. It's entertaining and thought provoking. You may even learn a thing or two. I certainly am. The book can be purchased and it can also be downloaded in several electronic formats for free.

Analyzing dumps

Likely someone at your company is doing this. From a QA perspective, it can shed quite a bit of light into some of the odd behaviors that is seen sometimes. This blog may be a good reference on understanding the details.

http://www.dumpanalysis.org/blog/

Automate Changes to web.config files for unit testing ASP.NET Web Services

We're starting to automate unit tests with our build process and I was told about some new features in VS2010 to help us out. Thanks to John Z. for pointing this out. With Visual Studio 2010, you can manage both the developer and release versions of web.config files and even publish different database versions.

Here is the main link on Microsoft's website for these features. http://msdn.microsoft.com/en-us/library/dd394698(v=VS.100).aspx

Specifically, Here is the link to creating a transform for the web.config file. http://msdn.microsoft.com/en-us/library/dd465318(v=VS.100).aspx

Also, you can specify a database by following the informaiton here. http://msdn.microsoft.com/en-us/library/dd465343(v=VS.100).aspx

Here is a summary of a blogger that has discussed these deployment options. http://vishaljoshi.blogspot.com/2009/09/overview-post-for-web-deployment-in-vs.html

Summer Hiatus for Meetings

We'll be putting the Red Earth QA monthly meetings on Hiatus for the summer and pick back up again in the fall.

Better Software in 60 Seconds, #1

Here is an early sample of a project I'm working on. It's very rough, but this should give you an idea of what I'm trying to do. All feedback is welcome!

Open Source Load Testing Tool

Recently, a friend asked if I would be interested in learning about Erlang, a functional programming language, for a project he was thinking about. Essentially, it's a language that lives on concurrency and recursive algorithms. Then it seemed obvious that it would be a great platform for performance testing.

And a quick search resulted in finding Tsung (http://tsung.erlang-projects.org/). It supports HTTP, WebDAV, SOAP, PostgreSQL, MySQL, LDAP and XMPP/Jabber. Give it a try and see if you find it useful.

Google's Website Has Vulnerabilities (and they want it that way)

I ran across http://jarlsberg.appspot.com/ the other day. It's a site that Google has assembled to teach anyone how to find vulnerabilities. It's set up so that you can walk through a live site and see exactly what kinds of things to look for, how they behave and how to fix them.

Why not get a few QA nerds (like yourself) together and work through these labs?

On-line demos for Unified TestPro manual and automation testing tool (please RSVP)

Presentations will be as follows:

Manual tool demo; 9am and 11am CDT, April 21st, or 11am CDT April 23rd
Automation tool demo; 9am and 11am CDT April 22nd, or 3pm CDT April 23rd.

GoToMeeting or GoToWebinar format.

This will be an RSVP event. Information will be provided upon RSVP.
James Wright jwright@okc.sdtcorp.com
General Manager, Test Service Operations
Software Development Technologies
405-232-6000 x.11
405-232-6008 Fax
580-641-1360 Cell

Online Demo of Unified TestPro, Friday, April 23rd

SDT's Enterprise proven, Keyword Driven, Software Test Design and Automation solution, Unified TestPro, will be demonstrated on-line Friday, April 23rd. The UTP manual product will be demonstrated at 9am CDT, while the automation product will be demonstrated at 11am CDT. This is an RSVP event. For more information, please contact James Wright at jwright@sdtcorp.com

International Symposium on Functional Programming (in Norman)

The Symposium on Trends in Functional Programming will begin its second decade at the University of Oklahoma, May 17-19, 2010, on the campus in Norman.

Additional details can be found at http://www.cs.ou.edu/tfp2010/

For local residents, contact Rex Page page@ou.edu for a special $50 reduced rate fee that includes the lectures, breaks, presentations, and the local proceedings, but does not include lunches or the excursion and banquet. Locals can also attend the excursion and banquet for and additional $75. The general entrance fee is $300 otherwise, so this is a great deal!

Last year's topics included:

• What is F# and Why Should I Learn It?

• Hands-on exercises in functional programming using C++ (with discussions about LINQ for C#)

• Implementing unit of measurement types in Visual Studio 2008

• Software Testing with QuickCheck

• Analysis of F# programs

• Formal validation methods

My Article on Software Quality History

Here is the first of three in a series of stories about the history of software quality.

http://blog.softwaretestingclub.com/2010/04/grace-hopper-had-a-sense-of-humor-part-1-by-robert-watkins/

Test Coverage Technique

Here is an excellent article on using Mind Map software to describe test coverage.

http://testconsultant.blogspot.com/2010/04/ewt12-mind-mapping-and-testing.html

The Good, The Bad and The Ugly of NTFS Alternate Data Streams

The Good

When NTFS was introduced for the Windows NT platform, one of the features that was added was 'alternate data streams' (ADS). This was specifically added to allow for a similar concept of 'forks' for Mac users to allow them to maintain their 'data fork' and 'resource fork' for files and use NTFS as a network data share.

A file stream is essentially metadata added to a file that doesn't interfere with the contents of the file. One way to see this data is to look at the 'Summary' tab of a file.

But as you can see, the comments section in particular can be quite lengthy. Note that not all file streams appear here, others may be included. Also, the space used by the file streams is not used when calculating the file size, so adding more data to the ADS does not change the size reported by the OS.

This method is also used in other ways. Icon files are associated with the URL shortcut files for IE as a file stream and the blocking of downloaded files from execution is handled through ADS. Microsoft provides a tool to let you find files that have streams attached to them. You can read about it here http://technet.microsoft.com/en-us/sysinternals/bb897440.aspx

The Bad and The Ugly
This method can be used to make both text and binary files difficult to find. It's even possible to use the command shell to make an executable file (such as notepad) an ADS for a pre-existing executable (such as the calculator application), rename the executable and run it! This sample screenshot shows what happens when notepad.exe is added as an ADS called 'runtime.exe' to the calc.exe file and is executed. In this case, the notepad executable is being run from the runtime.exe ADS for the calc.exe.

Fortunately, you see the shenanigans when looking at the task manager. Also, all of these activities require someone with sufficient permissions. This means, that your system would already have to be compromised for this to happen.

Also, this information is only accessible on NTFS file systems and is not included when sending files over the network (unless it's to / from an NTFS file share).

Hopefully, this will shed some light into the inner-workings of the file system and explain some behavior you may have seen.

Practical Software Test Automation Course - Oklahoma City, May 13 & 14, 2010

We are fortunate to have Randy Rice local to our area and are happy to announce that we are helping to sponsor his Practical Software Test Automation Course. Registration and course information can be found here.

https://www.mysoftwaretesting.com/ProductDetails.asp?ProductCode=OKCAUTO

Even if you are not in the Oklahoma City area, this course is sure to be useful to anyone getting started with (or trying to recover from) their automation efforts.

On his site, you will find online training as well. Have a look around and see if other courses meet your needs as well.

Understanding Computers from 'first principles'

While researching information on computer security, I ran across the Security Now! podcasts( http://www.grc.com/securitynow.htm ).

Among the podcasts is a series of discussions that provides insight into how computers work at a fundamental (and somewhat technical) level.

Here are the podcasts that I found particularly interesting:

Basic Architecture of computers from the 1950s

http://media.grc.com/sn/sn-233-lq.mp3

Machine Language

http://media.grc.com/sn/sn-235-lq.mp3

Pointers

http://media.grc.com/sn/sn-237-lq.mp3

Stacks, Registers and Recursion

http://media.grc.com/sn/sn-239-lq.mp3

Hardware Interrupts

http://media.grc.com/sn/sn-241-lq.mp3

These are the low-bandwith MP3s, there are high bandwidth versions as well as notes and supplimentary materials provided as well.

Meeting Announcement- Evening Meeting 4:30pm-6pm Thursday, March 25th

NEWS
- We're moving to evenings to accommodate your busy work schedule!

Time and Location
We will be meeting at the FIS/Metavante offices at 1200 Sovereign Row. The meeting is from 4:30pm to 6pm.

Free parking is available!

Topic

Robert Watkins will discuss how FIS approached performance testing for one commercial server product and the process to provide performance testing utilities to customers.

Directions

• From I-40, take Meridian South
• Turn Left at Will Rogers Parkway
• Turn Left at Sovereign Row
• The FIS/Metavante offices are on the right just before the curve in the road.

Sound Familiar?

I just started with xtranormal's tool for creating movies. Here's my first try.

http://www.xtranormal.com/watch/6209807/

Charles Babbage's connection to the Luddites

Charles Babbage is well known for his Difference Engine and lesser known for his Analytical Engine. When Luigi Menabrea (future prime minister of Italy and engineer) translated his notes of Babbage's lectures into French, Babbage asked a long-time freind, Ada Lovelace to translate those notes to English. That was in 1842. Ada's father was Lord Byron, the poet and Parlimentarian. In 1815, one of his first speeches to the House of Lords was in defense of the Luddites.

The Rule of 1000 Decisions (as a tie into myth-busters)

A couple 'prior-lives' ago, I worked with a CIO at a mid-sized bank who gave me quite a bit of encouragement and opportunity. There were several things I remember from working with him.

The first was his success criteria for the project to bring in the bank's website in-house. 'Just make it suck less'. I believe we did that in spades. The basic design was functional enough to last about five years before they out grew that architecture. Quite a feat for a site that was developed in 2001.

The second was his 'Rule of 1000 Decisions'. Essentially, this rule start with the premise that everyone makes mistakes and generally, everyone makes more good decisions than bad decisions. The caveat is that those who don't may not last long. Supposing that for every 1000 decisions you make, you have 900 good decisions, 50 bad decisions and 50 great decisions. If you start to avoid making decisions to avoid the bad ones, you are missing out on more good and great decisions that you could be making. The point is that you need to keep making decisions and accept the fallout as well as the praise for those decisions.

The tie into myth busters is this discussion on the importance of failures, even massive failures.

Who Says Writing Documentation is Dry?

Here is a great story about how not all computer documentation is dull reading (or writing).

Enjoy!

http://www.ironicsans.com/2010/02/they_dont_make_computer_manual.html

Free Seminar on User Stories, March 2nd.

The link to sign up is http://app.en25.com/e/es.aspx?s=1156&e=13096&elq=e6632e0b731a40f2b08b0e622e3b26b9

Here's an excerpt from the description

"Do you want more practical agile training on planning your iterations? Are your teams trying to get into flow but struggle to get answers to tactical questions on user stories? Join agile coach Ken Clyne as he shares successful tactics for writing value-focused stories; sizing, splitting, tasking, and accepting user stories; and strategies for working across teams from iteration to iteration. He also explains the differences between stories and tasks, how to update the backlog, ways to chart and track story point burn-up, write acceptance criteria, and understand when a story is done."

Follow Us On Twitter!

We're making our entries here available through Twitter! Follow us and get updates from our blog.

@redearthqa

My Review of "Modeling Software Behavior"

A few weeks ago, I read "Modeling Software Behavior" by Paul Jorgensen for StickyMinds.com.

If you haven't been there, it's an excellent resource for not only book reviews :) , but also lots of other information on tools and techniques. This is produced by the same company that puts on The Star Conferences.


http://www.stickyminds.com/s.asp?F=S1203_BOOK_4

New Meeting Location for February.

We will be meeting at the FIS/Metavante offices at 1200 Sovereign Row. The meeting is still from 4:30pm to 6pm.

Directions

• From I-40, take Meridian South
• Turn Left at Will Rogers Parkway
• Turn Left at Sovereign Row
• The FIS/Metavante offices are on the right just before the curve in the road.

Free parking is available!

Meeting Announcement- Evening Meeting 4:30pm-6pm Thursday, February 18th

NEWS
- We're moving to evenings to accommodate your busy work schedule!

Time and Location

The Red Earth QA's meeting will be held on the 3rd floor of 100 N. Broadway from 4:30pm -6pm on Thursday, February 18th. Look for the signs to direct you to the correct room.

Topic

We will be discussing 'Test Plan - Best Practices - Bring a Sample'

Randy Rice of Rice Consulting has agreed to lead the discussion!

Directions

• You can park in Main Street Parking on Main or you can find street parking.
• From I-40, take the Robinson Exit. Go North on Robinson to Main. Right on Main. You can either go to Main Street Parking or continue to Santa Fe Parking. You will see 100 N Broadway on your left across Broadway. The building says 'Chase' at the top.

Was your last project successful?

I was thinking about how the same project can be rejected or approved depending on who you ask. This flowchart is the result, enjoy.

Online UI Mockup Tool

A small company called Balsamiq is making an online version of their product to create screen mockups available.

http://www.balsamiq.com/demos/mockups/Mockups.html

It pops up a dialog every 5 min or so to suggest using a paid version. Even if you don't get licenses for this, it may be useful for small projects.

Details of the features are here:
http://www.balsamiq.com/products/mockups

Agile Technique Proposed in 1968

I ran across an article, "Iterative Multi-Level Modeling - A Methodology for Computer System Design" from 1968 that essentially talks about the benefits of an agile approach to system design.

"The paper presents a method of modelling a computer system design as it evolves, so that evaluation can be made an integral part of the design process. The paper introduces the concept of concurrent existence, within a single model, of several representations of the system being modelled, at differing levels of abstraction. Thus important design decisions are expressed directly in terms of appropriately abstract quantities, facilitating understanding, validation, and modification of the system design. The paper includes brief details of an experimental implementation of the modelling technique and of the use of the technique to model both hardware and software components of a multi-processing system."

One sign you've found somone that cares about your process

In light of our meeting this week, I thought I'd post a comic from a few years ago. Enjoy!

Meeting Announcement- EVENING MEETING! 4:30PM-6PM, Thursday, January 28th

NEWS
- We're moving to evenings to accommodate your busy work schedule!

Time and Location

The Red Earth QA's meeting will be held on the 3rd floor of 100 N. Broadway from 11:30am-1pm on Thursday, December 17th. Look for the signs to direct you to the correct room.

Topic

Compare Test Plans!
Bring your test plan template and compare with other companies.

Directions

• You can park in Main Street Parking on Main or you can find street parking.
• From I-40, take the Robinson Exit. Go North on Robinson to Main. Right on Main. You can either go to Main Street Parking or continue to Santa Fe Parking. You will see 100 N Broadway on your left across Broadway. The building says 'Chase' at the top.

Wired Magazine - "Software Flaws Let Christmas Bomber Get Through"

Wired is running a story about some results of the investigation into the Christmas bomber. It appears that sub-standard software contributed to the problems.

http://www.wired.com/dangerroom/2010/01/obama-software-flaws-let-christmas-bomber-get-through/

Project Management, 60s style

Here is a link to a book on project management from 1970. It's a second edition of the original published in 1964.

What I originally found amazing is that it's a fairly sophisticated view of project management. The first chapter is particularly enlightening since it discusses the inspirations for the methods detailed in the book. Basically, there was research from as far back as 1896 on streamlining production processes that are being applied to research in the late 1950s that culminated in the findings described in this book.

Free e-book!

You can download your own free copy of Tester Types. It's on a site called Software Testing Club, which may also be a good resource.

Easy Reader

Google's Reader is an excellent way to keep up with blogs, especially blogs that don't update daily.

Here is a list of the QA blogs that I track that you can import and have a single place to read all about QA in one place.

Enjoy!

Happy Holidays!

Here is a certificate for you or someone you work with. It 'allows the bearer to exchange one line of bad code for one line of good code' so that you can say that you have no bad code.

Code Offset Certificate

Rather than printing your own, you can actually buy a different one online and the proceeds go to supporting open source software.

Enjoy!

Meeting Announcement - (AT DEVON) Dec 17th

NEWS
- We're back at Devon this month.

Time and Location

The Red Earth QA's meeting will be held on the 3rd floor of 100 N. Broadway from 11:30am-1pm on Thursday, December 17th. Look for the signs to direct you to the correct room.

Topic

We will be having our annual holiday gathering. Bring your favorite treat to share with the group along with your lunch.

We will be having a planning session for 2010.

Directions

• You can park in Main Street Parking on Main or you can find street parking.
• From I-40, take the Robinson Exit. Go North on Robinson to Main. Right on Main. You can either go to Main Street Parking or continue to Santa Fe Parking. You will see 100 N Broadway on your left across Broadway. The building says 'Chase' at the top.

November Meeting Cancelled

Sorry for the mixup. We will not be having our November meeting.

Do you work with a Machiavellian Manager?

Suppose we 'update' Machiavelli's "The Prince" to be about corporate leaders instead of about being a prince. Here is how I could summarize his book.

Please firmly place your tounge in your cheek as you read this.

CONCERNING THE WAY IN WHICH THE STRENGTH OF ALL DEPARTMENTS OUGHT TO BE MEASURED

It is necessary to consider a point in examining the character of departments; that is, whether a manager has such power that, in case of need, he can support himself with his own budget and other resources, or whether he has always need of the assistance of others. And to make this quite clear I say that I consider those who are able to support themselves by their own resources who can, either by abundance of men or money, raise a sufficient strategy to confront anyone who comes to usurp them.

CONCERNING THINGS FOR WHICH MEN, AND ESPECIALLY MANAGERS, ARE PRAISED OR BLAMED

It remains now to see what ought to be the rules of conduct for a manager towards staff and peers. And as I know that many have written on this point, I expect I shall be considered presumptuous in mentioning it again, especially as in discussing it I shall depart from the methods of other people. But, it being my intention to write a thing which shall be useful to him who apprehends it, it appears to be more appropriate to follow the real truth of the matter than the imagination of it; for many have pictured companies and departments which in fact have never been known or seen, because how one lives is so far distant from how one ought to live, that he who neglects what is done for what ought to be done, sooner effects his ruin than his preservation; for a man who wishes to act entirely up to his professions of virtue soon meets with what destroys him among so much that is evil.

Hence it is necessary for a manager wishing to hold his own to know how to do wrong, and to make use of it or not according to necessity. Therefore, putting on one side imaginary things concerning a manager, and discussing those which are real, I say that all men when they are spoken of, and chiefly managers for being more highly placed, are remarkable for some of those qualities which bring them either blame or praise; and thus it is that one is reputed kind, another miserly; one cruel, one compassionate; one sincere, another cunning; one hard, another easy, and the like. And I know that every one will confess that it would be most praiseworthy in a manager to exhibit all the above qualities that are considered good; but because they can neither be entirely possessed nor observed, for human conditions do not permit it, it is necessary for him to be sufficiently prudent that he may know how to avoid the reproach of those vices which would lose him his status; and also to keep himself, if it be possible, from those which would not lose him it; but this not being possible, he may with less hesitation abandon himself to them. And again, he need not make himself uneasy at incurring a reproach for those vices without which the status can only be saved with difficulty, for if everything is considered carefully, it will be found that something which looks like virtue, if followed, would be his ruin; whilst something else, which looks like vice, yet followed brings him security and prosperity.

CONCERNING KINDNESS AND MEANNESS

Commencing then with the first of the above-named characteristics, I say that it would be well to be reputed kind. Nevertheless, kindness exercised in a way that does not bring you the reputation for it, injures you; for if one exercises it honestly and as it should be exercised, it may not become known, and you will not avoid the reporach of its opposite. Therfore anyone wishing to maintain among men the name of 'kindly' is obliged to avoid no attribute of magnificence; so that a manager thus inclined will consume in such acts all his resources, and will be compelled in the end, if he wish to maintain the name of 'kindly', to unduly weigh down his staff, take away their perks and do everything he can to increase his budget. This will soon make him odious to his subjects, and having his budget cut he will be little valued by any one; thus, with his kindness, having offended many and rewarded few, he is affected by the very first trouble and imperilled by whaterver may be the first danger; recognizing this himself, and wishing to draw back from it, he runs at once into the reproach of being miserly.

Therefore, a manager, not being able to exercise this virtue of kindness in such a way that it is recognized, except to his cost, if he is wise he ought not to fear the reputation of being mean, for in time he will come to be more considered if he were thought of as kind.

CONCERNING THE WAY IN WHICH MANAGERS SHOULD VIEW CORPORATE LAW

Every one admits how praiseworthy it is in a manager to follow legal guidelines, and to live with integrity and not with skill. Nevertheless our experience has been that those managers who have done great things have followed the law to little account, and have known how to circumvent the intellect of men by skill, and in the end have overcome those who have relied on their word. You mush know that there are two ways of increasing dominion the one by the law, the other by force. If men were entirely good this precept would not hold, but because they are bad, and will not work according to the law with you, you too are not bound to work with integrity with them.

But it is necessary to know well how to disguise this characteristic, and to be a great pretender and dissembler; and men are so simple, and so subject to present necessities, that he who seeks to deceive will always find someone who will allow himself to be deceived.

THAT ONE SHOULD AVOID BEING DESPISED AND HATED

The manager must consider, as has been in part said before, how to avoid those things which will make him hated or contemptible; and as often as he shall have succeeded he will have fulfilled his part, and he need not fear any danger in other reproaches.

That manager is highly esteemed who conveys this impression of himself, and he who is highly esteemed is not easily conspired against; for, provided it is well known that he is an excellent man and revered by his staff, he can only be attacked with difficulty. For this reason a manager ought to have two fears, one from within, on account of his staff, the other from without, on account of external powers. From the latter he is defended by having strong sanctions for violating cross-departmental policies and having good allies, and if he has strong sanctions he will have good friends, and affairs will always remain quiet within when they are quiet without, unless they should have been already disturbed by conspiracy; and even should affairs outside be disturbed, if he has carried out his preparations and has lived as I have said, as long as he does not despair, he will resist every attack. I consider that a manager ought to reckon conspiracies of little account when his staff hold him in esteem; but when it is hostile to him, and bears hatred towards him, he ought to fear everything and everybody.

ARE BURDENSOME CROSS-DEPARTMENTAL PROCEDURES, AND MANY OTHER THINGS TO WHICH MANAGERS OFTEN RESORT, ADVANTAGEOUS OR HURTFUL?

Some managers, so as to hold securely their department, have removed all decision-making powers from their staff; others have kept their teams distracted by in-fighting; others have fostered enmities against themselves; others have laid themselves out to gain over those whom they distrusted when they first became managers; some have built burdensome cross-departmental procedures; some have ignored and bypassed other's procedures. And although one cannot give a final judgment on all of these things unless one possesses the particulars of those departments in which a decision has to be made, nevertheless I will speak as comprehensively as the matter of itself will admit.

There never was a new manager who has taken away decision-making powers from his staff; rather when he has found them without the power to make decisions he has always granted this to them, because, by allowing them to make decisions, those decisions become yours, those men who were distrusted become faithful, and those who were faithful are kept so, and your

staff become your adherents.

Our predecessors, and those who were reckoned wise, were accustomed to say that it was necessary to foster quarrels in some of their teams so as to keep possession of them the more easily. This may have been well enough in those times when the company was in a way balanced, but I do not believe that it can be accepted as a precept for to-day, because I do not believe that factions can ever be of use; rather it is certain that when the enemy comes upon you in divided teams you are quickly lost, because the weakest party will always assist the outside forces and the other will not be able to resist.

It has been a custom with managers, in order to hold their departments more securely, to build over-burdensome cross-departmental policies and procedures that may serve as a bridle and bit to those who might desire to use their resources, and as a place of refuge from a first attack. I praise this system because it has been made use of formerly.

However the best possible policy is--not to be hated by the staff, because, although you may successful in enforcing these over-burdensome policies, yet they will not save you if the staff hate you, for there will never be wanting other managers to assist staff who have complained against you.

All these things considered then, I shall praise him who builds over-burdensome policies as well as him who does not, and I shall blame whoever, trusting in them, cares little about being hated by the people.

HOW A MANAGER SHOULD CONDUCT HIMSELF SO AS TO GAIN RENOWN

Nothing makes a manager so much esteemed as great enterprises and setting a fine example. And a manager ought, above all things, always endeavour in every action to gain for himself the reputation of being a great and remarkable man.

A manager is also respected when he is either a true friend or a downright enemy, that is to say, when, without any reservation, he declares himself in favour of one party against the other; which course will always be more advantageous than standing neutral; because if two of your powerful peers are at odds on some topic, they are of such a character that, if one of them wins, you have either to fear him or not. In either case it will always be more advantageous for you to declare yourself and to make sides strenuously; because, in the first case, if you do not declare yourself, you will invariably fall a prey to the winner, to the pleasure and satisfaction of him who has lost, and you will have no reasons to offer, nor anything to protect or to shelter you. Because he who wins does not want doubtful friends who will not aid him in the time of trial; and he who loses will not harbour you because you did not willingly court his fate.

A manager ought also to show himself a patron of ability, and to honor the proficient in every skill. At the same time he should encourage his staff to perform their work peaceably so that no one should be deterred from performing their duties for fear lest they be taken away from him or another from being hired because of poor benefits; but the manager ought to offer rewards to whoever wishes to do these things and designs in any way to honor his department.

Further, he ought to entertain the people with celebrations and parties at convenient seasons of the year; and as his department is divided into teams he ought to hold such bodies in esteem, and associate with them sometimes, and show himself an example of courtesy and kindness; nevertheless, always maintaining the majesty of his rank, for this he must never consent to abate in anything.

CONCERNING THE DIRECT REPORTS OF MANAGERS

The choice of direct reports is of no little importance to a manager, and they are good or not according to the discrimination of the manager. And the first opinion which one forms of a manager, and of his understanding, is by observing the men he has around him; and when they are capable and faithful he may always be considered wise, because he has known how to recognize the capable and to keep them faithful. But when they are otherwise one cannot form a good opinion of him, for the prime error which he made was in choosing them.

Prologue

There are many things that were left out. Some for space, some because I cringed as I was re-writing them. The cringing was because they hit pretty close to home. Overall, this should be read as a 'How Not To' rather than a 'How-To'.

"These are a few of my favorite techniques" slides and examples

You can download the slides and examples from the presentation here.

I hope you enjoy them. If you have any questions, please email me at robert@watkins.net

September Meeting Video

One of the perks of being at the okcCoCo is that we get a video of our meetings recorded and available online. Here is our September meeting. Enjoy!

Pex - automated white-box testing in .NET

For all you developers out there that are looking for unit testing tools, this may be worth looking into.

"Right from the Visual Studio code editor, Pex finds interesting input-output values of your methods, which you can save as a small test suite with high code coverage. Pex performs a systematic analysis, hunting for boundary conditions, exceptions and assertion failures, which you can debug right away. Pex enables Parameterized Unit Testing, an extension of Unit Testing that reduces test maintenance costs. Pex also comes with a lightweight framework for test stubs and detours, called Stubs and Moles."

http://research.microsoft.com/en-us/projects/pex/

Meeting Announcement- BRING A LUNCH! - Thursday, Oct 22nd 11:30am

Time and Location

The Red Earth QA's meeting will be held 723 N. Hudson Ave at our new location! We are gathering at Thursday, October 22nd at 11:30, the presentation starts at noon.

Topic: These Are a Few of My Favorite Techniques

Robert Watkins will be discussing some of his favorite testing tools/techniques/models including

- State-Transition Diagrams (manual and automated using Spec Explorer)

- Pairwise Testing (using allpairs)

- Truth Tables

- Domain Testing

The discussion will walk through several examples of:

- identifying what needs to be tested

- creating models

- using these models to develop tests

Can't attend in person?

We will record this session and post it online.

Directions/
The okcCoCo is located in midtown Oklahoma City at the intersection of 7th Street and Hudson Avenue.

Immediate access can be had from I-235 via 6th Street exists 1E and 1F. Take 6th Street West to Hudson Avenue, turn North onto Hudson Avenue and continue one block North to 7th Street.

The okcCoCo is located on the South West corner of 7th Street and Hudson Avenue.

Parking is available in four lots (see link). The primary parking lot has direct access to the okcCoCo property and is accessible from 7th Street. Auxiliary parking lots are accessible from 6th, 7th and 8th Street. One and two hour parking as well as metered parking are available along Hudson Ave, as well as along 6th, 7th, and 8th Streets.

Free Online Training!

http://now.eloqua.com/e/es.aspx?s=1156&e=7482&elq=e0ee27eca09b4976b94518322ee8016f

Live Web Seminar

"Delivering Business Value Through Exploratory Testing"

Join us - September 29, 2009

2:00 p.m. ET/11 a.m. PT

Exploratory testing is an often underutilized tool in software quality. Many organizations have trouble incorporating exploratory methods into their test strategy because it is viewed as "unstructured" or difficult to track back to productivity. However, given the complexity of today's systems and the difficultly of defining their expected state, outcomes, and use cases in advance, exploratory testing is a key tool in the software quality arsenal.

In this Web seminar, Michael Bolton and Chad Wathington will discuss how exploratory testing is particularly applicable to delivering business value in agile teams. They'll also cover:

• How to approach exploratory testing
• How to free up and structure time to use exploratory methods
• How to justify moving away from just test cases in your organization

"Feature Creep", the song and others...

http://tester.geordiekeitt.com/archives/75

Very funny song about feature creep...

Others
http://tester.geordiekeitt.com/archives/77

http://tester.geordiekeitt.com/archives/91

Meeting Announcement- NEW LOCATION! - Thursday, Sept 24th 11:30am

Time and Location

The Red Earth QA's meeting will be held 723 N. Hudson Ave at our new location! We are gathering at Thursday, September 24th at 11:30, the presentation starts at noon.

Topic: What’s New in Quality Center 10.0?

This informative session will highlight the new features and functionality included in the Quality Center 10.0 release including Quality Center Enterprise, Quality Center Premier, Requirements Management, Business Process Testing and Functional Testing. It will also cover not only the key integrations between the products but also new integrations with other products in the HP BTO portfolio and third party products

Can't attend in person?

Bridgeline: Dial In: 1.866.409.2889 (US); 1.702.696.4520 (Outside US); Participant: 6913575

Virtual Room Keys:

https://www.rooms.hp.com/attend/default.aspx?key=EPAADF7CCH

Directions
The okcCoCo is located in midtown Oklahoma City at the intersection of 7th Street and Hudson Avenue.

Immediate access can be had from I-235 via 6th Street exists 1E and 1F. Take 6th Street West to Hudson Avenue, turn North onto Hudson Avenue and continue one block North to 7th Street.

The okcCoCo is located on the South West corner of 7th Street and Hudson Avenue.

Parking is available in four lots (see link). The primary parking lot has direct access to the okcCoCo property and is accessible from 7th Street. Auxiliary parking lots are accessible from 6th, 7th and 8th Street. One and two hour parking as well as metered parking are available along Hudson Ave, as well as along 6th, 7th, and 8th Streets.

NEW MEETING LOCATION!

We're very excited to be meeting in a new location for our September meeting! The new location is the OKC Coworking Collaborative (aka OKC CoCo) in midtown OKC at Hudson and 7th.

We're still working on the topic for the meeting and our invitation should be out soon.

Meanwhile, you can check out their space and parking information.

We hope you can make it!

Meeting Announcement - Friday, August 28th 11:30am

Time and Location

The Red Earth QA's meeting will be held on the 3rd floor of 100 N. Broadway from 11:30am-1pm on FRIDAY, August 28th. Look for the signs to direct you to the correct room.

Topic

Devon will provide a view of their use of Silk Performer in their environment.

Directions

• You can park in Main Street Parking on Main or you can find street parking.
• From I-40, take the Robinson Exit. Go North on Robinson to Main. Right on Main. You can either go to Main Street Parking or continue to Santa Fe Parking. You will see 100 N Broadway on your left across Broadway. The building says 'Chase' at the top.

Regression Testing Without a QA team - Part II

Don't worry, while there's more to do, we are well positioned to make the most efficient use of our time with the remainder of the tasks.

Step 3 - Identify test cases / test data

For the purposes of this discussion, let's limit the concept of a 'test case' to a 1-2 sentence that describes the goal of each test you want to achieve. Start with the high-risk areas first. Depending on your situation, you may do only the risk level 1 items, or you may have time to do more. The answer depends on available time. Once you've built some test cases, you'll want to take a step back and build some models of your system to see if there are any more test cases you could write.

You'll also want to think about test data. You'll want to consider valid data, invalid data, large quantities of data, lack of data, etc.

For our example with the calculator, here are some preliminary test cases.

1. Perform valid basic arithmetic calculations
2. Perform 'invalid' basic arithmetic calculations (divide by 0, invalid input, etc.)
3. Perform scientific calculations
4. Perform 'invalid' scientific calculations (log 0, tan 90, etc.)
5. Perform statistical calculations

Hmmm.... While I was using the application to identify test cases, I realized that I've missed some of the features

- keyboard shortcuts (risk - 3)

- number format -decimal, octal, etc. (risk - 2)

- number type -degree, radian, grad (risk -2)

- use parentheses up to 25 levels at a time (risk -2)

The Model

Now that I've gotten into this a bit further, it seems that I'm ready to build a model to make sure that I'm covering these requirements. There are several types of models. Dataflow diagram, Workflow diagram, State Transition diagram, etc. At least one would be helpful, so here is one that I came up with.

State - Ready for a Number

Actions

- Enter a '('.

- Enter a 'binary' operator (a 'binary' operator takes two inputs like +, *, etc.)

- Enter a number,

State - Ready for an Operator

Actions

- Enter a ')'

- Enter a number

- Enter a 'unary' operator (a 'unary' operator takes one input like 1/x, sin x, etc.)

- Enter a 'binary' operator

As with all models, this model doesn't tell the whole story, but it is better than having no model. Among it's faults are that it does not cover the statisical mode calculations, the names of the states are not completely accurate, only valid actions are allowed, etc. You will have to get used to these sort of issues to be able to have a strong basis for testing available in a short time.

Planning for Test Cases

For now, let's assume that the starting state of the calculator is 'Ready for a Number'. I have three valid actions, enter a '(', a number or a binary operator. Suppose I choose to enter a number, then my state has changed to 'Ready for an Operator'. In this new state, I have four valid actions (listed above). You will quickly see that there are lots of paths I can follow where I select actions and that action may or may not change the state I'm in (though it will change the value of the calculation being displayed and stored). You can also see that there is no actual ending state, you can theoretically keep doing calculations idefinately.

Here's one example path to follow

- Enter a number

- Enter a binary operator

- Enter a number

In an actual testing situation, you may want to cycle through each binary operation to make sure they are correct. Similarly, you can do the same with unary operators.

It does get tricky when you start to plan for using parentheses. What test cases are best to use? This question will be your constant challenge. It's akin to asking "When are we done testing?". The answer is "It depends". It depends on how much time you have available, the level of quality that you are attempting to achieve, how much coverage of your test requirements you want to achieve, etc.

Now suppose that we've had to make these hard decisions and have come up with the following test cases based on using the model we have built. Note that these are not the actual tests, only the test cases. The tests (or test procedures) include specific data and specific validations.

- Verify unary operators

- Verify binary operators

- Verify nested calculations with both unary and binary operators

- one level of nested calculations (e.g. "500 - (45 / 5)" )

- two levels of nested calculations (e.g. "sqrt (a^2 + b^2)" )

- 25 levels of nested calculations

- verify calculations where operators are replaced by different operators (e.g. enter '2' then '+' then 'x' then '5' and verify it is interpreted as "2 x 5" )

There are certainly more test cases, we have several test requirments that are not covered by these tests. You'll want to make sure that you are covering all the requirements you have identified or be able to explain why tests are not warranted.

These will suffice for now. You can use an automated tool to generate tests based on your model, or you can pick them manually. Initially, I'd suggest doing it manally so that you can see how the automated tool is such a benefit.

Step 4 - Write Tests

There are many different approaches to writing the tests. Depending on factors such as time available, will the same person write it as will run it, what is the knowledge of the person running the test, etc. I'm going to suggest the following format because it's simple and covers just enough to be helpful and not so much to be overly cubmersome.

In a spreadsheet, create the following columns

- Test Name

- Step

- Expected Results

- Actual Results

- Pass/Fail

Now under the row with these headings, but the name of the fist test case. Under that, describe the action you want the tester to take. The action should be unambiguous, specific and only contain an action and not any validation. In the Expected Results column, describe what you expect to happen. Again, be unambiguous and specific. Leave the last two columns for later.

Here is an example using our test cases from above.

...

You may have noticed that on the second test, I have a step without an Expected Result. In this case, I don't expect anything to happen. I could say that, or I can leave it empty. You will need to determine which way works best for you.

Those that have an opnion on the matter may take exception to these tests as being poor examples of test cases, but for now, they will do.

We're almost done. To be continued.

Regression Testing Without a QA team - Part I

Many of you work in an environment where there are no QA people assigned to your software product. So when a new version is released, it's usually up to the developers or other non-QA staff to make sure the release is ready.

Usually, this ends up being the same people spending a few days with the new version to make sure nothing seems wrong. But wouldn't it be better to have some insight into what parts of the system were tested and what parts weren't?

Let's walk through a basic strategy for getting some insight into how you want to not only test your product, but how you want to report on those results. Even QA people forget to think about how they want to report their results and end up with a 1200 page document that nobody reads.

For the examples, I'll suppose you are on the team that tests the Calculator application that comes with Windows OS and you are preparing to get it ready for Windows Vista based on the Windows XP version. There are no new visible features to speak of. But there is a new API, Themes, etc. to take into consideration.

Step 1 - Develop a model of the system functionality

There is no single way to do this, but here are some suggestions and considerations.

Think of the 'ilities'

• Usability
• Functionality
• Security
• Performance
• Release (Backwards-Compatibility/Upgrade/Installation)

Think of how the product is organized:

• Are there specific workflows that are helpful to use to organize the list of funcitonality?
• Would it be helpful to organize by screen/page/mode? If it's a website, is it helpful to organize functionality by web page? If it is a command-line applicaiton, is it helpful to organize by comand-line switch?
• If there are multiple components, is it helpful to organize by component (or group of components)?

Keep in mind, that you will usually have a blend of strategies in the end.

For the calculator example, we need to include the scientific and statistical functions.

List of System Functionality (aka Test Requirements)

Functionality

- Perform Standard Calculation

- Perform Additional Scientific Calculations

- Perform Additional Statistical Calculations

- Perform Support Functions

- Support Copy/Paste

- Support Digit Grouping Function

- "About" functionality changes for Vista

Usability

- Support OS Themes

- Support OS Color schemes

- Support OS Font Size settings

- Support 'each' Vista version

NOTE

But there's also a question of detail. You can list test requirements in varying degrees of detail. I've been on both sides of the equation. Too many requirements can be difficult to maintain. Too few details can tend to be useless. You will have to determine what makes sense to you and be prepared to make changes accordingly.

You will also want to think of how you want to report these issues. It may be helpful to use some subset of these test requirements to group test results. Again, this will be dependent on your needs.

Step 2 - Identify 'high risk' areas of functionality

Not all features are created equal. Some workflows are used daily and others are used infrequently. Those used daily likely pose a higher risk to the product if they don't work properly. So-called 'core functionality' is good candidate as well. You will want to come up with a scale for the risk rating. In addition to the scale, it may be helpful to identify the criteria used to justify this risk rating.

For our example, here is the risk rating we could use.

1 - Without this feature, the product would be rejected by the users

2 - Without this feature, the product would be used rarely by the users

3 - Without this feature, the product would be used, but with reservations

4 - Unlikely to affect usage of this product

Here is how we could apply the risk rating to our test requirements.

Functionality

- Perform Standard Calculation (risk - 1)

- Perform Additional Scientific Calculations (risk - 2)

- Perform Additional Statistical Calculations (risk - 2)

- Perform Support Functions

- Support Copy/Paste (risk - 3)

- Support Digit Grouping Function (risk - 4)

- "About" functionality changes for Vista (risk - 4)

Usability

- Support OS Themes (risk - 3)

- Support OS Color schemes (risk - 3)

- Support OS Font Size settings (risk - 3)

- Support 'each' Vista version (risk - 2)

NOTE

You will want to make sure that there is somewhat of an even distribution in your risk levels. Too many items in one level means that you'll want to discriminate further items at that level. We will use this risk rating later.

Wow, this is getting long. I'll post what I have now and pick up on this in a day or so.

OKC Professional Groups

I had an opportunity to attend Refresh OKC's July meeting. They are a group of web developers and designers that meet on topics that generally tie into their work, but don't limit themselves to striclty web design/development.

If this is of any interest to you, check out their website.

Also, there is a information security group DC405 that meets monthly. I haven't checked them out, but hope to in the next month or so.

What other groups would you recommend?

Meeting Announcement - Thursday, July 23rd 11:30am

Time and Location

The Red Earth QA's meeting will be held on the 3rd floor of 100 N. Broadway from 11:30am-1pm on Thursday, June 25th. Look for the signs to direct you to the correct room.

Topic

Frank Roland will talk about his quest to prove that his Soduku solving application works as it supposed to.

Directions

• You can park in Main Street Parking on Main or you can find street parking.
• From I-40, take the Robinson Exit. Go North on Robinson to Main. Right on Main. You can either go to Main Street Parking or continue to Santa Fe Parking. You will see 100 N Broadway on your left across Broadway. The building says 'Chase' at the top.

A Child's View of Debugging

This summer, a friend of mine asked me if my eight year old son would be interested in a Summer Robotics Club. Completely thinking of my son (wink wink), I agreed.

The group meets about every other weekend over the summer covering various topics such as basic theory of atoms, introductory programming and of course, assembling a robot from a kit.

It was my job to do several sessions on introductory programming skills. We used a program called Alice, a free tool for learning programming skills by building 3D animations and even some simple games.

The first session was to go through a tutorial of the User Interface and modify a simple pre-made animation. The second session was troubleshooting a 'broken' animation.

The 'broken' animation showed a top view of an airport, with a single car in the parking lot. The car attempts to drive around the airport, but it goes through the building at one point. It also goes 'too fast' at other times.

Here is one solution by one team of kids. Note that these two were nine years old.

Here are some observations from this solution. They were time-limited, and didn't quite finish.

- The car ends up going off the screen at a couple points
- The car speed issue wasn't addressed
- In the original animation, the car ended up at the exact same spot it started from. In each solution, including this one, the car ended up in a different location (even if just slightly).
- etc.

Overall, I saw some very familiar themes in the whole process. See if any of these sound familiar.

Kids
The car ends up going off the screen at a couple points
Adults
Some attempts at repair cause other issues, even if the attempts don't have any impact on the issue being solved.

Kids
The car speed issue wasn't addressed
Adults
Some bugs will never get fixed.

Kids
In the original animation, the car ended up at the exact same spot it started from. In each solution, including this one, the car ended up in a different location (even if just slightly).
Adults
Sometimes, we break existing functionality when we fix issues. Functionality that isn't listed as a requirement is the most succeptible.

Kids
Note the pause at the end of the video, followed by a short drive forward. This was an attempt to get the car closer to the original location.
Adults
Not all solutions are elegant. It may get the job done, but it could be done more easily.

Kids
Is this close enough?" was a question I got in regards to getting the car back to it's starting point.
Adults
While it may be possible to get an 'exact' solution, 'good enough' may be all that is done.

Kids
"Just a minute! I'm almost done!" was a protest I heard when I told them we needed to move on.
Adults
Debugging is difficult to estimate.
Everyone wants to do a good job and finish the task given.

Kids
In the beginning, changes were being made to the wrong part of the code. There wasn't a clear understanding of where the actual issue was. When a change was made to the wrong part of the code and the animation didn't change as they expected, they blamed the computer for not working right.
Adults
Sometimes we get stumped by a problem. We think we are doing the right thing, but we don't see any progress.

Overall, we had a great time. We still have more sessoins left, so there may be more stories.