Free e-book!

You can download your own free copy of Tester Types. It's on a site called Software Testing Club, which may also be a good resource.

Easy Reader

Google's Reader is an excellent way to keep up with blogs, especially blogs that don't update daily.

Here is a list of the QA blogs that I track that you can import and have a single place to read all about QA in one place.

Enjoy!

Happy Holidays!

Here is a certificate for you or someone you work with. It 'allows the bearer to exchange one line of bad code for one line of good code' so that you can say that you have no bad code.

Code Offset Certificate

Rather than printing your own, you can actually buy a different one online and the proceeds go to supporting open source software.

Enjoy!

Meeting Announcement - (AT DEVON) Dec 17th

NEWS
- We're back at Devon this month.

Time and Location

The Red Earth QA's meeting will be held on the 3rd floor of 100 N. Broadway from 11:30am-1pm on Thursday, December 17th. Look for the signs to direct you to the correct room.

Topic

We will be having our annual holiday gathering. Bring your favorite treat to share with the group along with your lunch.

We will be having a planning session for 2010.

Directions

• You can park in Main Street Parking on Main or you can find street parking.
• From I-40, take the Robinson Exit. Go North on Robinson to Main. Right on Main. You can either go to Main Street Parking or continue to Santa Fe Parking. You will see 100 N Broadway on your left across Broadway. The building says 'Chase' at the top.

November Meeting Cancelled

Sorry for the mixup. We will not be having our November meeting.

Do you work with a Machiavellian Manager?

Suppose we 'update' Machiavelli's "The Prince" to be about corporate leaders instead of about being a prince. Here is how I could summarize his book.

Please firmly place your tounge in your cheek as you read this.

CONCERNING THE WAY IN WHICH THE STRENGTH OF ALL DEPARTMENTS OUGHT TO BE MEASURED

It is necessary to consider a point in examining the character of departments; that is, whether a manager has such power that, in case of need, he can support himself with his own budget and other resources, or whether he has always need of the assistance of others. And to make this quite clear I say that I consider those who are able to support themselves by their own resources who can, either by abundance of men or money, raise a sufficient strategy to confront anyone who comes to usurp them.

CONCERNING THINGS FOR WHICH MEN, AND ESPECIALLY MANAGERS, ARE PRAISED OR BLAMED

It remains now to see what ought to be the rules of conduct for a manager towards staff and peers. And as I know that many have written on this point, I expect I shall be considered presumptuous in mentioning it again, especially as in discussing it I shall depart from the methods of other people. But, it being my intention to write a thing which shall be useful to him who apprehends it, it appears to be more appropriate to follow the real truth of the matter than the imagination of it; for many have pictured companies and departments which in fact have never been known or seen, because how one lives is so far distant from how one ought to live, that he who neglects what is done for what ought to be done, sooner effects his ruin than his preservation; for a man who wishes to act entirely up to his professions of virtue soon meets with what destroys him among so much that is evil.

Hence it is necessary for a manager wishing to hold his own to know how to do wrong, and to make use of it or not according to necessity. Therefore, putting on one side imaginary things concerning a manager, and discussing those which are real, I say that all men when they are spoken of, and chiefly managers for being more highly placed, are remarkable for some of those qualities which bring them either blame or praise; and thus it is that one is reputed kind, another miserly; one cruel, one compassionate; one sincere, another cunning; one hard, another easy, and the like. And I know that every one will confess that it would be most praiseworthy in a manager to exhibit all the above qualities that are considered good; but because they can neither be entirely possessed nor observed, for human conditions do not permit it, it is necessary for him to be sufficiently prudent that he may know how to avoid the reproach of those vices which would lose him his status; and also to keep himself, if it be possible, from those which would not lose him it; but this not being possible, he may with less hesitation abandon himself to them. And again, he need not make himself uneasy at incurring a reproach for those vices without which the status can only be saved with difficulty, for if everything is considered carefully, it will be found that something which looks like virtue, if followed, would be his ruin; whilst something else, which looks like vice, yet followed brings him security and prosperity.

CONCERNING KINDNESS AND MEANNESS

Commencing then with the first of the above-named characteristics, I say that it would be well to be reputed kind. Nevertheless, kindness exercised in a way that does not bring you the reputation for it, injures you; for if one exercises it honestly and as it should be exercised, it may not become known, and you will not avoid the reporach of its opposite. Therfore anyone wishing to maintain among men the name of 'kindly' is obliged to avoid no attribute of magnificence; so that a manager thus inclined will consume in such acts all his resources, and will be compelled in the end, if he wish to maintain the name of 'kindly', to unduly weigh down his staff, take away their perks and do everything he can to increase his budget. This will soon make him odious to his subjects, and having his budget cut he will be little valued by any one; thus, with his kindness, having offended many and rewarded few, he is affected by the very first trouble and imperilled by whaterver may be the first danger; recognizing this himself, and wishing to draw back from it, he runs at once into the reproach of being miserly.

Therefore, a manager, not being able to exercise this virtue of kindness in such a way that it is recognized, except to his cost, if he is wise he ought not to fear the reputation of being mean, for in time he will come to be more considered if he were thought of as kind.

CONCERNING THE WAY IN WHICH MANAGERS SHOULD VIEW CORPORATE LAW

Every one admits how praiseworthy it is in a manager to follow legal guidelines, and to live with integrity and not with skill. Nevertheless our experience has been that those managers who have done great things have followed the law to little account, and have known how to circumvent the intellect of men by skill, and in the end have overcome those who have relied on their word. You mush know that there are two ways of increasing dominion the one by the law, the other by force. If men were entirely good this precept would not hold, but because they are bad, and will not work according to the law with you, you too are not bound to work with integrity with them.

But it is necessary to know well how to disguise this characteristic, and to be a great pretender and dissembler; and men are so simple, and so subject to present necessities, that he who seeks to deceive will always find someone who will allow himself to be deceived.

THAT ONE SHOULD AVOID BEING DESPISED AND HATED

The manager must consider, as has been in part said before, how to avoid those things which will make him hated or contemptible; and as often as he shall have succeeded he will have fulfilled his part, and he need not fear any danger in other reproaches.

That manager is highly esteemed who conveys this impression of himself, and he who is highly esteemed is not easily conspired against; for, provided it is well known that he is an excellent man and revered by his staff, he can only be attacked with difficulty. For this reason a manager ought to have two fears, one from within, on account of his staff, the other from without, on account of external powers. From the latter he is defended by having strong sanctions for violating cross-departmental policies and having good allies, and if he has strong sanctions he will have good friends, and affairs will always remain quiet within when they are quiet without, unless they should have been already disturbed by conspiracy; and even should affairs outside be disturbed, if he has carried out his preparations and has lived as I have said, as long as he does not despair, he will resist every attack. I consider that a manager ought to reckon conspiracies of little account when his staff hold him in esteem; but when it is hostile to him, and bears hatred towards him, he ought to fear everything and everybody.

ARE BURDENSOME CROSS-DEPARTMENTAL PROCEDURES, AND MANY OTHER THINGS TO WHICH MANAGERS OFTEN RESORT, ADVANTAGEOUS OR HURTFUL?

Some managers, so as to hold securely their department, have removed all decision-making powers from their staff; others have kept their teams distracted by in-fighting; others have fostered enmities against themselves; others have laid themselves out to gain over those whom they distrusted when they first became managers; some have built burdensome cross-departmental procedures; some have ignored and bypassed other's procedures. And although one cannot give a final judgment on all of these things unless one possesses the particulars of those departments in which a decision has to be made, nevertheless I will speak as comprehensively as the matter of itself will admit.

There never was a new manager who has taken away decision-making powers from his staff; rather when he has found them without the power to make decisions he has always granted this to them, because, by allowing them to make decisions, those decisions become yours, those men who were distrusted become faithful, and those who were faithful are kept so, and your

staff become your adherents.

Our predecessors, and those who were reckoned wise, were accustomed to say that it was necessary to foster quarrels in some of their teams so as to keep possession of them the more easily. This may have been well enough in those times when the company was in a way balanced, but I do not believe that it can be accepted as a precept for to-day, because I do not believe that factions can ever be of use; rather it is certain that when the enemy comes upon you in divided teams you are quickly lost, because the weakest party will always assist the outside forces and the other will not be able to resist.

It has been a custom with managers, in order to hold their departments more securely, to build over-burdensome cross-departmental policies and procedures that may serve as a bridle and bit to those who might desire to use their resources, and as a place of refuge from a first attack. I praise this system because it has been made use of formerly.

However the best possible policy is--not to be hated by the staff, because, although you may successful in enforcing these over-burdensome policies, yet they will not save you if the staff hate you, for there will never be wanting other managers to assist staff who have complained against you.

All these things considered then, I shall praise him who builds over-burdensome policies as well as him who does not, and I shall blame whoever, trusting in them, cares little about being hated by the people.

HOW A MANAGER SHOULD CONDUCT HIMSELF SO AS TO GAIN RENOWN

Nothing makes a manager so much esteemed as great enterprises and setting a fine example. And a manager ought, above all things, always endeavour in every action to gain for himself the reputation of being a great and remarkable man.

A manager is also respected when he is either a true friend or a downright enemy, that is to say, when, without any reservation, he declares himself in favour of one party against the other; which course will always be more advantageous than standing neutral; because if two of your powerful peers are at odds on some topic, they are of such a character that, if one of them wins, you have either to fear him or not. In either case it will always be more advantageous for you to declare yourself and to make sides strenuously; because, in the first case, if you do not declare yourself, you will invariably fall a prey to the winner, to the pleasure and satisfaction of him who has lost, and you will have no reasons to offer, nor anything to protect or to shelter you. Because he who wins does not want doubtful friends who will not aid him in the time of trial; and he who loses will not harbour you because you did not willingly court his fate.

A manager ought also to show himself a patron of ability, and to honor the proficient in every skill. At the same time he should encourage his staff to perform their work peaceably so that no one should be deterred from performing their duties for fear lest they be taken away from him or another from being hired because of poor benefits; but the manager ought to offer rewards to whoever wishes to do these things and designs in any way to honor his department.

Further, he ought to entertain the people with celebrations and parties at convenient seasons of the year; and as his department is divided into teams he ought to hold such bodies in esteem, and associate with them sometimes, and show himself an example of courtesy and kindness; nevertheless, always maintaining the majesty of his rank, for this he must never consent to abate in anything.

CONCERNING THE DIRECT REPORTS OF MANAGERS

The choice of direct reports is of no little importance to a manager, and they are good or not according to the discrimination of the manager. And the first opinion which one forms of a manager, and of his understanding, is by observing the men he has around him; and when they are capable and faithful he may always be considered wise, because he has known how to recognize the capable and to keep them faithful. But when they are otherwise one cannot form a good opinion of him, for the prime error which he made was in choosing them.

Prologue

There are many things that were left out. Some for space, some because I cringed as I was re-writing them. The cringing was because they hit pretty close to home. Overall, this should be read as a 'How Not To' rather than a 'How-To'.

"These are a few of my favorite techniques" slides and examples

You can download the slides and examples from the presentation here.

I hope you enjoy them. If you have any questions, please email me at robert@watkins.net

September Meeting Video

One of the perks of being at the okcCoCo is that we get a video of our meetings recorded and available online. Here is our September meeting. Enjoy!

Pex - automated white-box testing in .NET

For all you developers out there that are looking for unit testing tools, this may be worth looking into.

"Right from the Visual Studio code editor, Pex finds interesting input-output values of your methods, which you can save as a small test suite with high code coverage. Pex performs a systematic analysis, hunting for boundary conditions, exceptions and assertion failures, which you can debug right away. Pex enables Parameterized Unit Testing, an extension of Unit Testing that reduces test maintenance costs. Pex also comes with a lightweight framework for test stubs and detours, called Stubs and Moles."

http://research.microsoft.com/en-us/projects/pex/

Meeting Announcement- BRING A LUNCH! - Thursday, Oct 22nd 11:30am

Time and Location

The Red Earth QA's meeting will be held 723 N. Hudson Ave at our new location! We are gathering at Thursday, October 22nd at 11:30, the presentation starts at noon.

Topic: These Are a Few of My Favorite Techniques

Robert Watkins will be discussing some of his favorite testing tools/techniques/models including

- State-Transition Diagrams (manual and automated using Spec Explorer)

- Pairwise Testing (using allpairs)

- Truth Tables

- Domain Testing

The discussion will walk through several examples of:

- identifying what needs to be tested

- creating models

- using these models to develop tests

Can't attend in person?

We will record this session and post it online.

Directions/
The okcCoCo is located in midtown Oklahoma City at the intersection of 7th Street and Hudson Avenue.

Immediate access can be had from I-235 via 6th Street exists 1E and 1F. Take 6th Street West to Hudson Avenue, turn North onto Hudson Avenue and continue one block North to 7th Street.

The okcCoCo is located on the South West corner of 7th Street and Hudson Avenue.

Parking is available in four lots (see link). The primary parking lot has direct access to the okcCoCo property and is accessible from 7th Street. Auxiliary parking lots are accessible from 6th, 7th and 8th Street. One and two hour parking as well as metered parking are available along Hudson Ave, as well as along 6th, 7th, and 8th Streets.

Free Online Training!

http://now.eloqua.com/e/es.aspx?s=1156&e=7482&elq=e0ee27eca09b4976b94518322ee8016f

Live Web Seminar

"Delivering Business Value Through Exploratory Testing"

Join us - September 29, 2009

2:00 p.m. ET/11 a.m. PT

Exploratory testing is an often underutilized tool in software quality. Many organizations have trouble incorporating exploratory methods into their test strategy because it is viewed as "unstructured" or difficult to track back to productivity. However, given the complexity of today's systems and the difficultly of defining their expected state, outcomes, and use cases in advance, exploratory testing is a key tool in the software quality arsenal.

In this Web seminar, Michael Bolton and Chad Wathington will discuss how exploratory testing is particularly applicable to delivering business value in agile teams. They'll also cover:

• How to approach exploratory testing
• How to free up and structure time to use exploratory methods
• How to justify moving away from just test cases in your organization

"Feature Creep", the song and others...

http://tester.geordiekeitt.com/archives/75

Very funny song about feature creep...

Others
http://tester.geordiekeitt.com/archives/77

http://tester.geordiekeitt.com/archives/91

Meeting Announcement- NEW LOCATION! - Thursday, Sept 24th 11:30am

Time and Location

The Red Earth QA's meeting will be held 723 N. Hudson Ave at our new location! We are gathering at Thursday, September 24th at 11:30, the presentation starts at noon.

Topic: What’s New in Quality Center 10.0?

This informative session will highlight the new features and functionality included in the Quality Center 10.0 release including Quality Center Enterprise, Quality Center Premier, Requirements Management, Business Process Testing and Functional Testing. It will also cover not only the key integrations between the products but also new integrations with other products in the HP BTO portfolio and third party products

Can't attend in person?

Bridgeline: Dial In: 1.866.409.2889 (US); 1.702.696.4520 (Outside US); Participant: 6913575

Virtual Room Keys:

https://www.rooms.hp.com/attend/default.aspx?key=EPAADF7CCH

Directions
The okcCoCo is located in midtown Oklahoma City at the intersection of 7th Street and Hudson Avenue.

Immediate access can be had from I-235 via 6th Street exists 1E and 1F. Take 6th Street West to Hudson Avenue, turn North onto Hudson Avenue and continue one block North to 7th Street.

The okcCoCo is located on the South West corner of 7th Street and Hudson Avenue.

Parking is available in four lots (see link). The primary parking lot has direct access to the okcCoCo property and is accessible from 7th Street. Auxiliary parking lots are accessible from 6th, 7th and 8th Street. One and two hour parking as well as metered parking are available along Hudson Ave, as well as along 6th, 7th, and 8th Streets.

NEW MEETING LOCATION!

We're very excited to be meeting in a new location for our September meeting! The new location is the OKC Coworking Collaborative (aka OKC CoCo) in midtown OKC at Hudson and 7th.

We're still working on the topic for the meeting and our invitation should be out soon.

Meanwhile, you can check out their space and parking information.

We hope you can make it!

Meeting Announcement - Friday, August 28th 11:30am

Time and Location

The Red Earth QA's meeting will be held on the 3rd floor of 100 N. Broadway from 11:30am-1pm on FRIDAY, August 28th. Look for the signs to direct you to the correct room.

Topic

Devon will provide a view of their use of Silk Performer in their environment.

Directions

• You can park in Main Street Parking on Main or you can find street parking.
• From I-40, take the Robinson Exit. Go North on Robinson to Main. Right on Main. You can either go to Main Street Parking or continue to Santa Fe Parking. You will see 100 N Broadway on your left across Broadway. The building says 'Chase' at the top.

Regression Testing Without a QA team - Part II

Don't worry, while there's more to do, we are well positioned to make the most efficient use of our time with the remainder of the tasks.

Step 3 - Identify test cases / test data

For the purposes of this discussion, let's limit the concept of a 'test case' to a 1-2 sentence that describes the goal of each test you want to achieve. Start with the high-risk areas first. Depending on your situation, you may do only the risk level 1 items, or you may have time to do more. The answer depends on available time. Once you've built some test cases, you'll want to take a step back and build some models of your system to see if there are any more test cases you could write.

You'll also want to think about test data. You'll want to consider valid data, invalid data, large quantities of data, lack of data, etc.

For our example with the calculator, here are some preliminary test cases.

1. Perform valid basic arithmetic calculations
2. Perform 'invalid' basic arithmetic calculations (divide by 0, invalid input, etc.)
3. Perform scientific calculations
4. Perform 'invalid' scientific calculations (log 0, tan 90, etc.)
5. Perform statistical calculations

Hmmm.... While I was using the application to identify test cases, I realized that I've missed some of the features

- keyboard shortcuts (risk - 3)

- number format -decimal, octal, etc. (risk - 2)

- number type -degree, radian, grad (risk -2)

- use parentheses up to 25 levels at a time (risk -2)

The Model

Now that I've gotten into this a bit further, it seems that I'm ready to build a model to make sure that I'm covering these requirements. There are several types of models. Dataflow diagram, Workflow diagram, State Transition diagram, etc. At least one would be helpful, so here is one that I came up with.

State - Ready for a Number

Actions

- Enter a '('.

- Enter a 'binary' operator (a 'binary' operator takes two inputs like +, *, etc.)

- Enter a number,

State - Ready for an Operator

Actions

- Enter a ')'

- Enter a number

- Enter a 'unary' operator (a 'unary' operator takes one input like 1/x, sin x, etc.)

- Enter a 'binary' operator

As with all models, this model doesn't tell the whole story, but it is better than having no model. Among it's faults are that it does not cover the statisical mode calculations, the names of the states are not completely accurate, only valid actions are allowed, etc. You will have to get used to these sort of issues to be able to have a strong basis for testing available in a short time.

Planning for Test Cases

For now, let's assume that the starting state of the calculator is 'Ready for a Number'. I have three valid actions, enter a '(', a number or a binary operator. Suppose I choose to enter a number, then my state has changed to 'Ready for an Operator'. In this new state, I have four valid actions (listed above). You will quickly see that there are lots of paths I can follow where I select actions and that action may or may not change the state I'm in (though it will change the value of the calculation being displayed and stored). You can also see that there is no actual ending state, you can theoretically keep doing calculations idefinately.

Here's one example path to follow

- Enter a number

- Enter a binary operator

- Enter a number

In an actual testing situation, you may want to cycle through each binary operation to make sure they are correct. Similarly, you can do the same with unary operators.

It does get tricky when you start to plan for using parentheses. What test cases are best to use? This question will be your constant challenge. It's akin to asking "When are we done testing?". The answer is "It depends". It depends on how much time you have available, the level of quality that you are attempting to achieve, how much coverage of your test requirements you want to achieve, etc.

Now suppose that we've had to make these hard decisions and have come up with the following test cases based on using the model we have built. Note that these are not the actual tests, only the test cases. The tests (or test procedures) include specific data and specific validations.

- Verify unary operators

- Verify binary operators

- Verify nested calculations with both unary and binary operators

- one level of nested calculations (e.g. "500 - (45 / 5)" )

- two levels of nested calculations (e.g. "sqrt (a^2 + b^2)" )

- 25 levels of nested calculations

- verify calculations where operators are replaced by different operators (e.g. enter '2' then '+' then 'x' then '5' and verify it is interpreted as "2 x 5" )

There are certainly more test cases, we have several test requirments that are not covered by these tests. You'll want to make sure that you are covering all the requirements you have identified or be able to explain why tests are not warranted.

These will suffice for now. You can use an automated tool to generate tests based on your model, or you can pick them manually. Initially, I'd suggest doing it manally so that you can see how the automated tool is such a benefit.

Step 4 - Write Tests

There are many different approaches to writing the tests. Depending on factors such as time available, will the same person write it as will run it, what is the knowledge of the person running the test, etc. I'm going to suggest the following format because it's simple and covers just enough to be helpful and not so much to be overly cubmersome.

In a spreadsheet, create the following columns

- Test Name

- Step

- Expected Results

- Actual Results

- Pass/Fail

Now under the row with these headings, but the name of the fist test case. Under that, describe the action you want the tester to take. The action should be unambiguous, specific and only contain an action and not any validation. In the Expected Results column, describe what you expect to happen. Again, be unambiguous and specific. Leave the last two columns for later.

Here is an example using our test cases from above.

...

You may have noticed that on the second test, I have a step without an Expected Result. In this case, I don't expect anything to happen. I could say that, or I can leave it empty. You will need to determine which way works best for you.

Those that have an opnion on the matter may take exception to these tests as being poor examples of test cases, but for now, they will do.

We're almost done. To be continued.

Regression Testing Without a QA team - Part I

Many of you work in an environment where there are no QA people assigned to your software product. So when a new version is released, it's usually up to the developers or other non-QA staff to make sure the release is ready.

Usually, this ends up being the same people spending a few days with the new version to make sure nothing seems wrong. But wouldn't it be better to have some insight into what parts of the system were tested and what parts weren't?

Let's walk through a basic strategy for getting some insight into how you want to not only test your product, but how you want to report on those results. Even QA people forget to think about how they want to report their results and end up with a 1200 page document that nobody reads.

For the examples, I'll suppose you are on the team that tests the Calculator application that comes with Windows OS and you are preparing to get it ready for Windows Vista based on the Windows XP version. There are no new visible features to speak of. But there is a new API, Themes, etc. to take into consideration.

Step 1 - Develop a model of the system functionality

There is no single way to do this, but here are some suggestions and considerations.

Think of the 'ilities'

• Usability
• Functionality
• Security
• Performance
• Release (Backwards-Compatibility/Upgrade/Installation)

Think of how the product is organized:

• Are there specific workflows that are helpful to use to organize the list of funcitonality?
• Would it be helpful to organize by screen/page/mode? If it's a website, is it helpful to organize functionality by web page? If it is a command-line applicaiton, is it helpful to organize by comand-line switch?
• If there are multiple components, is it helpful to organize by component (or group of components)?

Keep in mind, that you will usually have a blend of strategies in the end.

For the calculator example, we need to include the scientific and statistical functions.

List of System Functionality (aka Test Requirements)

Functionality

- Perform Standard Calculation

- Perform Additional Scientific Calculations

- Perform Additional Statistical Calculations

- Perform Support Functions

- Support Copy/Paste

- Support Digit Grouping Function

- "About" functionality changes for Vista

Usability

- Support OS Themes

- Support OS Color schemes

- Support OS Font Size settings

- Support 'each' Vista version

NOTE

But there's also a question of detail. You can list test requirements in varying degrees of detail. I've been on both sides of the equation. Too many requirements can be difficult to maintain. Too few details can tend to be useless. You will have to determine what makes sense to you and be prepared to make changes accordingly.

You will also want to think of how you want to report these issues. It may be helpful to use some subset of these test requirements to group test results. Again, this will be dependent on your needs.

Step 2 - Identify 'high risk' areas of functionality

Not all features are created equal. Some workflows are used daily and others are used infrequently. Those used daily likely pose a higher risk to the product if they don't work properly. So-called 'core functionality' is good candidate as well. You will want to come up with a scale for the risk rating. In addition to the scale, it may be helpful to identify the criteria used to justify this risk rating.

For our example, here is the risk rating we could use.

1 - Without this feature, the product would be rejected by the users

2 - Without this feature, the product would be used rarely by the users

3 - Without this feature, the product would be used, but with reservations

4 - Unlikely to affect usage of this product

Here is how we could apply the risk rating to our test requirements.

Functionality

- Perform Standard Calculation (risk - 1)

- Perform Additional Scientific Calculations (risk - 2)

- Perform Additional Statistical Calculations (risk - 2)

- Perform Support Functions

- Support Copy/Paste (risk - 3)

- Support Digit Grouping Function (risk - 4)

- "About" functionality changes for Vista (risk - 4)

Usability

- Support OS Themes (risk - 3)

- Support OS Color schemes (risk - 3)

- Support OS Font Size settings (risk - 3)

- Support 'each' Vista version (risk - 2)

NOTE

You will want to make sure that there is somewhat of an even distribution in your risk levels. Too many items in one level means that you'll want to discriminate further items at that level. We will use this risk rating later.

Wow, this is getting long. I'll post what I have now and pick up on this in a day or so.

OKC Professional Groups

I had an opportunity to attend Refresh OKC's July meeting. They are a group of web developers and designers that meet on topics that generally tie into their work, but don't limit themselves to striclty web design/development.

If this is of any interest to you, check out their website.

Also, there is a information security group DC405 that meets monthly. I haven't checked them out, but hope to in the next month or so.

What other groups would you recommend?

Meeting Announcement - Thursday, July 23rd 11:30am

Time and Location

The Red Earth QA's meeting will be held on the 3rd floor of 100 N. Broadway from 11:30am-1pm on Thursday, June 25th. Look for the signs to direct you to the correct room.

Topic

Frank Roland will talk about his quest to prove that his Soduku solving application works as it supposed to.

Directions

• You can park in Main Street Parking on Main or you can find street parking.
• From I-40, take the Robinson Exit. Go North on Robinson to Main. Right on Main. You can either go to Main Street Parking or continue to Santa Fe Parking. You will see 100 N Broadway on your left across Broadway. The building says 'Chase' at the top.

A Child's View of Debugging

This summer, a friend of mine asked me if my eight year old son would be interested in a Summer Robotics Club. Completely thinking of my son (wink wink), I agreed.

The group meets about every other weekend over the summer covering various topics such as basic theory of atoms, introductory programming and of course, assembling a robot from a kit.

It was my job to do several sessions on introductory programming skills. We used a program called Alice, a free tool for learning programming skills by building 3D animations and even some simple games.

The first session was to go through a tutorial of the User Interface and modify a simple pre-made animation. The second session was troubleshooting a 'broken' animation.

The 'broken' animation showed a top view of an airport, with a single car in the parking lot. The car attempts to drive around the airport, but it goes through the building at one point. It also goes 'too fast' at other times.

Here is one solution by one team of kids. Note that these two were nine years old.

Here are some observations from this solution. They were time-limited, and didn't quite finish.

- The car ends up going off the screen at a couple points
- The car speed issue wasn't addressed
- In the original animation, the car ended up at the exact same spot it started from. In each solution, including this one, the car ended up in a different location (even if just slightly).
- etc.

Overall, I saw some very familiar themes in the whole process. See if any of these sound familiar.

Kids
The car ends up going off the screen at a couple points
Adults
Some attempts at repair cause other issues, even if the attempts don't have any impact on the issue being solved.

Kids
The car speed issue wasn't addressed
Adults
Some bugs will never get fixed.

Kids
In the original animation, the car ended up at the exact same spot it started from. In each solution, including this one, the car ended up in a different location (even if just slightly).
Adults
Sometimes, we break existing functionality when we fix issues. Functionality that isn't listed as a requirement is the most succeptible.

Kids
Note the pause at the end of the video, followed by a short drive forward. This was an attempt to get the car closer to the original location.
Adults
Not all solutions are elegant. It may get the job done, but it could be done more easily.

Kids
Is this close enough?" was a question I got in regards to getting the car back to it's starting point.
Adults
While it may be possible to get an 'exact' solution, 'good enough' may be all that is done.

Kids
"Just a minute! I'm almost done!" was a protest I heard when I told them we needed to move on.
Adults
Debugging is difficult to estimate.
Everyone wants to do a good job and finish the task given.

Kids
In the beginning, changes were being made to the wrong part of the code. There wasn't a clear understanding of where the actual issue was. When a change was made to the wrong part of the code and the animation didn't change as they expected, they blamed the computer for not working right.
Adults
Sometimes we get stumped by a problem. We think we are doing the right thing, but we don't see any progress.

Overall, we had a great time. We still have more sessoins left, so there may be more stories.

Spec Explorer - Validate your model and generate tests

Sounds like a tall order. Let's start at the end result, and then go through how we get there.

The specification we are describing here is for a product I work on where we would like system administrators to send requests to users in the form of messages. For this example, we are only looking at the state transition diagram for the messages.

What does Spec Explorer Give Me?
The following diagram and tests were auto-generated based on the model specified for Spec Explorer.

For now, note that the tests below are labeled 'test segment 0', 'test segment 1' etc. and that the transitions From S0 to S3 represent 'State 0' and 'State 3' respectively. The diagram correctly describes these states and I'll describe how to understand how 'State 0' is associated with the name 'NotCreated' later in this blog entry.

While the diagram isn't the easiest to follow, some deference should be given since it was generated by the model created in the spec. However, you should be able to see that the message starts in a 'Not Created' state, then is moved to 'Pending' etc. until it is ultimately in the 'Deleted' state.

The transitions are the actions taken to move from one state to another such as 'UserViewMessage' and 'CreateMessage'.

The test suite shows how to move through each path of this state diagram. In this particular case, we are able to go through all paths because we don't have any loops (such as you would get if you could take a 'RejectedByClient' message and move it back to 'Pending').

If we had such a model, the test suites would be selected using strategies built into Spec Explorer and could be used as a basis for testing.

We can also use Spec Explorer to ensure that our models were complete and accurate before we sent our Specs for review.

Who is Spec Explorer for?

Anyone that writes specs, interprets specs or implements specs can make use of this.

Business Analysts - This can auto-generate use cases and diagrams as well as validate that the model you intend on having implemented is complete and accurate.

Quality Assurance - This can be used to build more extensive models along with more detailed data tracking to generate test cases.

Developers - This can be used to validate technical designs and auto-generate unit tests including basic validation of those tests.

Depending on your desire to dig into the capabilities, you will get more or less from this tool.

How do I get started?

You can get it from Spec Explorer Site at Microsoft Research as well and copy the spec sample below.

Once you have installed Spec Explorer and launch it, create a new project.

click 'Next'

Select the 'Create Directory for Project' checkbox and then Next.

Select the "AsmL (Plain Text)" Category and "Empty Program" Template and click the "Finish" button. One feature that should be mentioned here is that we can embed the markup language into the Word documents used for specs and it will work the same. Spec Explorer will embed MS Word into the file edit window and you can modify the spec as you would normally do.

Replace the default text with the following:

enum MESSAGE_STATUSES NotCreated Pending ReceivedByCaptureEngine ReceivedByClient ViewedByClient Completed RejectedByClient Deleted var status as MESSAGE_STATUSES = NotCreated [Action] CreateMessage () require status = NotCreated status := Pending [Action] CERequestMessage () require status = Pending status := ReceivedByCaptureEngine [Action] ClientRequestMessage () require status = ReceivedByCaptureEngine status := ReceivedByClient [Action] UserViewMessage () require status = ReceivedByClient status := ViewedByClient [Action] UserReject () require status = ViewedByClient status := RejectedByClient [Action] UserComplete () require status = ViewedByClient status := Completed [Action] UserDelete () require ((status<> NotCreated) and (status <> Deleted)) status := Deleted Main ()

There are 3 main parts to this spec. Variables and Constants, Actions and the 'Main()' statement.

In the Variables and Constants section for this example, we define the Message Status options and define a variable to keep track of the specific state for a given message.

In the Actions section, we describe each action to the message.

The 'Main()' statement is a necessary part for Spec Explorer to do it's anlaysis. More complex usage of the tool will use this, but not this example.

To generate the graph, we need to do a couple things. First, we'll 'build' a reference implementation of the specification. To do this, click 'Project' -> 'Build'. You'll see messages in the 'Output' tab. Note that in the output, you'll see a hint for each action described.

Next, we'll need to run this reference implementation to generate the diagram. To do this, click 'Execute' -> 'Run'.

You'll get the 'Select Execution Goal' dialog.

Select 'FSM Generation' then 'OK'. (FSM = Finite State Machine). Remember the S0 and S1 references from the tests generated earlier? Here is how they are seen by default. In order to make the diagram easier to read, we need to associate each state with some value, in this example, we'll associate it with the variable 'status'.

Right-click on the diagram and select 'Node Label' then 'Custom Expression'.

Now, we have the diagram we are looking for. Next, to generate the tests. Simply click 'Test' -> 'Generate Test Suites' and you get the list of tests listed above.

What are some juicy details that can't be covered here?

Using Word Documents with the AsmL Styles is a simple way to ensure that the model is tied to it's diagram and other analysis. You can publish the word document to a document repository and still be able to pull it back into Spec Explorer for future analysis.

There are several markup languages you can use to generate these models, AsmL language and Spec#.

There are other tools to use as well to build and evaluate these models, NModel , dia2fsm and others.

This is not a deep dive into this area, but I'm sure there will be cases where this is helpful.

Meeting Announcement - June 25th

NEWS
- We've moved our meeting to the 4th Thursday of each month


Time and Location

The Red Earth QA's meeting will be held on the 3rd floor of 100 N. Broadway from 11:30am-1pm on Thursday, June 25th. Look for the signs to direct you to the correct room.

Topic

Jeff Stanley from Metavante will talk about his role in building a Test Technology team to support various QA teams with new tools.

Directions

• You can park in Main Street Parking on Main or you can find street parking.
• From I-40, take the Robinson Exit. Go North on Robinson to Main. Right on Main. You can either go to Main Street Parking or continue to Santa Fe Parking. You will see 100 N Broadway on your left across Broadway. The building says 'Chase' at the top.

Book Review - "Essential Software Test Design"

Earlier this year, I wrote a review of "Essential Software Test Design" for the StickyMinds Website.

http://www.stickyminds.com

It's on the front page this week, and will be found in the archives after this week.

http://www.stickyminds.com/books.asp?ObjectId=1148&Function=FEATUREDETAIL&ObjectType=BOOK

Using Root Cause Analysis for Process Improvement

A few weeks ago, I discussed the idea of Root Cause Analysis. The approach there works well when you want to analyze each individual issue and mitigate it. If you start to do this often, you'll find that to be somewhat unweildy. Further, you may want to see trends in common root causes over a time period.

I'll discuss how to take this information and do some basic analysis to find trends. That trend information can help identify the most common causes and most common areas that cause issues.

In order to do this tracking we need to generalize and standardize some of the fields already tracked. For example, the specification of "Function" and "Cause" should come from their own respective lists. This will allow these to be grouped and each occurance counted for analysis. Additionally, you may want to add other aspects such as version, or sub-function to do analysis on.

If you plan on identifying multiple causes for a given issue, one way to identify that is to replicate the row in the spreadsheet for each subsequent cause so that one individual issue will have one row for each identified cause.

You can open a sample spreadsheet in OpenOffice format and follow along if you would like to see how to take this spreadsheet and do some basic analysis.

This is a sample of not just a single application, but multiple applications. The Root Cause Analysis has already been performed and we're ready for the analysis.

Let's assume we want to count the causes by application feature (called Issue Area in the spreadsheet). In Open Office, you want to use the "Data Pilot" feature. (In Excel, it's called Pivot Table and Pivot Chart).

You'll get the configuration for the Data Pilot (This is nearly exactly how Excel does it as well)

Drag the "Root Cause" button to the "Row fields" area, the "Issue Area" button to the "Column fields" area and Issue ID to the "Data Fields" area.

By default, Open Office wants to sum the data field, we'll want to change that to count, so when we click on the "data field" area, we are prompted to select the function we want to perform. Let's select 'count'.

I don't prefer the default graph, so I delete it, select the data fields without the totals and create a new graph of type 'stacked'.

I can repeat that for cause count by version, cause count by application, overall cause count, etc.

Once I have all my graphs, then I can look for the things that are in most need of change. From the graph above (and reading the issue spreadsheet attached), you'll see that there was a huge issue for login that caused all kinds of havoc. The login issue was related to the main domain server going down due to a faulty network card. Depending on your situation, this may or may not be something that can be mitigated by process change. However, it does give a sense of where the most issues lie and justifies the need for specific changes.

Good luck!

Free e-book (until May 27th) - Defect Prevention

A colleague of mine told me about this book. It is a free download (though requires a signup to Microsoft's training site). Here is an overview and link.


Body:

The Practical Guide to Defect Prevention

This practical, hands-on guide captures, categorizes, and builds a process of best practices to help avoid creating defects during the development process—rather than fixing them after extensive analysis.

The Practical Guide to Defect Prevention

Part I Introduction to Defect Prevention
1 Defect Prevention
2 Defect Prevention Frameworks
3 The Economics of Defect Prevention

Part II Defect Detection Techniques
4 Quality and the Development Process
5 Using Productivity Games to Prevent Defects
6 Improving the Testability of Software

Part III Defect Analysis Techniques
7 Software Measurement and Metrics
8 Risk Analysis
9 Using Simulation and Modeling for Organizational Innovation
10 Defect Taxonomies
11 Root Cause Analysis

Part IV Defect Prevention Techniques
12 Adopting Processes
13 FMEA, FTA, and Failure Modeling
14 Prevention Tab.

Part V A Culture of Prevention
15 Scenario Voting
16 Creating a Quality Culture
17 Moving Quality Upstream
18 Rewards, Motivation, and Incentives
19 Knowledge Management and Communication
20 Pulling It All Together

Meeting Announcement - May 28th

NEWS
- We've moved our meeting to the 4th Thursday this month. (It may be permanent)


Time and Location

The Red Earth QA's meeting will be held on the 3rd floor of 100 N. Broadway from 11:30am-1pm on Thursday, May 28th. Look for the signs to direct you to the correct room.

Topic

Michael Penny from Tek Systems will provide some insight into the local job market.

We have some unconfirmed additions as well. So come and be surprised!

Directions

• You can park in Main Street Parking on Main or you can find street parking.
• From I-40, take the Robinson Exit. Go North on Robinson to Main. Right on Main. You can either go to Main Street Parking or continue to Santa Fe Parking. You will see 100 N Broadway on your left across Broadway. The building says 'Chase' at the top.

Creating User Interfaces - Design Patterns

Do your users complain about your application, even though it gets the job done?

Are you asked to 'spice it up' or make it 'Web 2.0-ish'?

Well Design Patterns are for you! Here are some helpful links (though you can find your own easily) to inspire design of User Interfaces.

 

http://developer.yahoo.com/ypatterns/

http://ui-patterns.com/

http://www.uidesignpatterns.org/designPatterns

http://uipatternfactory.com/

 

For example, UI-Patterns.com lists these patterns among many.

 

Inline Input Adder
Minimize the amount of input fields by allowing the user to add more input fields if he needs them.

 

Undo
Give the user an option to easily undo an action.

 

Edit-In-Place
Use a dynamic text editor to allow the user to edit text

directly “in-place”.

 

Primary & Secondary Actions
Match the visual presentation of actions to their importance to get users through a form as quickly as possible.

 

Inline Suggestions
Help the users to give an answer by suggesting valid answers from which to pick from.

 

Carousel
Show thumbnail images of items on a scrolling menu, which allows users to browse through them.

 

Starting With Root Cause Analysis (and mitigation)

Root cause analysis doesn't have its roots in Software. It was originaly applied to manufacturing processes and has been adapted to Software defects. 

One approach is to list known causes (defect in requirements, defect in design, defect in coding, defect in testing, defect in environment, etc.) and associate each defect with that cause. That is not what I'll be doing here.

I'll be discussing going through a thought experiment for some subset of failures to determine what is likely a series of root causes and how to manage that long-term to make your product better by learning from your team's mistakes. 

First, it's likely that you only have the bandwidth to do this for a relatively few items and it's important to pick those items well. One excellent candidate is user-reported bugs that are deemed 'critical' (Hopefully, you have some reasonable way to do this). In all cases, it's helpful for the bug being analyzed to be fixed and be verified as fixed. 

Then for each defect, record the following: (This is my list, and can be suited to your needs)

• ID  / title - This comes from your bug tracking system and is used to clearly identify what issue is being discussed
  
• Function -  This is the actual system capability that failed and it may be part of a formal list of system capabilities or may be some general statement such as 'Detailed Data Display'
• Effect -  This is the impact to the user or some discoverable impact to the system.  It may or may not be the same as your title, depending on your defect report standards. A good example would be 'User unable to log in after changing to a long password'
  
• Failure -  This is a description of the behavior or design where what was implemented differs from the expectations. It may or may not be the same as your title, depending on your defect report standards. Often, it is more detailed than the Effect and may require some code or environment analysis to clarify.  A good example of this is 'Users are able to create new passwords that are more than 45 characters long, but any password longer than that will not be validated successfully'
• Notes - This is you can discuss any historical / contextual information that doesn't fit elsewhere, but that would be helpful if reviewed in the future. An example could be "This appears to be an issue that has existed since the product's first release, before we did formalized testing"
• Cause(s) -  This is where you apply one of several techniques. Rather than go into them here, you can read about Ishikawa Diagrams and make a Pareto Chart to fill out the next steps. There are other related topics you can apply to fill out this section if you find they suit your needs better.
• Recommended Actions - If you created a Pareto chart, you have the highest contributors to the cause at the top of your list of causes. You work your way down the list to address items insofar as they are helpful. It's likely that some of the causes will need to be left as continued exposure to risk if the cost to implement is not acceptable.
• Responsibility - Not only do you identify actions, but you need to assign them to someone. If you have other methods of assigning work, you can simply refer to that here. For now, we'll assume that this spreadsheet is used to track these items.
• Target Completion Date - The person that is responsible for completing this action should come up with some acceptable date to complete this. Record that date here.
• Action Taken - In the end, it's possible that the action isn't exactly what was targeted. If different, record here.
• Date Action Taken - Record the date that the action was completed so that you can focus only on the items that are not completed.

There is an implied workflow here. On some schedule, you will need to update this list (unless you are doing this as a one-time exercise). Once you have assigned actions, you will need to follow up with people to ensure they are complete or that if any changes need to be made, that those changes are made and that the mitigation is complete.

In addition to online resources regarding root cause analysis, you can also look for classroom training.

Fun Bugs

Have you ever found a bug that you had to say "Hey! Lookit!"? Most of the ones like this I find are only of interest to my team. However, here is a link to bugs found (and fixed) along with a discussion of each.

18 Embarrassing Game AI Bugs

Old-school programming techniques you probably don't miss

Computer World has an article on techniques you rarely need to consider for so-called 'modern' computer systems.

Among those listed are
- hand-coded sorting algorithms
- GUI elements
- self-modifying code (this is particularly interesting for anyone under 40)
- punch cards
- pointer math
- and much more...

The article is well-written and a fun read. From a QA perspective, the 'other shoe' aspect is that these no longer need to be tested. Whew!

Call for Bloggers

If you are interested in blogging on this site, please send us an email at redearthqa@sbcglobal.net.

If selected, we will ask that you commit to a regular schedule and be subject to editorial approval for a short trial period.

Testing Without a QA Team

Yikes! That sounds like heresy! Regardless, it's the position that many developers find themselves in. Typically, this is for internal applications or small-ish products. There is no easy justification to hire a dedicated person (let alone a whole team) to do the QA. That doesn't mean that testing can't be done.

Here are some tips from a book "Beta Testing for Better Software" by Michael R. Fine on approaching the testing effort when there are no full-time QA assigned to the product.

• Start with a usable test product
  Even if you have a list of known issues, if you can describe them clearly you can avoid unnecessary churn.
  
• Target actual users
  Find people that not only know the domain for which the product is intended for, but also someone that is able to think clearly and provide good feedback.
• Be sure you are mentally ready for the feedback
  You may have to ask for clarification for bugs that are reported. Users are not experienced in writing bug reports and likely cannot do any sort of analysis on what the root cause is.
• Be sure your schedule is ready for the feedback
  The users that are helping with the feedback will be more likely to provide feedback if they feel that their issues are addressed quickly. If there is a server portion that is offline for several hours, users may loose confidence in the product. Knowing that it's down and reacting to that quickly is good. If there is a simple fix you can provide, be ready to send that out as appropriate. Even better, plan on updates to the software as a result of the beta testing so that the users can see the improvement (and have some pride knowing that they contributed to this)
• Have a clear plan
  Knowing how many users is one thing. Clearly guiding them on what new features they should be focusing on or bug fixes to verify is important. You should also have a clear timeframe in which to ask for this feedback.
• Identify what success looks like
  How do you know if the beta test was successful? How do you know that the new features have been sufficiently exercised? How do you know that your participants provided some value? How do you plan on addressing these issues before you leave 'Beta'? How does this beta compare to previous betas?

The book goes into more details and I encourage you to read it. Even if you have a full-time QA staff, this book provides some insight into how Beta testing can be used effectively within your current processes.

Meeting Announcement - April 16th, 2009

Time and Location
The Red Earth QA's meeting will be held on the 3rd floor of 100 N. Broadway from 11:30am-1pm on Thursday, April 16th. Look for the signs to direct you to the correct room.

Topic

Sky IT Group will be presenting Quality Center 10 new features.

Requirements Management Only License – enables business analyst & owners to leverage QC to create a central location for requirements management, versioning and reuse.
Versioning - Manage project chaos by versioning requirements, tests and components.
Integrated dashboard module.
New Baseline Functionality - Capture project assets at critical stages with new baselining functionality.
Risk Based Quality Management - Enhanced Risk Based Quality Management with testing effort.
Quality Center Premier Edition.
a Reuse of Requirement / Tests across Projects.
b Defect sharing across Projects.
c Integrated dashboard module across Projects.

The presenter, Sky IT Group will be providing Subway for lunch.
We need a response by Wednesday 8:00 am only if you plan to attend for lunch at 11:30.

Can't attend in Person?

phone dial-in (877) 848-7030

code 8218181

Join the meeting

Computer Audio
To use computer audio, you need speakers and a microphone, or a headset.

First-Time Users
Make sure the Office Live Meeting client is installed before the meeting:

Troubleshooting
Unable to join the meeting? Launch the Office Live Meeting client and join the meeting with the following information:
Meeting ID: 253cc236575a4d95b7662f69832764d9
Entry Code: sYj1ys9uweq
Location: meet:sip:Bill.Rice@dvn.com;gruu;opaque=app:conf:focus:id:253cc236575a4d95b7662f69832764d9%3Fconf-key=sYj1ys9uweq

Notice
Office Live Meeting can be used to record meetings. By participating in this meeting, you agree that your communications may be monitored or recorded at any time during the meeting.

Directions

• You can park in Main Street Parking on Main or you can find street parking.
• From I-40, take the Robinson Exit. Go North on Robinson to Main. Right on Main. You can either go to Main Street Parking or continue to Santa Fe Parking. You will see 100 N Broadway on your left across Broadway. The building says 'Chase' at the top.

Meeting Announcement - March 19th

Time and Location
The Red Earth QA's meeting will be held on the 3rd floor of 100 N. Broadway from 11:30am-1pm on Thursday, March19th. Look for the signs to direct you to the correct room.

Topic
Randy Rice will be giving us a preview of his StarEast presentation on Dashboards. Randy is local to the Oklahoma City area and travels internationally to consult with companies regarding their development and testing processes and provide training in many areas including Quality Assurance.

http://riceconsulting.com/home/

Can't attend in Person?

dial-in number: 877 848- 7030
passcode: 8218181

Directions

• You can park in Main Street Parking on Main or you can find street parking.
• From I-40, take the Robinson Exit. Go North on Robinson to Main. Right on Main. You can either go to Main Street Parking or continue to Santa Fe Parking. You will see 100 N Broadway on your left across Broadway. The building says 'Chase' at the top.

Meeting Announcement - February 19th

Time and Location
The Red Earth QA's meeting will be held on the 3rd floor of 100 N. Broadway from 11:30am-1pm on Thursday, February 19th. Look for the signs to direct you to the correct room.

Topic
Sonata will be doing an online meeting showing their automation framework that ties into Quick Test Pro.

Can't attend in Person?

Here are the details for online access.

When: Thursday, Feb 19, 2009 12:00 Noon (CST)

Duration: 1:00

https://www.livemeeting.com/cc/sonata/join?id=CCM7DP&role=attend&pw=4%7C%3AtT%5Bf%262

Meeting time: Feb 19, 2009 12:00 Noon (CST)

Add to my Outlook Calendar:

https://www.livemeeting.com/cc/sonata/meetingICS?id=CCM7DP&role=attend&pw=4%7C%3AtT%5Bf%262&i=i.ics

AUDIO INFORMATION

-Telephone conferencing

Use the information below to connect:

Toll: +1 (218) 486-3850

Participant code: 331404#

FIRST-TIME USERS

To save time before the meeting, check your system to make sure it is

ready to use Microsoft Office Live Meeting.

http://go.microsoft.com/fwlink/?LinkId=90703

TROUBLESHOOTING

Unable to join the meeting? Follow these steps:

1. Copy this address and paste it into your web browser:

https://www.livemeeting.com/cc/sonata/join

2. Copy and paste the required information:

Meeting ID: CCM7DP

Entry Code: 4|:tT[f&2

Location: https://www.livemeeting.com/cc/sonata

If you still cannot enter the meeting, contact support:

http://r.office.microsoft.com/r/rlidLiveMeeting?p1=12&p2=en_US&p3=LMInfo&p4=support

NOTICE

Microsoft Office Live Meeting can be used to record meetings.

By participating in this meeting, you agree that your communications

may be monitored or recorded at any time during the meeting.

Directions

• You can park in Main Street Parking on Main or you can find street parking.
• From I-40, take the Robinson Exit. Go North on Robinson to Main. Right on Main. You can either go to Main Street Parking or continue to Santa Fe Parking. You will see 100 N Broadway on your left across Broadway. The building says 'Chase' at the top.

Meeting Announcement - January 15th

Time and Location
The Red Earth QA's meeting will be held on the 3rd floor of 100 N. Broadway from 11:30am-1pm on Thursday, January 15th. Look for the signs to direct you to the correct room.

Topic
Software Development Technologies will be discussing their offerings.

Can't attend in Person?
dial-in number: 877 848- 7030
passcode: 8218181

Directions

• You can park in Main Street Parking on Main or you can find street parking.
• From I-40, take the Robinson Exit. Go North on Robinson to Main. Right on Main. You can either go to Main Street Parking or continue to Santa Fe Parking. You will see 100 N Broadway on your left across Broadway. The building says 'Chase' at the top.