"Judge a man by his questions rather than by his answers."
- Voltaire

Software Testing - A Primer

Planes, trains, and automobiles.  Telephones, microwave ovens, and television sets.  Medical monitoring devices, power stations, and banks.  Software is just about everywhere nowadays in one form or another.

There are many highly-intelligent men and women who create the software that drive all these wonderful, magical machines.  The act of translating ideas into reality is kind of like solving a complicated puzzle.  And to make things more difficult, each (software development project) puzzle changes depending on a lot of things.  As such, there are no Rules in this "puzzle" of software development (except maybe to remember to play nicely with your coworkers) so much as Guidelines.

There are many illustrative analogies for describing software development projects.  Here is another one:  developing software is kind of like building a house.. at night.  Until you shine a light on the work completed, you don't really know how well it was built or meets the customer's expectations.  Testers bring an assortment of lights and ways of looking at things to help everyone see as much as possible in the time available.

Almost everyone plays a "testing" role at one point or another when they are involved in creating software.  Some do it full-time.  There is both an Art and a Science to Testing Software.  Unfortunately, until recently, there has been a great lack in formal education and training for developing good testing skills.  The software testing skills-spectrum is very broad though, so it will likely take some time before this all becomes "academic" and a natural part of the way we look at things.

Here is some information to get you started:

The People You Should Know

In addition to understanding the playing field, it helps to know who the players are.  Here are some of the people I know and respect in the Software Testing profession.  If you happen to see one of these names in an email, next to an article, or on a book, read it!  Better yet - attend one of their workshops, courses, or go to the same conferences that they go to.  It's a good bet that you will probably learn something useful.  I have.

There are many more terrific people in this industry.  Start by getting to know the ones doing work and research in an area of your interest.  You may find some of their articles on StickyMinds.com, and some interviews on WhatIsTesting.com and DDJ Testing Blogs.

The Things You Should Know

What makes a great tester?  A great tester is always learning, always improving themselves.  The following is a list of some of the key areas for development for a Software Tester.  (Many of these areas may also generally improve the quality of your life outside of work too!)

Think like a Scientist

Integrity and Discipline

Think like a Hacker

Problem-Solving and Analysis

Testing Theory, Skills & Knowledge

People Skills

Technical (Report) Writing

Quality Oriented

Additional Technical Skills

Communication

Professional Development

BTW, I'm always adding to and clarifying this list, so please send me any suggestions you may have.

1. Think like a Hacker

• How To Become A Hacker - A must-read for any serious Tester!
• How To Ask Questions The Smart Way - Do you know how to ask questions when you post to public forums and newsgroups?  Do you?
• A good hacker is a Systems Thinker - read Gerald Weinberg's An Introduction to General Systems Thinking

2. Think like a Scientist

• Learn or relearn the Scientific Method.  It is an indispensible process to help you understand and do good testing.  (Don't just rely on the one link I include here.  Google "Introduction to Scientific Method" and see what turns up.  Always read from multiple sources!)
• Learn Mathematics - in particular: Probability and Statistics.  Mathematics is the language of Science.  If you want to understand the controls and variables in your testing 'experiments' then you should have a good understanding of Stats.
  • In Performance Testing, the Scientific Method and Statistics are critical success factors.
  • If you wish to develop Metrics for managing the testing effort, you need to understand how to validate your assumptions in order to understand what it is that you are really measuring.  (Most metrics I've seen and heard used on development projects are just really, really bad because they have no rational mathematical basis whatsoever.)
• Psychology can give you some good insights.  Cognitive Psychology can help you understand the nature of observation (also critical in good testing), while other branches of Psych can help you learn to work effectively with project team members.  Read "The Irrational Tester" by James Lindsay.  See the People Skills section below for more.

3. Problem-Solving, Analysis, Critical & Lateral Thinking

• Become a Critical Thinker!  You don't need to be a genius to test software.  It helps if you have a consistent problem-solving approach that helps you to track down and identify problems.
• Practice Lateral Thinking and Parallel Thinking!  Browse Edward De Bono's web site and material.  The Six Hats method is particularly effective in team environments.
• Learn how to organise information in various ways.  Whether you are working with Mind Maps or Pareto Charts, learn how to collect and analyse data and then present it in a meaningful way to achieve results.  Every new technique you learn becomes a new tool to add to your utility belt.
• You can find many useful books out there on Problem Solving if you just look.  Here are a few to get you started:
  • An Introduction to General Systems Thinking, by Gerald M. Weinberg (more advanced)
  • Rapid Problem Solving With Post-It Notes, by David Straker (novel team-based problem-solving approaches)
• Exercise your brain to keep it sharp!  Solve puzzles - especially lateral thinking ones.  There are many puzzle web sites out there - browse them on your lunch breaks! (My Fun page has some examples.)  Not into puzzles?  Pick up a Mystery novel instead.  Make critical & lateral thinking a part of your fun time and you'll soon discover that work might be easier than you thought!
• Become an Investigator.  There is a lot to be gained by knowing how to interview people while working on a software development project.  Learn to ask the right questions.  Learn to read the cues from those providing the answers.  Learn how to ask the questions so that you get the information you need.
• Practice developing Mental Models.  This is especially useful when "Black-Box" testing.  Use your observations to come up with a theory of how you think the software or just some feature works.  Discuss the ideas with your peers and explore the various boundaries of your model to see if you discover anything interesting about what you are (really) working with.  Some call this process "Reverse Engineering" the design.  It's a useful tool for testers.. whatever you call it.

4. Technical (Report) Writing

• It doesn't really matter how good a tester you think you are if you can't communicate clearly and concisely.  A tester who can't write a clear bug report is like a carpenter who can't use a hammer - useless.
• Society for Techical Communication.  Ever wonder how Technical Writers become good at writing the user documentation?  Practice.  Join the STC for a year or two, attend their local chapter meetings, read their magazines, befriend the Technical Writers in your organization and learn how to become a better writer.
• Language barrier?  If you are having troubles putting your thoughts into words, just remember that a picture is worth a thousand words.
  • Take a screen capture of the problem you are looking at and edit the picture in a simple "Paint" program that comes with most Operating Systems.
  • Use the drawing tools to add arrows and simple text to point out the problem(s) you see.
  • Save the file as a PNG, GIF, or JPG (just *not* a BMP!) and upload it as an attachment into your Bug Tracking system or email it to the developer directly.
• Start a Blog.  Practice may not always make perfect, but it might get you pretty close.  If you want to become a better writer, then write!  Write often!
• Take a Journalism course.  Learn to focus on the facts and present them in meaningful ways.
  • One of my favourite quotes: "I keep six honest serving-men (They taught me all I knew); Their names are What and Why and When and How and Where and Who." - Rudyard Kipling, The Elephant's Child (1902)

5. People (a.k.a. the "Soft") skills

Making Software is first and foremost a people-activity.  Approach it from a strictly technical perspective and you will go down in flames every time!  It's not about brown-nosing and politics.. it's about being personable, respectful, and professional.  You will always be more successful as part of a team than as an individual.

• Learn to Listen.
• BOOK: The Team Handbook, by Scholtes et al. (read the Amazon reviews.)
• "Working Effectively With Developers" by Brian Marick
• The AYE Conference
• Psychology: Personality Profiles (e.g. MBTI), Johari Window, etc.
• Learn to give and receive feedback.  Join ToastMasters!
• Learn to Negotiate.

6. Testing Theory, Skills and Knowledge

This is, of course, one of the most important areas to focus your professional development on as a Software Tester.  Without a doubt, you will need to understand things like: Risk Assessment; how to develop a Test Strategy; what are the various Test Techniques and when to apply them; and the role and strengths of the various types of testing performed throughout the development of software.  (The latter two in particular will require experience, not just knowledge.)

• TestingEducation.org - your one-stop Software Testing Knowledge resource!  Follow the links to Course Notes, Lab Articles, and Workshop materials.  Did I mention that it's all FREE?!  Start Reading and going through the stuff today!
• BOOKS: Lessons Learned in Software Testing, by Kaner, Bach and Pettichord; Testing Computer Software, by Cem Kaner.
• Software Engineering.  Discover the "big picture".  Find out where testing fits into the "Quality Assurance" picture and what are some the many different verification and validation activities performed throughout a development project.  Read The Mythical Man-Month by Fred Brooks.

7. Additional Technical skills

• Project Management.  You want to understand Test Management and what a "Test Plan" is really about?  Take a Project Management course.  Don't take one of those wimpy one-day courses either.. go for at least a three-day hands-on workshop.  You won't regret it.  Check out the Project Management Institute web site.
• Programming.  Learn a scripting language like JavaScript, Perl, PHP, Python, Ruby, TCL, or VBScript.  Scripting comes in handy when you are testing, but more importantly it helps you get a better appreciation of some of the types of bugs that may crop up during software development.
• Databases.  Learn SQL! Become familiar with Oracle, MS SQL, or just play around with MySQL.  Any tester worth their weight in salt isn't afraid to find out where the data goes after you type it into a User Interface.  You become a more effective and valuable tester when you are comfortable looking at and modifying the data in a database directly.
• Get friendly with the Web.  Learn HTML.  Learn what a WIKI is and how you might use one within your Test group or department.  Twiki is rather popular within companies because it provides security and revision history.
• Information Management.  As Testers, our job is to provide information to the decision-makers.  A good part-time hobby would probably be to investigate this interesting area of Information Science and Knowledge Management a bit further.  (It's not a coincidence that the "I" in "IT" stands for "Information"...)
• Hardware.  It's not mandatory for all kinds of testing but I've found knowledge of hardware components to be very useful to me over the years.  Knowledge of how RAM works, RAID and hard drives, Modems, Video cards, CD and DVD ROM technology, Networks, Printers, etc. can all be beneficial when testing software and systems.  What's the best way to get to know hardware?  Turn it into a hobby!  There are some good courses and certifications available if you want to get serious about it.  Check out certifications like "A+" (A Plus), "Network+", and so on.

8. Quality-oriented

• Quality Software Management: Systems Thinking, by Gerald M. Weinberg.  A must read!!  This is the first in a 4-volume set.  Get them all if you can.
• Spend some time in Technical Support.  (Some time in Sales or Marketing might be an okay second choice.) Understand your customers first-hand and find out what's really important to them.  You may be surprised.
• Learn about Quality Theory from the masters: Deming, Juran, Watts Humphrey, etc.  Read the publications coming out of the Carnegie Mellon Software Engineering Institute.  The PDCA cycle and some other ideas may prove useful.
• Learn what Software Development Life Cycles are and what the different development models have to offer.  Knowing the strengths and weaknesses of each model should help you know when to apply the best one for the situation at hand.  Just remember this rule: One size NEVER fits all.
• Learn about CMMI.  And then forget all about it.  Remember: you are a Tester.  This info should help you better understand your role in the big picture as well as understand the differences between "Software Quality Assurance" and "Testing".

9. Integrity and Discipline

• Take the job seriously.  Be assertive - stand up for yourself without putting others down.  Lead by example.  Accept and admit your mistakes.  Better yourself -- others will follow.
• What does Integrity mean to you?
• Find a good "Code of Ethics" and follow it.  For example, check out the ones for ACM, ASQ, and IEEE.
• Tact and Diplomacy.

10. Communication

• Develop Presentation Skills.  Learning how to summarise important information into a presentation with impact is not easy.  There are books, courses, and workshops available to help you make the most of your presentations and charts -- look into them.  Edward Tufte is one person you should get to know.  Get one of his books, or better yet attend one of his workshops if you can!
• Here's a hard one for many people I've met: Learn how to get to the point.
• Become a Mentor or Coach.  Besides the immediate benefit of sharing knowledge, coaching helps you to develop patience, respect and understanding, and how to build relationships.  All of these will strengthen your team.

11. Professional Development

• Participate in the Software Testing community by joining some mailing lists.  Participate!  Everyone's skills, backgrounds, and experiences are different, so you never know when your feedback may be the key to helping someone else with a problem similar to one that you have already encountered.
• Attend a conference or workshop.  Check out the Continuing Education courses at your local college or university.  Take a writing course, Project Management course, or look for some good Leadership or Management courses that might give you insights into working with others.
• Join a local or international Professional networking organisation.  Take note of the Association for Software Testing.  Some "QA" groups don't cover much Software Testing, but this group is all about the Testing!
  • For example, some peer networking groups in my area include the KWSQA, SWOSQG and TASSQ.
• Start a Library!
  • The age of paper books has not yet come to an end.  Start collecting!  There are many suggestions already made on this page.
  • Everyone's library is customised to their own interests.  At a minimum, you should have some books from each of the above categories.
• Get a magazine subscription. There are many other good magazines out there but not many of them focus on Software Testing. Some good ones that I have come across include:
  • Software Testing Club Magazine - free online magazine.  Start here.
  • Better Software - comes in two flavours: hardcopy and Zinio electronic format. (I prefer the hardcopy to take with me places, but the electronic format could be really handy if you live far from the U.S. or if you prefer reading magazines online.)
  • StickyMinds.com - the electronic companion to Better Software magazine. Many discussion forums you can participate in and several eLetters that you can sign up for to keep you abreast of current submissions to the site.
  • Harvard Business Review - help you build leadership skills by giving you the lingo and information you need
  • Software Test & Performance - issues available in PDF format for free.  Great gems in some of these issues!
  • LogiGear Newsletter Articles on important software QA testing topics.

Contact me at: paul [at] staqs [dot] c o m