What Was the Question?

In my last blog entry, Obsessing over Assessment, I recommended choosing question formats that make sense for the level of learning you need to assess. I then went on to discuss the most popular form of test item: the multiple choice question. However, there are other options, and some of them may be more appropriate for use, depending on what you are trying to assess.

Here are five common question types along with a few guidelines as to how and why you would use them.

1. Short Answer Questions - also known as fill-in-the-blank - are best used to assess basic information that learners need to commit to memory. They are helpful for testing terminology, facts and simple computations. Short answer completion items should have only one brief correct answer. Typically, the blank for completing the statement is placed at the end of the test item. Blank spaces for all items should be equal in size, and should not be any larger than necessary.
2. True/False Questions – are second in popularity to multiple choice questions. Like multiple choice questions, they require the participant to select a response. In this case, there are only two options: True or False; Yes or No; Agree or Disagree. They are useful when there is a black and white distinction between two alternatives. There can be no gray areas. Well-written True/False questions are usually stated as declarative sentences that focus on a single idea. A common mistake that test writers make is to put two ideas in one statement, requiring both of them to be true in all cases. Another common error is to tip off the answer by including words such as “always” or “never” in the statement, which usually means it is false. And of course, on the downside test participants always have a 50-50 shot at the answer, so they may be likely to venture a guess.
3. Matching Column Questions - are used to assess content knowledge and associations between ideas. They fit into the category of selection items along with True/False and Multiple Choice. They are constructed by stating the premise for each test item in the first column, and listing options or responses in random order in the second column. Typically, the items in the first column are identified by number, and the response choices in the second column are identified using letters. Some test makers are reluctant to use matching columns because they seem harder to construct than other question types, but they are really very similar in to multiple choice questions. The format of premises and responses is very clear for the test taker, and they are easy to score.
4. Multiple Choice Questions - are everyone's favorite, and rightly so because they are so versatile. They give you the ability to go beyond testing for facts. You can write multiple choice questions to measure learning outcomes that test for knowledge, comprehension, and application and, to a lesser degree, analysis, synthesis and evaluation. A good multiple choice question includes a clear, well-written premise, and a list of reasonable response choices, one of them being the correct answer (which should not always be choice C!). You can also couple them with reading passages, tables or charts that include the correct answers to assess your test takers ability to interpret important information.
5. Essay Questions – are useful for assessing how well test takers can analyze, compare, contrast, evaluate, interpret or integrate ideas. The obvious downsides are that they rely heavily on writing skills and they are a challenge to score. Those issues can be minimized by attending to how the questions are written. If the essay question is too open-ended, it leaves it up to the test taker to decide on which direction to go. Instead write questions that ask specific questions. The person participating in the assessment must be able to clearly identify what it is you want to know from them. Give them parameters, by using phrases such as, “provide five ideas on how you would…” or “give three reasons why you would…” You can also provide direction by asking the test taker to “consider the following factors…” when writing a response. Consider identifying word count minimums and maximums so the person will be able to gauge the level of detail expected. Also, for every question you write, make sure you create a model answer to aid scoring.

For more detailed information, check out Tests & Measurement for People Who (Think They) Hate Tests & Measurement by Neil J. Salkind. Part III of his book is called The Tao and How of Testing. It is a great resource for anyone who needs to construct assessments.