Checklists for eModerators

It seems to be topical and much in demand, so for a limited period I’ve uploaded a copy of our generic checklists for eModerators here http://dl.dropbox.com/u/3063607/Checklists_generic_030620110.pdf

A practical guide to creating reference information: part 3

Practical guides

In part 1, we examined how reference information differs from other learning content, the purposes it fulfils and the formats that it can take. In part 2, we explored best practice in terms of the presentation of information. In this third and final part, we look at the various ways you can provide access to your information

Tables of contents

Tables of contents (TOCs) are usually associated with print publications but still have a valid role to play in digital media. Whereas searching can be a rather hit and miss affair, a TOC provides an organised and orderly gateway to a body of information. Every situation is different, but there are some general rules that will help you in preparing a TOC:

  • Organise the table in a way that makes sense to your user, not what is easiest for you. Think of all the situations in which a user might come to your information and how they would expect to drill down to find what they want. Should the information be organised in task order, by category, chronologically or alphabetically?
  • Keep menus to a reasonable length while keeping the loading of new menu pages to a minimum. This might sound an impossible compromise to make, but there are all sorts of ways of hiding and revealing lower-level menu items using the sorts of tree menus you find in your computers file managers.
  • Draw your user’s attention to the most commonly sought after items of information. You could even have a top ten list.
Chrome TOC

Google provides access to its Chrome help library though a table of contents, but also - as you would expect -through a search box

Chrome help

The length of this menu is kept to a minimum by the use of collapsible menus. The user chooses which menu is appropriate to their operating system. Notice how Google draw your attention to a popular favourite - '20 things I learned about browsers and the web'.

Search

Search used to be considered a rather unfriendly way for users to access information, but search technologies and users skills in searching have come on in leaps and bounds. The advantage from a user’s perspective of using search over a TOC  is that it shortcuts all that drilling down through menus. A search facility is now expected and should definitely be provided if at all possible.

Search can be improved by tagging, the process whereby descriptive labels are applied to content. The process of tagging can be managed on a top-down basis, by content authors, or bottom-up, when users supply their own labels.

Wordpress tagging

This box is used to add tags to postings on the Onlignment blog

Making suggestions

Another way to get users to information that they could find useful is to provide them with intelligent suggestions. An easy and obvious way to do this is to provide ‘Related items’ links at the bottom of each piece of information. You could go a little further by building up a profile of each user and then suggesting links that you know from their past usage history would be relevant to them – something like what Amazon do with their ‘People who bought this title also bought …’ suggestions.

PowerPoint help

This entry in the PowerPoint help system is supplemented by links to related items. Notice also how the word 'object' is linked to a pop-up glossary entry.

Peer recommendations are always the best, so you may also want to provide a facility to let users recommend items of content to their colleagues, perhaps by emailing them a web address or through some social networking tool.

Unless you have a great deal of programming expertise at your disposal, chances are you’ll be limited in the way you can provide access to information by the tools already available to you. Where you do have choices, use them wisely. Listen to your users and let them tell you what they find useful and what’s just dressing.

This guide is now available as a PDF download

Coming next: Sorry, that’s all our practical guides finished. Unless there’s something we’ve missed … ?

A practical guide to creating reference information: part 2

Practical guides

In part 1, we examined how reference information differs from other learning content, the purposes it fulfils and the formats that it can take. We move on to explore best practice in terms of the presentation of the information.

General design guidelines

Reference information is different from other learning content because its purpose is to meet a short term need, not to have some long-term effect on the user. It does not need to engage the user, because you can assume that anyone who has made the decision to consult reference material is already engaged. It does not seek to stimulate thought or discussion. It does not need to assess the degree to which learning has taken place, because learning is not its purpose. You should have one simple aim in designing your materials: getting the user to the information they need as quickly and simply as possible and then presenting that information in such a way that it can be used immediately.

Here are some general tips that you can apply to any type of online reference material:

  • Provide structured information, not prose: What works in a book, a report or a magazine does not work for reference. Very few people will read your material from beginning to end, if they read it at all. They will jump around and skim at lightening speed until they find what they are looking for. Long paragraphs of text are very hard to skim, so focus instead on using tables, lists and diagrams. If you must use prose, keep paragraphs and sentences short and focus on one point at a time. Use plenty of clear, descriptive headings. Leave out the anecdotes and witticisms.
  • Be consistent: It is much easier to navigate information that is presented consistently. Choose a format for each type of information and use this every time. Adopt a typographical standard and apply this universally. Never vary your means for navigation. What should stand out should be important information such as rules and warnings. The only way you will achieve this is against a uniform backdrop.
  • Ditch the numbering system: It is debatable whether elaborate numbering systems ever helped anyone to navigate a print document, but they are completely unnecessary and unhelpful online. Online information is accessed using search boxes and links. Numbers only get in the way.
Prose v structured text for reference information

Unstructured prose is fine if you're relaxing reading a novel but is no good if you're hunting down information. Contrast the extract on the right from Adobe Community Help.

Presenting factual information

Factual information, such as names, dates, email addresses, spellings and codes, is best structured into databases or tables, and sequenced alphabetically, numerically or chronologically. As software becomes more intelligent, we will have less need for these tables; we should expect frequently-used data to be suggested to us automatically, based on our historical usage pattern and the particular context. But there will always be exceptions and we do not only need factual information when we using a software application.

Keyboard shortcuts

This alphabetical table of keyboard shortcuts from Techicious makes it easy to reference factual information

Presenting concepts

According to Ruth Clark, a concept is a category of objects or ideas that is usually designated by a single word. Essentially when we refer to concepts, we are talking about the terminology which helps us to find our way around a particular knowledge domain. Without a common understanding of this terminology, we cannot communicate with our peers.

An example of a concept is the hashtag used by Twitter. To understand this term we need a definition, one that makes clear what a hash tag is and what relevance it has. We could do with some examples and some non-examples; for example, #devlearn is a hashtag to be used when referring to the DevLearn conference, but @devlearn is a Twitter username.

Some concepts are less abstract and benefit from a non-verbal approach. So don’t just list the characteristics of different species of bird; show what they look like and what they sound like.

Sometimes we need to explain a concept in conjunction with a procedure, because the procedure only makes sense when the concept is clear. More typically, when we have many concepts to explain we are better off building a dictionary or glossary.

Moodle glossary

This Moodle glossary provides easy access to definitions of e-learning terms

Presenting procedures

A procedure is a series of rules for carrying out a routine task in a logical sequence. Some procedures are essentially linear: we carry out the same steps each time. Others are conditional: at each step we need to progress differently depending on the particular situation. Linear procedures can be laid out in a simple tabular form, with tips and warnings at each step where appropriate. If you are presenting a software procedure, use screen shots at each step, but only if these are really needed to make the procedure clear. Remember that these will need revising regularly as the software is updated.

Conditional tasks can be hard to explain in tabular form, in which case you could use a flow chart. These too can become complex and unwieldy, in which case an interactive tool might be more appropriate, in which each step is presented in turn, with conditional links depending on the decision made at each point.

Presenting processes

A process is a description of how something works. Processes typically involve a number of stages, with the output of one stage being the input to the next. Some processes, such as the water cycle, are cyclical. Others, such as a disciplinary process, continue until some goal or end point is reached. When users understand a process, they get the bigger picture; they understand how what they do impacts on others.

Processes can be presented in a tabular fashion, but they will often benefit from a process flow diagram of some sort.

Process flowchart

This flowchart from Edraw Soft provides an overview of how a business process works

Presenting structures

Some information is essentially structural in nature; it describes the ‘parts of’ things, like the bones in the body, the towns in a region, the elements in a software interface, the roles in an organisation chart, the events in a timeline. Unsurprisingly, this information is best presented visually, with each part clearly labelled. You can also configure each element as a link to further information.

Technology timeline

This interactive timeline, produced using Articulate's Engage tool, provides a way for users to explore the history of learning technology

Problem-solving and decision making

Sometimes it is not just information that a user needs, but help in troubleshooting a problem or making a decision. One of the most common ways of addressing the former is with an FAQ, a list of frequently-asked questions, but for more complex problems, such as a network fault, an interactive troubleshooting tool is likely to be much more useful.

Decision aids could be structured as a simple comparison table, listing the pros and cons of each option, or more helpfully as an ‘if … then’ list, which recommends a different option for each potential circumstance. A more interactive tool might ask a series of questions in order to narrow down a suggestion to a single alternative.

Coming in part 3: Providing access to your information

A practical guide to creating reference information: part 1

Practical guides

Strictly speaking, reference information isn’t learning content at all, because its purpose is on-demand performance support, not learning. In a performance support context, there is no requirement for the information that is being referenced to be learned, i.e. to be stored in long-term memory. The information is required only to answer a current question or solve a current problem and, as such, it will be processed only in working memory. Because working memory is so limited (current thinking is that humans can hold only 3-5 items of information in their conscious working memory at any one time), it is vital that reference information is extremely clear, simple and concise, minimising the risk of cognitive overload.

Working memory

Reference information need go no further than working memory

Reference information is playing an ever bigger part in our lives. There’s so much we could know and it’s changing so regularly that it really is pointless trying to remember it all – we couldn’t do it if we tried. True, it is still as important as ever that we understand the key concepts, principles and rules that underpin our work, as well as the skills to apply these on a day-to-day basis, but the rest we can draw down as and when it’s needed.

As a learning designer you may be wondering what reference information has to do with you, but you can play a key role. When you design a learning solution, you have to decide what’s course and what’s resource, and in many cases you will share materials between the two. And, as an expert in communication, you are better placed than many to do a good job of putting together reference materials that are clear, concise and usable.

Media elements

Reference information can utilise any media element, but tends to centre on text. There are good reasons for this. Text is fast to load, it can be quickly skimmed, it can be easily cross-referenced with links and can be copied and pasted with ease. The key with reference information is to get users in, get them to what they want and get them out again as quickly as possible. Text – supported where appropriate by photos, illustrations, charts and diagrams – performs these functions really well.

Textual information

Assuming it is formatted properly for reference use, text does an excellent job of getting the user to the information they require

There are exceptions of course. Some tasks are best explained visually, using video or screencasts with accompanying narration. As long as these are kept short and sweet, they can do the job. As we have dealt with both of these formats in previous guides, we won’t be covering them here.

Interactive capability

Because the purpose of reference materials is just-in-time support and not learning, interactions which encourage users to explore ideas or which assess learning are not relevant. Apart from anything else, they would drastically slow up the user in getting in and getting to the required information. On the other hand, as we shall see in part 3 of this guide, navigational interactivity is critical to good reference materials, whether that’s through search, indexes, tables of contents or cross-references. Some more advanced performance support materials may also allow users to configure the information they want to receive (think of something like a weather or stock price app on a mobile device) or will ask users a series of questions to help them trouble-shoot a problem or narrow down a decision.

Applications

Reference materials fit classically within the exploration learning strategy. As such they are designed not to be ‘pushed’ at the user but ‘pulled’ as required. In the context of a blended learning solution, they supplement courses with easily accessible resources. Whereas historically some 80% of a learning designer’s effort might have been put into the courses and only 20% into providing on-demand resources, we can expect that ratio to reverse in coming years. People no longer expect to have to store loads of information in their heads; they do expect to be able to access it online.

Formats

Reference information can be presented in a number of formats:

  • Embedded in a software application: One of the most common requirements for on-demand help is to explain how to use a particular software application, or how to enter appropriate data into that application. An advantage of embedding the help within the application itself is that the information presented can be context-sensitive, i.e. directly relevant to the activity that the user is currently undertaking.
  • In a native document format: Much reference information is stored in a native document format such as Microsoft Word. While tools like Word have very sophisticated editing capabilities, they are not best suited to online use. Native files are slow to open and depend on the user having a copy of the application that was used to create them. More importantly, it becomes clumsy to link from document to document and it is all too easy for different versions of the document to be available at the same time.
  • As a PDF file: PDF is versatile in that a wide range of applications can save in this format. It also overcomes the need for users to have copies of Word, PowerPoint or whatever  other applications were used to create the materials. PDFs are particularly suitable when users need to be able to access information without an online connection or when they want to print materials out. In all other circumstances, HTML wins out.
  • In HTML: HTML is the format of the web and, as such, is standard across all computing devices. Web pages are ideal for displaying reference information because they download quickly, can be accessed from any device, and can be kept up-to-date centrally. They can also be easily cross-referenced using hyperlinks.
  • As a mobile app: As any smart phone or tablet user can attest, apps are the quickest and easiest way possible to access information. They are particularly suited to situations in which a body of content or an up-to-date information source needs to be accessed very regularly. They score over mobile web browsers which access simple HTML pages online because the way the information is displayed can be designed specifically for the mobile device in question. The application itself is stored locally which speeds up access, and less volatile information can be stored on the device itself, making the information accessible even when there is no internet connection.
Embedded help

Embedded help can be context-sensitive, taking you directly to the information relating to your current task

Reference app

Reference apps for mobile devices are ideal for information you need to access regularly

So how do I get started?

Assuming you have researched fully what information is needed by whom and in what circumstances, your next job is to choose the format in which the information will be presented. The format will dictate the tools you’ll need:

  • Embedded information: Here your best route is to liaise with the developers to see what’s possible and what help creation tools are available.
  • Native document format: This is simple enough – just use Word, PowerPoint or whatever to create your materials and then publish in the version most widely available to your users.
  • PDF: It used to be you had to purchase special Adobe Acrobat software to create PDFs, but now you’ll find that many of the applications you’re already using can save directly to PDF format.
  • HTML: The tools you use here will depend on the infrastructure your organisation already has in place for distributing information. There is probably a content management system in place for your intranet and that’s a good place to start looking. If not, and you’re looking particularly to provide information to support software users, you may want to purchase a system that’s specially designed for creating online help materials. Whatever the case, you do not want to be hand crafting HTML pages. Those days are long over. Creating web pages should be no more complicated than filling in a form.
  • Mobile app: Until more easy-to-use tools become available, developing an app is a specialised and highly technical process. For now, best to liaise with your own IT team or engage an external contractor.

Your next concern is how you are going to present the information and that’s where we’re heading next.

Coming in part 2: Presenting reference information

A practical guide to creating quizzes: part 5

Practical guides

In part 1, we looked at the characteristics of online quizzes and explored how they could be used to assist or assess learning. In part 2, we explored the various question formats and the types of learning for which they are best suited. In part 3, we moved on to the writing of the questions, in particular the traps to avoid. In part 4, we saw how quizzes can be presented as games. In this final part, we look at the steps you can take to make your quizzes robust and reliable.

Being thorough

Assuming your quiz is being used to test knowledge, then you need to take some care to ensure that it performs this function effectively. Prepare at least one quiz question for each of your knowledge objectives. You cannot be sure that a learner has achieved mastery if you test only a sub-set of your objectives. To be absolutely sure the learner has not simply got lucky by guessing answers, you may well prepare more than question for each objective. Don’t write questions to test skills, unless you are absolutely sure quiz questions are capable of assessing these effectively, which is likely to be rarely.

As we discussed in part 2, you need to select a question format that’s appropriate to the type of knowledge you are testing, For example, if you need to test recall of a technical term, use a text input question and not a multiple choice, which only tests recognition of the term. Don’t be tempted to select different formats simply to increase variety – that’s not your purpose here.

If your objective is that a learner is able to come up with a response quickly, then add time limits to your questions.

Discouraging guessing

Some people reckon they can pass any multiple choice quiz by guessing the right answers. Your job is to prove them wrong. In part 3 we looked at techniques you can use to make life difficult for the chancer – no give-away distractors, no obvious right answers. A simple improvement would be to prepare at least four options for each multi-choice question, and even better five. That does make it even harder to write the questions, but then there really is no pain, no gain when it comes to question writing.

Guessing

Hopefully all that deep thinking is to come up with the right answer, not just to make a good guess

Another technique you could try is to include a ‘don’t know’ or ‘not sure’ option for each question. This would score no points. Then penalise wrong answers with negative scores. This ups the stakes for the learner who wants to guess the right answers.

Discouraging cheating

The greater the reward for passing an assessment, the more tempting it becomes to cheat. Really high-stakes assessments are beyond the scope of this guide, but you should be aware of the difficulties in authenticating whether the person answering the questions really is who they say they are. All sorts of complex and expensive technologies are available to authenticate remote users, including finger-printing and retina scanning, but the only way you can be really sure that the learner is who they say they are and is getting no help from a third-party or some reference source is to have them attend a testing centre which has an invigilator present. Most quizzes are not that serious, so there’s no point getting carried away with the security!

Retinal scan

Sophisticated technologies such as retinal scanning can be used to aid the authentication of learners, but are only really necessary for very high stakes assessments

A more routine way to avoid cheating is to randomise the order in which the questions are presented and the order in which options are displayed within the questions themselves. That way, no-one can simply write down the question and option numbers and pass them on to others. A step further is to create a bank of questions from which the system selects the questions to display, which means that every learner will receive a different set of questions. Yes, this is a lot more work, but the chances of successful cheating will be much reduced.

Quiz options in Moodle

Every quiz tool will provide you with different configuration options. This shows some of the options available in the Moodle quiz module, including the ability to set time limits and shuffle questions and options.

Providing feedback

Assuming you are using a quiz as a form of assessment, then if you tell the learner whether they have got each question right or wrong, you are making it easy for them to pass the quiz on a second attempt, without necessarily curing any misunderstandings they may have had. To avoid this problem, you could create a completely different quiz for second attempts, or have the system draw questions from a bank, as described above.

At the end of the quiz, inform learners whether or not they have passed. If your software allows it, let them know how they performed against each of the topics addressed by the quiz. Pass or fail, provide advice to learners on what they should do next.

If the quiz is being used in a formative manner (to help the learner progress towards the learning objectives), rather than summative (to assess mastery), then it is vitally important that you provide helpful feedback for every question. Ideally this should be provided for each option of each question, rather than just for all correct answers and all incorrect answers. The purpose of this feedback is to correct errors and misunderstandings and to reinforce key learning points.

Scoring fairly

Another consideration is how you score correct answers. Most authoring tools will allow you to specify the number of points you will award to each correct answer. In a simple multiple-choice question, this is straightforward enough – you either allocate the same number of points to each question or award more points for particularly difficult questions.

The difficulty comes with questions that ask for multiple responses. The first consideration is whether these questions should score higher than MCQs because they are actually asking the learner for a series of decisions, not just one. Another issue is how you apportion the points across the various options. Let’s say there are five alternative options, three of which are correct. Ideally, each correct option will score 20% of the available points. But the learner should also be rewarded for not choosing incorrect options, so each option not chosen should also score 20% of the total. Whether you can achieve this with your authoring software remains to be seen!

That concludes this practical guide. A PDF version will be available shortly.

Next up: how to create reference material.

A practical guide to creating quizzes: part 4

Practical guidesIn part 1, we looked at the characteristics of online quizzes and explored how they could be used to assist or assess learning. In part 2, we explored the various question formats and the types of learning for which they are best suited. In part 3, we moved on to the writing of the questions, in particular the traps to avoid. In this fourth part, we have a little light relief, as we see how quizzes can be presented as games.

Quiz games are still quizzes, in that they can be used assist and assess learning, but they employ gaming techniques to increase learner engagement. We all know how compulsive games can be, so it takes little in the way of imagination to appreciate how much they can add to what would otherwise be a very dry process of drill and practice.

To demonstrate a wide variety of quiz game techniques, I’ve taken examples from a quiz making  package called Quizit, unfortunately no longer available. Similar results could be achieved by those with coding skills using Adobe Flash Professional or HTML 5, or by using a number of off-the-shelf quiz game tools.

Ident

Players try to identify as many as possible of the six pictured people, objects or places.

This first example, a classic ‘picture board,’ requires players to type in the name of the pictured object.

Quotient

Players try to get as close as possible to the right answers for each question. Each option is graded as to how right or wrong it is and scored accordingly.

In this variant of a multiple choice quiz, player get rewarded for how close they can get to the right answer. The rather irreverent feedback is delivered randomly from a pool, depending on the accuracy of the answer.

Summit

Players attempt questions of increasing difficulty, with the aim of getting to the highest level that they can (10 being the highest). Players have three ‘lives’, which allow them to have another go when they make a mistake.

Levels are a classic gaming feature. As the player moves up the levels, the questions get correspondingly more difficult.

Guess

Players attempt to guess the identity of a person, object, place or event from the clues provided. The more time they take, guesses they make or clues they ask for, the lower their score.

This game is unusual in that it works entirely as a ‘conversation’ between questioner and player. All input is by natural text. Time pressures add to the level of engagement.

Target

Players demonstrate how well they know the subject of the quiz by estimating a series of percentages relating to the subject’s behaviour. They nearer they get to the actual percentages, the higher their score.

This game works with a slider, which the player uses to make estimates.

Sprint

Players answer a series of questions as fast as they can. They can have as many attempts as they like at each question, but this reduces their score accordingly.

This time players can have multiple attempts at every question, but in the process waste time and points.

Teamplay

Three teams or three individual players each answer a series of questions, to see who can answer the most correctly.

This competitive game can be used with teams of players in a classroom.

faceoff

Two teams or two individual players each answer a series of questions, to see who can answer the most correctly.

This variant on the competitive game pits two players sitting round the same computer against each other.

Coming in part 5: Making your quizzes robust

A practical guide to creating quizzes: part 3

Practical guidesIn part 1, we looked at the characteristics of online quizzes and explored how they could be used to assist or assess learning. In part 2, we explored the various question formats and the types of learning for which they are suited. In this part, we move on the writing of the questions, in particular the traps to avoid.

Question writers are faced with two tricky problems:

  1. They do not want to make the correct answers too obvious but, on the other hand, they don’t want to leave these answers open to challenge. So, they put an awful lot of care into how they phrase the right answers, to make them absolutely clear. In the process, they often give the answers away.
  2. They need to tempt the user with distractors (wrong answers) that are genuinely distracting. To be distracting, they must be plausible. The trouble is that plausible distractors – tempting but still unambiguously wrong - can be hard to come by, particularly when you’re testing knowledge of rules and principles. Still sure you want to write quiz questions?

We’ll explore both of these traps, with the help of some examples.

Obvious answers

In this first case, most learners will pick the fourth option. Why? Because it is the longest. That’s because the question writer will spend a disproportionate time making sure the correct answer is absolutely precise. As a result, it stands out. Also, the question writer was obviously struggling to come up with a plausible fourth option, which is why they threw in option 3 – an amusing throw-away, but certainly not a genuine distractor.

This second example is just carelessness. Only the word ‘elephant’ fits grammatically with the stem of the question.

Here’s another frequent mistake. The absolute answers (‘no men’ and ‘all men’) are clearly less likely than the softer ‘some men’. No learner is going to be fooled by these distractors.

Another lazy cop out is to use ‘none of the above’. It’s a way of telling the learner that you’re having trouble phrasing a correct answer. And option 3 is another throw-away.

Confusing questions

Learners don’t mind obvious answers, because it makes their job easier. But they certainly get annoyed if they are presented with questions that they don’t understand. Take these examples …

It’s bad enough having a negative in the question stem (“Which of the following is NOT …”), but a double negative such as “never unnecessary” makes the question really hard to fathom out.

There is no grammatical consistency in these options. Each one should be phrased in the same way and flow nicely from the question stem.

The problem here is that the learner could enter their answer in so many different ways: “CERN”, “CERN in Geneva”, “Switzerland”, even “Centre Européen de Recherches Nucléaires”. Text input questions are fine, but you must make it absolutely clear what sort of answer you are expecting, for example, “In what city was the research establishment where the World Wide Web was invented?” Even then you be prepared to accept both “Geneva” and “Genève”.

The problem here is that it is not clear whether you are looking for a single response or multiple responses. Experienced computer users will recognise that the use of check boxes implies you can pick any number of options, but that won’t be obvious to everyone. Better to say “You can choose more than one option.”

Chances are you won’t see your own obvious answers and confusing questions. You have no option really but to have your quiz questions tested by typical learners. Believe me, you will learn lots from what they have to tell you.

Coming in part 4: Quizzes as games

A practical guide to creating quizzes: part 2

Practical guidesIn part 1, we looked at the characteristics of online quizzes and explored how they could be used to assist or assess learning. In this instalment, we look at the various question formats and the types of learning for which they are suited.

Factual knowledge

In an adult learning context, factual information is usually supplemental to the core learning objective and more often than not just for general interest. However, some facts really do need to be known by heart: When was … ? What is … ? Who is … ?

If it is essential that the learner can recall the information without prompting, then you have little choice than to ask a question that requires them to type the answer in. If it is only necessary that they are able to recognise the right answer, then various forms of multiple choice will do.

Testing factual knowledge

In the example on the left, the learner is tested for recall of a date, which they must type in to the text box. The example on the right also checks for recall, in this case of a chronological sequence.

Conceptual knowledge

Concepts provide a common language for understanding a subject. Generally the aim is for the user to be able to identify the class or category to which given objects belong, whether these are tangible (like types of computer) or abstract (like schools of thought). The most common way of checking this knowledge is to provide the learner with examples and ask them to place these in the correct categories, as in the examples below:

Testing conceptual knowledge

On the left is a typical matching exercise in which learners place example foods into categories. The alternative format on the right has learners select those examples which belong to a given category.

Process knowledge

A process explains how something works as a chain of cause and effect relationships. To check understanding of a process, you can ask questions about causes or about effects, as shown below:

Testing process knowledge

In the left hand example the learner identifies a probable cause. On the right, the learner looks at possible effects.

Spatial knowledge

In this instance our aim is for the learner to be able to identify the locations of parts of an object, device, physical space or system. The easiest way to check this knowledge is with a question that has the learner click on a given part as shown below:

Testing spatial knowledge

The example on the left has the learner identify a particular bone on a picture of a skeleton. The task on the right is similar but in this case the object is a software interface.

Procedural knowledge

Procedural knowledge is tougher because in many cases what you really want to test is whether the learner can actually carry out the procedure rather than just answer questions about it. However procedural knowledge is a first step and you can use a variety of questions to check learning:

Testing procedural knowledge

Rules and principles determine how a procedure is implemented in specific cases. The example on the left explores how different principles could be applied to a particular situation. The example on the right checks that the learner knows the correct order in which procedural steps should be applied.

These examples were created in Articulate QuizMaker, although many quiz tools could do a similar job. In the next instalment we look at the principles underlying the writing of quiz questions.

A practical guide to creating quizzes: part 1

Practical guidesWe all know what a quiz is. It’s a test of knowledge, typically accomplished by asking a series of questions.

Quizzes are popular in the digital environment, not least because computers find it so easy to deliver the questions and score the answers. In fact, if you were in your first week of a programming course, you’d probably have a go at putting together a multiple choice quiz. Quizzes are an entertaining diversion, particularly when delivered within the context of a game, with rules, levels, competition and prizes, but they can also play a useful role within a learning solution. A function that is often abused, perhaps, but the potential is there.

Quiz shows

TV quiz shows make for good popular entertainment and similar formats can be used within computer-based quizzes

Media elements

Although many quizzes are primarily textual, the possibility is there to use every media element. Images can provide the basis for questions that test for recognition of people, objects or places or to locate elements within interfaces and other spaces. Video can be used to portray situations that test the learner’s ability to make critical judgements. Audio can be employed to check for recognition of voices or pieces of music. A variety of media can also be used to introduce questions and provide feedback.

Interactive capability

Quizzes are essentially interactive. They serve their function in testing knowledge only by eliciting responses from learners. Just about any input device imaginable can be used as the basis for that interaction – key presses, mouse clicks, touches, the lot.

Applications

The most common application for a quiz is as a test of mastery. This is fine in principle as long as it really is possible for the knowledge and skills in question to be assessed by the sort of questions that a computer can deliver. To state the obvious, you might be able to check that a pilot understands the principles of aerodynamics using a quiz, but you can’t check they can fly the plane. Some caution also needs to be taken in terms of when a quiz is delivered. If the quiz comes right after the delivery of content (and the learner knows it’s coming), it is all too easy for the learner to hold on to enough of the information to get them through the quiz, but then forget it all the day after. We can probably all remember how possible it was to cram in information before an exam, only to see that evaporate almost as soon as we committed it to paper. A much more valid test of knowledge comes weeks, months or years after original exposure to the information.

Math quiz creator

Tools like this low-cost Math Quiz Creator can be used not only for assessment, but to provide loads of valuable practice

Although their potential is rarely exploited to the full, quizzes can actually play a useful role at just about every stage in the learning process:

  • As a way, right up front, for the learner to find out how much they already know and how much they need to know. This sort of diagnostic pre-test not only demonstrates the need for learning, it helps to direct the learner to content that is likely to be most useful.
  • As a vehicle for delivering the learning content itself. One way to create an engaging lesson is to use a series of quiz questions to challenge and then build on the learner’s prior knowledge. Every question alerts the learner to a gap to fill and all you have to do is oblige.
  • As a means for repetitive drill and practice. Unlike teachers, computers never get bored asking questions and they don’t lose their patience when the learner takes a little longer than expected to get the point. In the classroom, most knowledge is under-rehearsed and most skills under-practised. Quizzes represent a good way to remedy that.

So how do I get started?

There is no shortage of tools for creating quizzes. Most cover the usual range of questions types – multi-choice, multi-answer, free text response, sequencing, matching, selecting hotspots and all sorts of variations. All e-learning authoring tools come with a quiz making capability, plus there are specialist stand-alone tools, including ones that you can use for high-stakes assessments or for quiz games.

Articulate Quizmaker

You will find a quiz-building capability in just about any authoring tool, including this one which forms part of Articulate Studio

In practice, it’s likely that tools will be the least of your problems. Writing the questions is a much more challenging task, and that’s where we’ll be directing our attention next.

Coming in part 2: using the correct question for the job

A practical guide to creating learning tutorials: part 3

Practical guidesIn part 1 of this Practical Guide, we examined the history, characteristics and benefits of the digital learning tutorial. In the second part, we explored some strategies you can use to design tutorials that impart important knowledge. In this third and final part, we look at how tutorials can be used to teach procedures.

Engage the learner

As we discussed in the previous part of this guide, you cannot simply assume that the learner will come to your tutorial full of enthusiasm for the topic. Your task is to convey the importance of the topic and its relevance to the learner’s job. The simplest way to do this is just to explain, but you can achieve a more powerful effect through some form of introductory activity.

Engaging the learner

This activity opens a tutorial on greeting a customer on the telephone. It demonstrates why the opening is so important.

Engaging the learner

This activity, in a module that teaches how to use styles in Microsoft Word, aims to establish relevance of the topic.

Engaging the learner

The next step in the module is to demonstrate just how much time can be saved by using Word styles.

Explain and demonstrate

Your next step is to provide a quick overview of the steps in the procedure. It will help the learner if you present the big picture before going into detail.

Then explain or demonstrate the procedure step-by-step, explaining any special rules that need to be followed at each step.

Demonstrating 1

Here the five steps in the procedure are presented one at a time using simple animations and text. This is followed by a simple tabular summary, which could also serve as a job aid.

Demonstrating 2

In this example, screencasts are used to present each step. Again, the key points are also summarised in simple tablular form.

Provide an opportunity for safe practice

It’s one thing to understand a procedure. It’s quite another to be able to put it into practice. It takes time to turn knowledge into skill and it’s unlikely that your tutorial will do much more than kick-start this process. It’s your job to provide the learner with the opportunity to take their first step, with a simple yet challenging activity which mirrors the real world as closely as possible.

With a complex procedure, you may want to provide a practice activity at each step. In this case, it’s likely that you’ll cover each step in a separate tutorial. Don’t forget to bring the whole procedure together at the end, as in real life steps are not carried out in isolation.

Providing an opportunity for practice

This drag and drop activity requires the learner to place the steps in the procedure into sequence. This is still not checking for the ability to apply the skill, so another activity will be needed which has the learner carry out a task using this knowledge.

One of the ways that you can provide practice opportunities is using learning scenarios. For more information, see Onlignment’s Practical guide to creating learning scenarios.

Point to the next step

A how-to tutorial is the first step in learning a new skill. In many cases the learner will be able to take things on from there on their own, but where the skills require a great deal more safe practice before they are applied on-the-job, you may find you have to organise further practice opportunities using simulations, role plays and workshop activities.

That concludes this Practical Guide. It is now also available as a PDF download.

Next up: A practical guide to creating quizzes.