How do you respond when stakeholders ask you to measure how fast learners respond to questions? How do you train for fluency when the speed and accuracy of performance truly matter?
During my recent webinar on crafting better feedback for scenario-based learning, I talked about the benefits and drawbacks of using time as feedback. Measuring and assessing users on fast they respond to questions seems like good idea, right? After all, you want people to perform quickly and think fast.
But in practice, focusing on the speed of responses to questions can backfire. It creates accessibility barriers and can encourage random guessing rather than thoughtful responses. Artificial time limits can increase stress and decrease motivation. Therefore, most of the time I recommend avoiding timing users and requiring responses within a certain time frame.
When speed matters
In my webinar, after I suggested that we generally avoid using time, Clark Quinn asked about the situations when time is a criteria in the performance environment. Anuradha Gopu gave a good example in chat: in call centers, it matters how long customers are put on hold. Response time matters to customers, so the speed with which agents can answer calls and solve problems is a performance measure.
Acting quickly in an emergency is another skill where seconds matter. One great example is the Lifesaver training on what to do if someone has a sudden cardiac arrest. For those skills, assessing time to respond in the training is clearly tied to performance.
Speed + accuracy = fluency
When we talk about speed, what we’re usually really talking about is fluency: a combination of both speed and accuracy. Fluency is often what differentiates experts from others in a field. Speed without accuracy clearly isn’t the desired outcome; we don’t want a bunch of mistakes.
But what about the reverse? If someone can perform with 100% accuracy but is very slow at the task, would you consider them an expert? Probably not. They might be good enough to get started in a performance task, knowing that their speed will improve over time. But they’re not an expert–at least not yet.
Fluency requires lots of practice
Most of the time, when we create training, we provide perhaps a little bit of opportunity to practice. I use mini-scenarios and branching scenarios to help learners practice making relevant decisions. Sometimes I use simulations to practice in a more realistic environment. Those strategies are valuable and important, but they’re not enough to train people to fluency.
Fluency requires lots and lots of practice. If people need to learn to the level of competence where they can perform with both speed and accuracy, they have to practice the skill over and over.
We don’t really have any shortcuts to avoid repeated practice. We can make the practice more effective and efficient by providing feedback that helps learners make better decisions. Interleaving, spaced repetition, and retrieval practice are all evidence-informed techniques for improving practice activities.
But ultimately, any training that is a “one and done” event can’t get learners to the level of fluency. There just isn’t enough time to practice. People may perform with 100% accuracy in practice activities without reaching a level of fluency.
What if a stakeholder asks for training for fluency?
“Can we time how fast the learners answer these questions?” Camden, the manager, asks.
“Well…timing users can be problematic. It creates accessibility issues…” you reply.
Camden interrupts. “But it’s really important! In their actual work, they need to do it quickly. It really matters how fast they are at this skill.”
You ask some follow-up questions and conclude that Camden has a point. Fluency probably is important for this skill.
“OK, thanks for the additional information. Given those requirements, yes, we can time the learners, and we’ll need to provide multiple practice activities. At the beginning, when they’re first learning, we won’t time them. That will give them the initial opportunity to practice without the pressure of a stopwatch. We’ll need several practice opportunities over time to help reinforce that. We might need a job aid to provide support and reminders too. Once they’re fairly accurate without time limits, then we’ll work on increasing speed. We’ll have to look at accessibility too; I need to find out what accommodations people actually use on the job to help with this.”
“Do we have to do all of that? Can’t we just jump right to timing them in the first practice and be done with it?”
You explain, “You said this is really important, and speed matters for this skill, right?”
“Right,” Camden acknowledges.
“So if it’s important, then we need to spend more time on it. Think about something you can do really fast now like…oh, multiplication facts. How much time did you need to spend practicing those to memorize them and be able to do them quickly?”
“Oh, I remember spending a lot of time with flashcards and games to learn multiplication!”
“That’s how much practice it takes! Even something like using software is the same. You know Ctrl+C and Ctrl+V for copy and paste, and those are so fast that they’re automatic for you now. But how many times have you used those shortcuts?”
“Hundreds, maybe thousands. OK, I get your point. And yeah, I think this is important enough to be worth spending more time practicing.”
“Yes, and” responses to stakeholder requests
As a slight tangent, did you notice the response in the example conversation? “Yes, we can do that, and we’ll need XYZ.” “Yes, and” is a great approach to handling many different stakeholder requests. You tell them that yes, you can do what they want, and then build on it. In the follow-up, I often explain the consequences of that request. It’s a way of agreeing to what stakeholders ask for (like timing how fast learners are) while managing expectations. Using a “yes, and” approach lets you keep the conversation going while being realistic about scope and other constraints.
Here are a few more examples of “yes, and” responses for managing project expectations.
- “Yes, we can add two more videos to this training, and it will push the delivery date out by another four weeks.”
- “Yes, we can replace the photos with custom illustrations throughout the whole module, and it will cost $$$$ for the illustrator plus add 3 weeks to the development time.”
- “Yes, you can have as many reviewers as you want provide feedback on the storyboard. You will need to compile all of that feedback into a single document and assign one person to be the ultimate approver and ‘tiebreaker’ for any conflicting feedback.”
Fluency with practice and feedback
Back to the original topic! Training people to a level of fluency requires a significant amount of relevant practice with effective feedback. It’s possible to do, but it’s an investment to reach that goal. Save this for the times when it really does matter. But, if a stakeholder asks for training for speed, view it as an opportunity to advocate for more practice with feedback.
More resources on practice and feedback
Watch for clips from my webinar Beyond Right or Wrong: How to Craft Better Feedback for Scenarios on the TrainingPros YouTube channel. As of this writing, three clips are available.
- Improve Feedback to Increase the Impact of Your eLearning
- What is the Difference Between Intrinsic and Instructional Feedback?
- Forge of Empires: Inspiration for Better eLearning Feedback
If you’re interested in learning more about practice and feedback, check out these books.
- Practice and Feedback for Deeper Learning by Patti Shank
- Make It Stick: The Science of Successful Learning by Brown, Roediger, and McDonald (This book is about much more than practice and feedback. If you want to learn more about the techniques I briefly mentioned like interleaving, spaced repetition, and retrieval practice, check out this book.)
Update: More resources on training for fluency
I reviewed some of Carl Binder’s work when I wrote this post. The definition of speed plus accuracy that I linked to above comes from his work. Carl commented below, and I want to include his links here so they’re easier to find.
The most complete summaries of what we learned over several decades about building fluent performance are here. First, a publication that was one of the most frequently cited in the 40 year history of The Behavior Analyst, the flagship journal of the Association for Behavior Analysis International (http://binde1.verio.com/wb_fluency.org/Publications/Binder1996.pdf), then a publication done a bit earlier more for an instructional technology audience (http://binde1.verio.com/wb_fluency.org/Publications/Binder1993.pdf), and finally an example of building fluency in a customer call center where we were able to reduce time-to-full productivity for new customer service reps from two months to two weeks (http://binde1.verio.com/wb_fluency.org/Publications/BinderSweeney2002.pdf). The bottom line about measuring performance with the time dimension (usually count per minute) is that without it we actually cannot define what it means to have mastered a skill or body of knowledge.-Carl Binder
Tuesday, October 31, 3:00 PM EDT: Level Up Your Elearning: Character Creation for Scenario-Based Learning. Part of TLDC’s free event From Instructional Design to Dungeons & Dragons: The Chronicles of Educaria.
In Dungeons & Dragons, character creation is the foundation of epic storytelling. In learning and development, the creation of characters plays a pivotal role in scenario-based learning. For this session, you will complete activities focused on shaping character backstories, defining their objectives, and constructing challenges that spark curiosity and foster learning. Learn tips for creating characters who are both relevant to your training context and interesting enough to spark attention. A good character for scenario-based learning is one your learners can identify with and that draws them into the story. Just like in RPGs, creating characters for workplace training scenarios requires a bit of imagination. Plan to actively participate in this session and practice creating both protagonists or player characters (PCs) and additional non-player characters (NPCs).