Stephen Lowe |

Zero budget gamification of online learning

Posted on September 11, 2019 by Stephen Lowe

In my last post I wrote about game-based learning and differentiated it from gamification. In this article I look at the gamification of online courses. I suggest that gamification is not only about points, leaderboards, and badges. It may simply be about the language we use, and our approach. However the learning experience designer chooses to go about it, I assert that it will always have three dimensions: harder, faster, and levelling up.

Image source: Wikimedia Commons, public domain

Gamification of online learning is a serious business. Let’s start by making a collaborative document and setting out a table of goals and actions. Let’s wrap some theory around this, and pull together a cross-disciplinary team. Yeah, right. Or let’s sit on the floor, behave like kids, think like Māui the Trickster, and have some fun with it. Which do you think will work best? Are you on for this? Press START.

Harder

Sure, you want to make it easy to get started, but soon after that you want to get busy putting artificial obstacles in the way. Not too many. You have to get just the right balance between difficulty and progress. If you ever played Machinarium (insert any dungeon crawl you did play) you will know that the thing that kept you coming back was that it was insanely hard to find your way out of one room and into the next.

Everyone likes to quiz

Quizzes, puzzles, and challenges are a great way to structure the content. You don’t necessarily even have to lay the content out first, rather you just embed it in the quiz. Or flip the usual layout: put the quiz first and put the content second, served up like a resource to be trawled for facts if required. Crafting Socratic questions into the feedback fields of a multi-choice will lead the learner to the right answer. Give them multiple attempts, encourage guessing, require a high passmark like 90% or even 100%. Moodle and H5P (and many other platforms) offer the learning designer different flavours of multi-choice, no longer are you stuck with radio buttons and checkboxes.

Demand submissions to unlock the levels

There’s a good learning model that lends itself well to gamified online learning: aggregate, remix, repurpose, feed forward (Cormier). Require your learners to collect new knowledge, synthesise it into something of their own, apply it to a problem, and submit it for examination by the Old Wizard to allow them passage to the next level. No passkey, no access. Learners of all ages understand and acknowledge this motif; that is why Rowling and Tolkien are such popular authors.

Instant feedback

For every hard thing there must be immediate recognition and reward for achievement. For every failure there must be a penalty, an instant fine. Forget higher-order learning strategies, this is pure behaviourism. That is not to say that learning strategies that work with the intrinsic motivation of the player-learner cannot be deployed inside each discrete activity. But progress through the learning landscape is old-school carrot and stick. It’s all about framing, you have to make receiving the penalty part of the fun. Drop and give me fifty, soldier. If that doesn’t sit well with you, then maybe this is time to go for a long walk and ask yourself The Big Question: Is gamification for me?

Well-crafted games are an artful blend of intrinsic pleasure and extrinsic scaffolding.

Amy Jo Kim

Faster

If computers sap your energy, they also eat time. When someone says to me ”online learning”, I see a pool of light from an angle poise light. The clock says one a.m. and everyone in the house has gone to bed except me. But you can do more with a clock than stare at it. You can use it as a powerful motivator. Just as a fitness freak says I wonder if I can run 10 kilometres in an hour, so a learner releases endorphins by saying I wonder if I can complete this quiz in under five minutes for 500 bonus XP. It’s a balance of anxiety against boredom. Yes, your river runs deep, but it flows swiftly too.

Far-off deadlines kill online learning

Like Christopher Columbus, online learners need rhythm for a compass. A strict weekly beat marked out by the Metro Gnome is what makes eLearning go. Practice it in the mirror: “I need you to do the reading, score 100% in the quiz, and submit the assignment by this time next week.” Depending on your context you might even want to substitute ‘day’ for ‘week’. Oh, and some points, tokens, awards, and a place on the leaderboard will really help to make this happen. It won’t work for everybody. You’ll need a rescue truck. Tick the boxes that enable multiple attempts and late submissions.

Treasure hunts are fun

Put those potentially isolated individual learners into teams. Give those teams fun names like ‘The Tigers’. Put up a prize like 1000 XP for the winning team and 250 XP for entering. Now, set the Challenge. Imagine you can contrive for each individual to find a nugget of learning, carry it to some forum or wiki where the first team to assemble all the nuggets into a golden chalice wins. I’m being metaphorical here, I’m not suggesting you have to be a world-class animator. What I am saying is that dividing the learning, running with it along some pathway, collecting it into one place, and combining it into an exhibit is a powerful model. This kind of thing will get the majority of the learners through into the next Level. I’m assuming you’re either a teacher or a learning designer, so you no doubt have a rich repertoire of creative ideas you can bring to dry old online learning.

Nano-learning builds skills

Provide resources that learners can do in the interstitial spaces in their life: waiting for mum, stuck in traffic, medical centre waiting room. Flip cards are a good example. If you get 25 XP every time you run through them, why would you not run through them 4 times? Drag and drop, same story. Barbara Oakley emphasises retrieval practice. She says it’s easy to kid yourself you’re learning when all you’re really doing is repetition. Try breaking this cycle by creating a five-question quiz of the text box type where the learner must actually type five remembered words, correctly spelt, to gain the XP. Repeat the quiz on each level, but progressively reduce the time in which it must be done. Now the learner is tapping right into their neural networks, retrieving, and articulating. Harder. Better.

Level Up

In games, we are the protagonist—the person with agency, facing a series of choices and challenges along our journey towards mastery.

Amy Jo Kim

The reason progression through Modules 1, 2, 3, and 4 can get boring is that there’s no sense of getting stronger to face greater challenges. Too often it’s just more of the same, resulting in attrition from about mid-course on. In games your character gains knowledge, skills, tools, and powers in each Level to prepare them for the challenges that lie ahead. Online learning is sometimes doing that, but it’s not worn on the sleeve like it is in games.

Mind your language

Gamification of online learning could be achieved simply by the language we use. For example, “Be sure you know the twenty terms in the glossary by heart because you are going to need this knowledge to complete the challenges in the next Module. There is a twenty-question quiz on the terms in the glossary, you must get 90% to unlock the next Module.”

Nudge, tempt, shove

Gamification of online learning may be no more than using some well-crafted well-placed nudges. We are herd animals. Some will be happy to be the stragglers at the back, but they won’t want to be left out altogether. Try announcing “half the class have now moved to the next Module”. The principles of universal design for learning apply to gamified courses just the same, so you want to offer multiple pathways to unlock the next Level. However, if one of these pathways has a track-record of success you may want to make it the default pathway, or sign-post it as ‘recommended’, or simply call it the Yellow Brick Road.

Provide real time support

Pretend you’re Florence Nightingale and create a cool chart. It might show how half the learners drop out when they fail the first level-up quiz. Watch the activity logs or if you’re geek-gurl set them up to notify you. Get in there at fail time with consolation prizes and words of encouragement. Throw XP around like confetti (after all, it’s virtual and it doesn’t cost a cent). Do not do what so many online teachers do, and abandon them in their hour of need. Richard Bartle, in his wonderful book Designing Virtual Worlds, calls this the Live Team. You are the live team.

It’s a wrap

There are a lot of products on the market. I am assuming, reflected in the title to this piece, that you are not in the market for those. Moodle continues to be one of the most powerful free environments for online learning. Each release it gets better and better. Now you can record video and audio directly into a forum post. How good is that? This is where the individual learners could bring their nuggets of new-found knowledge to assemble them into that golden chalice.

As a teacher you must design it. You must create situations that demand sharing.

Jesse Schell

If you’re a teacher you may like to approach it from first principles and know why you’re doing it before you address the how. Like me, you may have to approach this in a creative way and hack your way to success on a shoestring. Good luck, and follow the code: harder, faster, level up!

By the way, did you uncover the Old Pirate Pass Phrase hidden in the text? Clue: “Let no word go unturned until you have the key. Hover your mouse my hearties only then I’ll set ye free. For where you see gold, there treasure be!” Email the pass phrase to stephen.lowe@core-ed.org to claim your share of the treasure!

Suggested resources

Jesse Schell, Learning is beautiful. YouTube Video (23 mins)

Amy Jo Kim. Game Thinking: Innovate smarter & drive deep engagement with design techniques from hit games. (Available from Amazon)

Richard Bartle. Designing Virtual Worlds 1st Edition. (Available from Amazon)

Ploy Buraparate. Dungeons & Dragons & Design Thinking on the UX Collective Blog.

Moodle plugins Level Up.

Featured image by N. on Unsplash

Posted in

Rocket science

Posted on April 10, 2019 by Stephen Lowe

Game-based learning is a big subject, and you could go broad, or you could go deep. It means very different things to very different people. I’m thinking about just one facet of it, gaining an introduction to computer science and computational thinking through the design of games.

Before we go any further, we need to do some disambiguation.

This is not gamification. Gamification is the application of game techniques in non-game environments. It is used as an extrinsic motivator in regular courseware, and it usually takes the form of points, a leaderboard, and badges. It may be used in more subtle ways, like upvoting in forums and a reward for best post.

Nor am I talking about fully-worked strategy games like Civilization, or fast action multi-player games, not in this article. Creating games like these is at a level of human achievement second only to building a railway through the jungle.

What I am talking about is scrolling platform games, like Super Mario, and top-down adventures, explorations, and simulations. The emphasis is on storytelling, and there are strong links to the curriculum. Using a web application such as Gamefroot, which supports block coding, students can start from a young age exploring the wonderful sense of agency that happens when you tell computers what to do.

In this initial sketch the scoping exercise is done. It is easy, for example, to draw a robotic arm for manipulating rock samples, but to animate and code it would truly be a mission into deep space. Note that the learner is starting to consider the forces acting on the rocket: gravity, weight, and drag and the thrust that will be needed to overcome them. Graphic by Stephen Lowe, All Rights Reserved. — In this initial sketch the scoping exercise is done. It is easy, for example, to draw a robotic arm for manipulating rock samples, but to animate and code it would truly be a mission into deep space. Note that the learner is starting to consider the forces acting on the rocket: gravity, weight, and drag and the thrust that will be needed to overcome them. Graphic by Stephen Lowe, all rights reserved.

In the future everyone will not be a computer programmer. So what is the point of students learning coding at school?

Edsger Dijkstra, the revered matua of computer programming, is famous for saying, “Computer Science is no more about computers than astronomy is about telescopes.” What he was trying to say is that Computer Science is a way of thinking, a lens through which to understand the world. Living in a world where even your wristwatch is a computer, where robots do factory jobs, and driverless taxis take us across town it has to be important to have this Computational Thinking lens in your kete. It’s the new literacy.

“Computer Science is no more about computers than astronomy is about telescopes”
― Edsger Dijkstra

Isn’t Computational Thinking just a buzz word for mental arithmetic?

I’d be the first person to lament the passing of mental arithmetic. It’s a skill, a knack, that was distributed through all layers of our society in a time when we still carried notes and coins in our purses and calculators were heavy mechanical devices. Take the slide-rule, a more portable calculator of the recent past. For an engineer or a navigator to use a slide rule they had to know the order of the answer before they started, the slide rule merely filled in the detail. Shop assistants working the till could do mental gymnastics at light speed. Gone. Now the domain of eccentric hobbyists.

Computational thinking is a different animal altogether. It’s about decomposition, abstraction, algorithm design, and pattern recognition. When we do stuff like that, far from weakening our brains, we start to see the world in a whole different way. Seeing things in a different way is one of the most important things we as humans can learn, because it gives us what we need to adapt and survive.

Firm foundations

Game-based learning as it might be practiced in schools today stands on firm foundations. Surprisingly perhaps, it is neither new, nor is it experimental.

Seymour Papert’s book Mindstorms: Children, Computers, and Powerful Ideas was published in 1980. That’s forty years ago. In the foreword Papert refers to the transitional object. That’s what the sprites in the game become, objects that embody higher and more abstract concepts. Objects to think with.

Mitchell Resnick, who was a student of Papert, started Computer Clubhouse in 1993. This free after school club instantiated Papert’s thesis of social constructionism. Today it has matured into The Clubhouse Network “where young people from underserved communities work with adult mentors to explore their own ideas, develop new skills, and build confidence in themselves through the use of technology”. It is a worldwide network with nodes in eighteen countries.

Sugata Mitra is best known for his Hole in the Wall Project in 1999. It has come in for quite a bit of criticism, but personally I buy into it. The gist of it is that, left to get on with it, children learn naturally. The role of the teacher changes, providing the learners with a safe environment, resources, and encouragement. Game-based education has elements of MIE (Minimally Invasive Education) and SOLE (Self Organised Learning Environments). This is in keeping with modern trends in education where children are afforded greater agency.

Here the final artwork has been completed. Fully saturated colours have been chosen, and some of the finer detail dropped to create a pleasing cartoon representation of a rocket. Note that the learner has had to explore the genre and will have tested their colour scheme against intended backgrounds. Graphic by Stephen Lowe, all rights reserved.

Objects to think with

I’d like to quote Seymour Papert here, from the transcript of a speech he gave in 1998: “My goal in life, which has been my major activity over the last 10 years, has been to find ways children can use this technology as a constructive medium to do things that no child could do before, to do things at a level of complexity that was not previously accessible to children.”

So what are these transitional objects called sprites and where do you get them?

Sprites are the actors in a game. In a gaming world an object like a planet or a rock might be as much an actor as is a rocket or a cosmonaut. Learners drag objects from a library onto the stage, at which point they become sprites. Scripts are attached to them to cause them to do things like move, interact with other sprites, and interact with the scene. The scene is made up of tiles. Learners can use ready-made collections of objects, or they can draw their own according to the time they have available, their level of study, and their aptitude.

What are the powerful ideas, the higher and more abstract concepts?

So, it’s pretty easy to understand that you can attach scripts, and tell your rocket to move. But let’s extend this just a little bit and see where it goes. Assume in the first place you want to escape Earth’s gravity. How much thrust will you need, and for how long? The space shuttle needed 1.2 million pounds of thrust for 6 minutes to reach orbit at 17,000 miles per hour. Now do you see where this is going? How this meshes with STEM?

Let us now visit three planets, each with a different gravity to Earth. We want gentle landings and successful escape from their gravitational fields. This simulation, with some timely prompts from a game-savvy teacher, will cover Science, Technology, Engineering, and Maths. It’s going to be fun. But it’s going to be “hard fun”, a term Papert himself used.

And the higher and more abstract concepts, what about them? Computational Thinking, as we explored earlier, is not just maths. Computational Thinking is about how to think and how to solve problems. It’s about Decomposition, Abstraction, Algorithm design, and Pattern recognition. The meat in tomorrow’s sandwich.

While Gamefroot comes with a good library of game objects there is something satisfying about drawing your own. Gamefroot is about learning coding, but it can be about creating game art too. Note that in the attached script values are assigned to the concepts of gravity, thrust, acceleration, and drag. Experimenting with the balance of these values will affect the behaviour of the rocket. Graphics by Stephen Lowe and Gamefroot, all rights reserved.

The challenges we face

I hope the argument I have presented here helps in the challenge to convince sceptical parents that games-based learning is not an ill-defined liberal whim, but a well-established pedagogy standing on solid foundations of research and practice.

Working with individual students to move them from fun to hard fun will sometimes be a challenge, but in many cases will be surprisingly easy. The joy of the games-based learning approach is that each learner will be able to perform to the best of their ability scaffolded by a framework that really knows no bounds.

Teachers will face challenges. It is important to remember that you do not have to keep one click ahead of the kids, you can happily let them overtake you in that respect. If this aspect of game-based learning is stressing you I suggest you follow the links I offered earlier to Sugata Mitra’s work.

Suggested resources

Kia Takatū ā-Matihiku CORE Education are partners in this valuable self review tool. Find out how ready you are to implement the new curriculum content and catch up with students. Completing the review takes five to ten minutes and all of the results are confidential to you.

Educational Games Design Fundamentals: A journey to creating intrinsically motivating learning experiences by George Kalmpourtz. (Expensive but comprehensive).

Algorithms to Live By The Computer Science of Human Decisions by Brian Christian and Tom Griffiths (Requires no maths to get the most from it).

The Art of Game Design A Deck of Lenses by Jesse Schell (Useful in the classroom).

Posted in

Towards excellent user support

Posted on July 10, 2018 by Stephen Lowe

Background

If you attended university, you will remember that nobody really cared that much about whether you passed, failed, or simply dropped out. Unlike school, nobody asked you where you were yesterday. It was something of an awakening. You could learn from it, or you could make some expensive mistakes. Whether you passed or failed you would still have a student loan that looked like a telephone number!

There is a real danger that online learning can become a system of empty corridors and empty rooms. Opening a door, you may find a person in there silently reading a book, determined to complete some module. If there was a window, we’d see that it’s dark outside. That’s because, for many young professionals, the night is the only time they can find for their studies.

Taking a design approach

I’m starting with this dystopian view of online professional development on purpose to make a point. Excellent user support begins with empathy. I’m proposing that we start our journey towards excellent user support by picking a process. One such process that has emerged as a universal tool is Design Thinking. The flavour we use is the one taught at Stanford’s d.school under Professor Leticia Britos Cavagnaro. We empathise, define, ideate, prototype, and test — and we do it in a series of fast iterations.

We can’t go one step further without considering the morality of A/B testing. Josh Constine covers it off pretty well in a blog post he titles, The Morality of A/B Testing. If you want the skinny on it, it’s this: If you try out new ideas on Group A and they prove to be of great benefit, Group B have been disadvantaged. Conversely, if the new ideas prove to be a disaster, then Group A have been disadvantaged. So, what I am proposing are not experiments on real learner groups, but some thought experiments. Create five personas — archetypes of your learner base. Multiply them by two. Call them Group A and Group B. Use them how you want. Don’t worry, they’re only cardboard.

Let’s get empathising

These people who are our participants, it doesn’t really matter who they are, most have a number of things in common. Into their bucket they have to place some big rocks, then they’ll fit in some pebbles around those, as best they can. Then they’ll pour in the sand.

Let’s take Toni as an example. One of Toni’s big rocks is to take part in a marathon, for which she must get fit. This is not something she just wants to do, it is something she is absolutely committed to doing. Her next big rock is to get out of a shared flat and into a house of her own; she feels it as a pressing need and it can’t come soon enough. That makes keeping her job another big rock, because she needs the money. That’s three big rocks she has in her bucket already. She enjoys her job as a teacher. Being the best teacher she can for her class, she is always looking for new and better ways to facilitate their learning.

Time to start defining

0600 Alarm clock
0630 – 0700 Run
0730 – 0800 Prepare for some class
0800 – 0830 Eat breakfast on the train
0900…

OK, I won’t go on. You get the idea. If you’re reading this blog you may even be a school teacher, so you know what it’s like.

Ready to ideate

The seasoned design thinker will never shortchange the first two steps of the process. Peeps who haven’t yet learned to do Design Thinking tend to jump straight to ideating, based on a whole mass of assumptions. Those first two steps of Empathising and Defining will have uncovered some truths. One of these might be that professional development is not one of Toni’s big rocks. Even if it is, her bucket is already full of big rocks. I have an idea! Yes, what? Let’s make her online professional development pebble-sized. Cool idea, let’s try it!

Get cardboard and glue, prototype!

Wow! This is an amazing process… are we prototyping already? Yes, we are. But, before we can hope to have a workable solution we can expect to go through these steps a thousand times. It is said, if you really want to Samba you have to be prepared to dance.

So, one idea we are going to prototype and test is to make her online professional development pebble-sized. Make it so she can fit it in around her big rocks.

Now, we’re cutting out and gluing up a cube. A cube has five (just kidding) six faces. From the Empathising we did, we look for a problem Toni might have, and we write that on one face. Let’s say it’s Anzac Day (easy one for an example). Much of the school day will be taken up with Anzac Day type activities. So, Toni wants to get some mileage out of this and tie it to the curriculum. Write that on another face. But she also wants to peg it on a recognised learning strategy. Write some links on another face. Enough… you can work out the rest.

Let’s try it out on Toni

We don’t have a real Toni. But we’ve got this cardboard one. Let’s pretend she’s real and ask her what it will be like for her to use our cube. It’s portable, so she can do it on the train with one hand while she holds her croissant in the other (she can hold her coffee with her knees). It’s stateful, so, when she has to put it in her bag to get off the train, it remembers where she’s up to. When she gets to school, she can put it on the desk in front of her. It’s light and tough and wipe-clean so she can pass it around the room. We think we could be onto something.

It’s time to take stock of where we are in this piece about our journey towards excellent user support. Up until now it doesn’t seem to be about user support. But that is the point I’m making. User support begins with learning experience design. If the online professional development doesn’t meet the user’s needs, there will be no user to support. The better the learning experience design, the less need there will be for support. Design out every wrinkle that you can, and document what you can’t. A course that needs lots of meta-instruction and scaffolding is a poorly designed course that nobody wants. Our moon-shot is online professional development that you can pick up and run with without one word of explanation. Something that is truly fit for purpose.

What does excellent user support look like?

By following the process outlined and illustrated above, we came up with a definition of excellent user support. Now we have a manifesto to ideate around, prototype, and test.

The Learning Experience team at CORE think it looks like this:

Baked into the course design
Proactive and unobtrusive
Caring and supportive
Non-judgemental
Multi-modal.

Let’s unpack that now. At CORE, we currently work on three platforms:

iQualify
Matrix
Moodle.

I’m going to use Moodle for my examples, but I’m sure the concepts apply equally to any mainstream Learning Management System.

Baked into the course design

It’s really easy to build activities into a course that will tell you who is engaging and who needs some support wrapped around them. A synchronous session like a webinar requires that everybody rocks up at a pre-ordained time. It is usual to record the session for those who genuinely cannot attend due to other commitments. If a participant fails to show at the webinar and fails to access the recording, that’s a big white truce flag being waved by somebody who is losing the battle. You can switch on your protocols for an intervention. Forums are not so much of a barometer because some participants just hate forums, but they may still be accessing all the other resources and activities. Extrinsic motivators such as points, leaderboards, and badges may give an indication, but, be aware that some people will game the system. Let’s come back to that one another day.

Proactive and unobtrusive

A design flaw in early e-learning was the omnipresence of the course owner. Talking-head videos, overlay video lectures in unconvincing virtual environments, and responses to every post in the forums became seriously irksome for participants who felt they had no escape from their over-enthusiastic teacher. In my opinion, the online space belongs to the student, and the teacher is an observer who should only interject when necessary. Scaffolding, if it is required, should be baked into the course.

Platform support is similar. Tech support should be like the genie in Aladdin’s lamp. When you need them, they magically appear; when you don’t need them, they’re totally invisible. That doesn’t mean they can’t be watching. I dream of an online programme that has a Houston-like control room at its back. The learning designers go about their regular work on their next project, but in the room are large monitors displaying the live logs. When they see three failed login attempts they pop up a dialogue: “We see you’re having trouble logging in. How may we help you?” Help is not a dumb chatbot but a direct line to a human who knows their stuff. In this room, a morning stand-up meeting shares information about participants the system has advised are ‘at risk’. Interventions are planned. No-one is abandoned. No-one falls through the cracks.

Caring and supportive

Most learners are fragile. As I explored earlier in this piece through the personas, online professional development has to find its place in an already full bucket. When an intervention solicits an email response from a participant who says her computer crashed and she lost her assignment, we don’t go, “Oh, yeah! Heard that one before haha!” Instead, we put together an extension plan or offer her the option of dropping out of this delivery and joining the next one without incurring any extra charges. We do what we can, and our tone reflects the genuine concern we feel for her circumstances. Hopefully, we’re all human.

Non-judgemental

It is extremely interesting to me that when online learning designers sign up for MOOCs, they invariably fall behind and trot out all the usual excuses for failing to keep up! It is not our business to judge others. Our job is to reach out an arm to people drowning in the busyness of their own lives or facing their inability to assimilate and retain new information. A holistic approach, seeing online professional development in the bigger picture of their work-life balance, and their general well-being pays the greatest dividends in the long haul.

Multi-modal

Many communications channels are available to us now. That the IT department of a large organisation offers help only through the one channel of a ticket system makes good sense. But the learning experience team can probably afford to be more generous and offer support through email, instant messaging, and mobile phone. The main thing is to know who on the team has picked up a support call, so we don’t fall over each other in the rush to help. Our method is a Gmail group, and we have one for each of our main platforms, Moodle, and Matrix. Any support call becomes somebody’s top priority until the issue is resolved.

Conclusion

I like to understand things by extrapolating to the ludicrous. Let me do that now. If we could design online learning that had zero defects and a perfect user interface, then the need for user support would vanish. But, in this imaginary future, we also have a home-help robot that spends the day playing Canasta with granny. It cleans the house and starts cooking tea ready for when we get home. If anything goes wrong with the robot, I think I’d want to be able to call a helpline and talk to an empathic human agent.

If your users are having anything other than an excellent learning experience, you might want to engage CORE’s LX team in a conversation. Previously, we have worked in the primary, secondary, tertiary, and public sectors. Our experience in the private sector is somewhat limited, but all the same principles apply. We can create an action plan for you to take forward, or we can stay on and work with you to implement it. Contact us.

Resources

CORE LX can supply you with some cardboard cut-out personas, we’ll be interested to hear what you’re doing. Send email to Stephen Lowe.

A website that offers free patterns for prototyping/teaching materials is http://atozteacherstuff.com/

Anyone can do a Virtual Crash Course in Design Thinking on the d.school website.

See other posts in this series

Stephen is writing a series of posts on online learning design. Here are the first two in the series

Photo credits:
Feature image (study desk at night) by maticulous on Flickr under CC 2.0 Generic (CC by 2.0)
Graphics by CORE

Posted in

Making a start with student data analysis

Posted on May 24, 2018 by Stephen Lowe

Background

In my last post, I unpacked the Learning Management System (LMS) and the Learning Record Store (LRS). In it, I introduced the idea of capturing fine-grained student data using the Experience Application Programming Interface (xAPI) to inform programme, course, and module design. I felt it may have left many readers with a bit of a tall order, and in need of a simpler place to start.

That was the case for me back in 2010 at a regional polytechnic. With classes of 30 students, lectures to prepare, assignments to mark, at the same time developing an online community and resource, to also try to go deep into user data analysis would have been a bridge too far.

In this post, I plan to offer a starting place; some low-hanging fruit. It’s a complete starter pack for someone who has a course in Moodle and no idea where to start. I am sure it would easily transfer to any mainstream LMS.

Identifying students at risk

In your role as an online teacher (tutor, instructor, facilitator), the first thing you will want to do is identify those students who are at risk.

There is no greater risk for a student than not being present. In a physical classroom that is obvious; there are some empty seats. On the first day of the class, Sheryl and Tania are missing. The teacher wonders if they are aware they should be in this class. Perhaps they did not read the notice that said the room had changed. Perhaps they have discovered a clash with another class. After the class, the teacher has a duty to follow up and find out where these two girls are.

The same thing happens in the online world, but all too often the absence goes unnoticed. The teacher’s attention is focused on the people who are in, say, the introductory webinar — whose sound is not working, who have too many distracting questions. Especially if the enrolment is large, Sheryl and Tania may not be missed.

There are ways to resolve the problem, like recruiting an assistant facilitator, but that’s not our focus here; I was using it only by way of illustration. Let’s focus on Sheryl and Tania and use the tools available to us to determine if they are at risk.

By clicking on the link in the People block, Moodle displays not only a list of the people in the course but also when they last logged on. If it says “NEVER” beside Sheryl and Tania, then you know you have two students at risk.

If that was all an online teacher ever did — follow up on the NEVERs — that would be good in itself. But, if you have an ounce of geek in you, then there is a whole lot more that you can do.

Exploratory data analysis

The thing about exploratory data analysis is… it’s exploratory. Think of your data as being like a territory, and you set out on foot to get to know it.

Whether you are a beginner, a journeyman, or an expert in data analysis you will always start by just considering the data. Look at it, shuffle it about a bit, make a cup of tea, ponder. Don’t rush in, you may miss something that is staring you in the face.

For example, look at these five email addresses:

john.smith@stjosephs.school.nz
mary.roberts@stjosephs.school.nz
lbooth@middlebeach.school.nz
quackers@hotmail.com
garyb@chchpoly.ac.nz

I won’t say anything for now, but we’ll come back to this list later in the post.

John Tukey

John Tukey is the father of exploratory data analysis, and he wrote a textbook of that title. Despite all the advances since he wrote it in 1977, it is still a good place to start your journey. If you are the textbook type, that is. You may prefer to simply learn by doing.

Let’s do a little Tukey-type thing now. He talks about, ”scratching down numbers”, and this method is called stem-and-leaf. It gives you a quick visualisation and, therefore, the beginnings of an understanding of the numbers.

I’ll make that a little easier to see:

15	9
14
13
12	6
11	4
10
9	9
8	113
7	00953
6	3208473
5	501479001
4	622712541
3	3741011679
2	63224
1	3217
0	9800

Let’s say that you’d like to quantify a feeling you have that your cohort of 60 students is contributing quite well to the discussion forum. The stem to the left of the line is the tens column; to the right the ones. As you can see in the image above, two students did not post at all. You can explain this by looking to see if they are Sheryl and Tania. If they weren’t logging in, they won’t have been posting. Two other students did a bit better: one of them posted 9 times, and one posted 8 times.

By looking at the shape of the numbers you’ll soon see that most students are posting between 26 times and 79 times. However, more interesting than the safe majority are maybe the unsafe minority. The two who have not posted at all we have already noted. Six students have posted less than 20 times whereas most of the class have posted around 30 or 40 times. Equally concerning could be that far-outlier; the one student who has posted 159 times! Is he or she devoting enough time to other parts of the work, or are they starting to display obsessive tendencies?

In the real-world classroom, you may know Mister159 and you may have strategies for him. In the online space, it is too easy to forget that each of these avatars, these electronic tokens, represents a real human being with real needs. It can also be hard to learn those needs over the mediated channel of a computer network.

Brinkerhoff SCM

Robert Brinkerhoff’s Success Case Method uses impactful user stories to evaluate and later inform the design of educational interventions and programmes. To make a proper study of the method would be beyond the scope of this article, but I will give you the plain language guts of it here, and you can start using it straight away.

Here’s how:

Ignore the average and the median students
Focus on the outliers
- identify 1 – 3 students who did exceptionally well on the course
- identify 1 – 3 who did particularly poorly
Arrange to interview them
Document their stories.

Try to avoid surveying them with a set of standard questions. That practice is boring, and students may amuse themselves by skewing their answers, and it will miss some of the rich detail a conversation can uncover. Put those who did particularly poorly at their ease and explain that you are speaking with them in order to improve the course. Also, beyond the scope of this article is a primer on techniques of coaching, mentoring, and interpersonal communications that we will assume you have.

These are the stories we collected:

Mister159 had found the material easy and had self-appointed himself Helper of the Weak and Stupid (his phrase), a kind of unofficial assistant teacher. When we looked into it a bit more deeply by reading a sample of his posts, we found that he was sharing ready-made solutions to the problems!

The two students with 114 and 126 posts both really loved the subject. They collaborated on the work and were planning to study it in greater depth next year, continuing their collaboration. They were good friends out of school, too; high achievers.

At the other end of the scale, the orientation email had gone into Sheryl’s spam filter and Tania admitted she hadn’t read her email. We did a root-cause analysis on the latter to be sure we had not uncovered a case of cyberbullying. Happily, we hadn’t; she’d just not been focused.

The student who managed just eight posts said he found the subject uninteresting and he was thinking of changing his subjects.

As a result, the design changes the teacher might consider before the next delivery includes:

an optional section with some harder problems for those students who want extending
a once-a-week check on students who have not logged in
grading or peer upvoting of discussion posts that meet certain criteria.

Correlation is not causation, but…

That everyone who buys dried mangoes also buys sparkling water does not mean that the dried mangoes are causing them to be thirsty. From the shop keeper’s perspective, it doesn’t matter why, the fact is — they are. Therefore, he places a new line in sparkling spring water near the dried mangoes.

Between us, my wife and I have several decades experience as Moodle admins, and we have noticed that students who have silly Hotmail addresses very rarely complete. We don’t have causation here, only correlation. But, we do have a correlation, and it does work. Remember that list from the start of this article? Which student automatically goes on the at-risk list?

john.smith@stjosephs.school.nz
mary.roberts@stjosephs.school.nz
lbooth@middlebeach.school.nz
quackers@hotmail.com
garyb@chchpoly.ac.nz

This raises an issue of which we all need to be aware when we start profiling and working with student data to inform our course design and our interventions. Just because we’ve profiled quackers@hotmail.com does not give us the right to treat him or her differently to the rest of the cohort, that would be discrimination. But there is no law or moral code that says we can’t keep a watchful eye.

Designing for data collection

You can build a Moodle course with just two components: label, and forum. But, if you do take this streamlined, and dare I say lazy approach, you’re not going to be in a position to collect rich user data. The awesome power of Moodle is in all the activities it has built-in, and if you want to go down the plug-ins route, then hundreds more again.

Here are just a few ideas for how you can give your students interesting things to do in their learning environment.

Include the:

Choice activity to gather student opinion
Glossary activity to invite student contributions
Wiki activity to enable students to create exhibits
Blog activity so students can learn out loud
Journal activity for students to record their learning journeys.

How much more student data you can now collect to build a picture of their performance on and their engagement with the course.

There are some blocks you can usefully include in the sidebar design too.

Include the:

People block, so everyone can see who is engaging and who is at risk
Online users block to enable a real-time community of learners
Blog block to further develop the community of learners
Activity block to give easy access to everything there is for them to do.

The complete online tutor

I challenge you to change if you have not already done so. To be a great online teacher is not to create a whole mural of content and then stand back while the students admire it. To be a great online teacher is to create an interesting, safe, and supportive learning environment in which they can thrive. By measuring their progress, and encouraging them to make more, you will all have a wonderful experience. By knowing which students are at risk you can reach out to them and try and get them aboard your magic school bus.

LMS or LRS – It takes two to tango

Posted on April 10, 2018 by Stephen Lowe

What should be the vehicle for our online programmes? Do we want a Learning Management System (LMS), or a Learning Record Store (LRS), or both, or neither? Ten years ago, this would have been a simple question to answer. Today there are several layers to it.

Unpacking the LMS

Let’s unpack learning management systems and try to align the needs of the organisation with the needs of the learner.

The organisation’s needs are real:

authentication
access control, and
reporting.

They have to create a walled garden in which their materials are secure, and they need to record the learner’s presence and actions. In certain cases, they need to be able to levy a charge for this access.

The learner’s needs are real, too:

authentication
access, and
recognition of effort.

They are required to learn certain things, the materials they find in the LMS presumably help them to do that. And, they need the results of their activity, and the results of any quizzes or submitted assignments to be recorded.

Moodle, as the most common example of an LMS, meets these needs by providing a user account, enrolment in a course or courses, and a grade book. If the school or organisation doesn’t want to go so far as assigning grades to submitted work, then they can simply track progress and completion if it occurs.

What we’ve just described is the digital version of the old industrial model. It’s got wheels, and it’s got momentum.

Course design patterns

There are some definite design patterns to the courses you see inside learning management systems.

The LMS itself might be seen as a kind of zoological park. The area is ring-fenced, and people have to pay to get in. Once inside, they can go to the enclosures that interest them most and interact with the animals and habitats. This metaphor invites a question about who are the animals and who are the people, and that’s where the metaphor starts to break down.

Courses as habitats, however, is a powerful metaphor and one you may wish to explore in more depth. As an inquiry-based teacher, you may cast yourself in the role of a zoologist: observing behaviours, collecting data, forming and testing your hypotheses.

In the Library pattern, the course designer creates a curated collection of readings and videos that they think the learner should or may want to ingest. Ben Betts, CEO at HT2 Labs calls this “self-directed”. The LMS is just like a filing system now, but with the difference that the course facilitator can see who has read or viewed what.

An extension of the Library pattern is the Book Club. Now, the course designer adds a forum and invites students to discuss what they have seen or read, either with the author or with a recognised expert in the field. Betts calls this “expert guidance”.

Then there’s the Caves Tour pattern. A facilitator is assigned to a cohort, much in the way a guide is assigned to a group of tourists. The facilitator walks the cohort through the collection of materials and activities pointing out things of interest, answering questions, and sharing wonderment.

Then, again, there’s the Campus Map pattern. “You are here”, the map says. Then it provides pathways to all kinds of other places you can go from “here” to learn this or that. Another way of looking at the Campus Map pattern is as a curated collection of internal and external resources.

Pitfalls

Course designers who have not considered these and other patterns, who themselves have little experience as students using online programmes, may create very messy hybrids of these models. A great number of online courses seem to lack any coherent structure at all as they try to be all things to all people.

Often, course designers create an online resource where there is no interaction with a facilitator. They may think that there is money to be made or saved from e-learning; that e-learning can remove the cost of the teacher, instructor, or facilitator. Perhaps they dream of a kind of dollar mine that will churn away unmanned and untended providing a rich vein of income. In two decades of e-learning that has not been my experience.

You have to work at e-learning, ever watchful over your cohort, encouraging, cajoling, and generally massaging them along to completion and a successful outcome. You need to be constantly updating and improving your courses to keep them relevant and current.

Designing courses for success

For a very short course covering just one topic, the Book Club is a good model. For a course that goes into more depth, I think the Caves Tour pattern generally works best. Still, I think the tour wants to be kept short. Up to about four weeks seems to work well; much longer and students start to drop off. With highly motivated groups — say, masters students — they will possibly endure eight weeks. User data we have collected indicates that anything more is simply too long. Of course, I’m making generalisations here; there will be cases that refute my assertion.

Do not expect a good classroom teacher or instructor to necessarily be a naturally good online facilitator. School teachers like to play to a live audience on the stage their classroom presents; they can find the LMS a hard space to command. Trainers within commercial industries like to work nine-to-five, but a good online facilitator will pop into the forums for ten minutes at nine o’clock at night.

The LMS provides a rabbit-proof fence and, like Zealandia, an eco-sanctuary can exist within its boundary. You can charge people to get in if you want, you can control their movements once they are inside, and you can monitor their progress. If that’s what you want to do there is no better tool. But, do not expect everyone to want to come.

Unpacking the LRS

There is another way, so let us now unpack the alternative, the Learning Record Store.

The Learning Record Store is one component of a system that might be called a next-generation learning solution. It is a database of fine-grained actions and experiences by the learner and it is always accompanied by or partnered with an analytics engine of some sort.

Evidence-based

The LRS gathers, analyses, and presents evidence of student activity and experience. It is all about evidence-based learning. Not the self-serving manicured evidence of the type that a student collates in an e-portfolio. Rather, the hard evidence observed by the systems with which a student interacts. This is a particularly good fit with the needs of modern employers.

An LRS gathers evidence about a person’s learning journey

In the future, fine-grained data will exist that tracks the learning journey from primary school, through secondary and tertiary, into employment and promotion. Almost anything can be wired to return a record to the record store. The LRS can listen for messages from an LMS, a blog, social media, an event registration system, a just-about-anything system, or with the software interface of a mechanical device. If you think the 70-20-10 model holds some measure of truth, then you will immediately grasp the potential of the LRS when combined with analytics and visual reporting.

Simply connected

The system uses a protocol and language called xAPI to send a simple sentence back to the store.

So, lines in the store might read:

John completed Automation in fish factories 101
John read article Greenlip mussel industry
John watched video Pristine waters
John published blogpost Richmond Bay nutrient levels

And, xAPI works with the Internet of Things:

John Smith registered for symposium Aquaculture 2020
John Smith operated a Simms auto-grader at Aquaculture 2020

xAPI’s syntax is both simple and powerful:

subject — verb — object — context

The context element enables a richer picture of the development path:

John completed Automation in fish factories 101 in 8 days with an overall grade of 93%.
John published blogpost Richmond Bay nutrient levels which was upvoted 57 times.

LRS and Machine Learning

If you want to geek out for a moment, consider the power of machine learning applied to the large datasets that an LRS will accrue. These are datasets pertaining to the individual, to study groups, to cohorts, and to wider student populations. An LRS could be owned and operated by an individual teacher as a personal instrument for analysing their own performance and that of their students, by an organisation, by a nation, or by a world organsiation. Unsupervised learning algorithms can uncover hidden patterns of behaviour in populations of learners that provide actionable insights for marketing, sales, programme designers, course designers, and teachers. For example, in the school sector, the predictive abilities of machine learning could inform curriculum development and guide education policy makers.

Conclusion

So, do we want an LMS, or an LRS, or both, or neither?

The answer is probably both.

If a school was starting from scratch, I’d say:

set up Moodle
build courses using a definable pattern on which the various departments are agreed, and
consider the LMS’s role as an activity provider to an LRS from the very start.

Moodle supports xAPI. You don’t have to implement that straight away — walk before you try to run — but don’t omit it from the design. Moodle 3.4 has some much better built-in analytics and reporting tools than ever it did in the past; that may be a good starting place.

CORE Education’s interest in xAPI combined with analytics and visual reporting lies in the area of professional development. Where xAPI transforms the old SCORM standard from which it was spawned is in its ability to not only track learning, but to link that to job performance. That creates a closed loop that is the quest of every learning designer. Not a closure created by an assertion or a presentation, but by factual evidence.

xAPI disrupts, but not to such an extent that it challenges the very existence of the LMS. The LMS continues to serve a useful purpose, connected to the learning network in its role as a curated activity provider.

Stephen Lowe

Harder

Everyone likes to quiz

Demand submissions to unlock the levels

Instant feedback

Faster

Far-off deadlines kill online learning

Treasure hunts are fun

Nano-learning builds skills

Level Up

Mind your language

Nudge, tempt, shove

Provide real time support

It’s a wrap

Suggested resources

Before we go any further, we need to do some disambiguation.

In the future everyone will not be a computer programmer. So what is the point of students learning coding at school?

Isn’t Computational Thinking just a buzz word for mental arithmetic?

Firm foundations

Objects to think with

So what are these transitional objects called sprites and where do you get them?

What are the powerful ideas, the higher and more abstract concepts?

The challenges we face

Suggested resources

Background

Taking a design approach

Let’s get empathising

Time to start defining

Ready to ideate

Get cardboard and glue, prototype!

Let’s try it out on Toni

What does excellent user support look like?

Baked into the course design

Proactive and unobtrusive

Caring and supportive

Non-judgemental

Multi-modal

Conclusion

Resources

See other posts in this series

Background

Identifying students at risk

Exploratory data analysis

John Tukey

Brinkerhoff SCM

Correlation is not causation, but…

Designing for data collection

The complete online tutor

Further reading

Unpacking the LMS

Course design patterns

Pitfalls

Designing courses for success

Unpacking the LRS

Evidence-based

Simply connected

LRS and Machine Learning

Conclusion

Suggested further reading