Black Swan Society

Regular readers may remember the trauma we experienced with the maths department in respect of our elder son. At the time that this was going on, I was venting to a friend (let's call her Rachel) with kids at the same school. She could relate to my situation, because her own experience of the maths department with one of her older sons (let's call him Tim) was not a happy one.

Tim, Rachel tells me, is one of those people who takes a little longer to grasp a new concept. But, once he's got it, he's got it good. Having done well at GCSE maths, he had opted to take the subject at A level. But things were not going well, and he was becoming steadily more discouraged. At the first progress meeting, Rachel was advised that Tim should give up maths because he was simply never going to get it.

This pushed Rachel's 'I'll show you' button. When it comes to her kids, the mild-mannered Rachel gives me a run for my money in the tigress stakes!

She found her son a private tutor and enlisted the support of the same Ms Verity I have mentioned before. Tim passed A level maths strongly and went on to study it at university.

He has just taken up his first post as a maths teacher... at the very same school he attended himself.

This means that Tim and the teacher who wrote him off will be colleagues... and this is not the first story of this kind I have heard.

Teachers please take care. I have personal experience that crow and humble pie don't taste very good and egg makes an uncomfortable facial accessory.

The same teacher who was so discouraging of Tim, recently advised my younger son - during 6th form induction - not to sign up for maths A levels. This after an impassioned speech to the whole group about how it was the most important subject in the curriculum and an absolute must, yadda yadda. What changed her tune was that she learned that my son and one other child present had been in set 5 of 9 for GCSE maths, and she felt that they would 'struggle'. She had apparently already advised all the kids in set 6 and below against attending the talk on maths at all.

The daft thing is that, 5th set or not, he sat exactly the same exam as the kids in the top set... and he got a solid B. What's wrong with that, I ask you?

He returned from the induction full of doubt, and started casting about for an alternative subject. I was livid, nay, apoplectic. Those of you who know me well can just imagine how this played out.

This is a kid who, since the age of 3, has wanted to be an explosives demolitionist. Never once has he wavered from this goal. If he is to realise this ambition, he will need to study structural engineering. One of the entrance requirements is a solid pass in maths at A level. He knows how hard he is going to have to work, and is prepared for the challenge.

As my husband said, "So, he may struggle. So what? Is there a rule against struggling somewhere? If he struggles, we'll find him some extra help. That's what parents do."

We get the impression that the department - or at least, this teacher - is only interested in kids who are going to sail through and score the sort of results that will keep the school in its lofty position on the local league tables. My mother in law has a sticker on her fridge that says, "The forest would be a quiet place if no birds sang but the best." Our kids may not be among the best, but, by 'eck, they're going to get to sing! To put a (slightly) more polite spin on a South Africanism, that woman does not get to urinate on our son's campfire! It may be that he changes his mind about what his career choice (our elder son did exactly that, shortly after starting 6th form) but it is to be his choice to make! It is not to be foisted upon him by someone with an agenda.

So, tomorrow, he goes into school to sign up for his A level subjects, and maths (with mechanics) will be one of them. He may well struggle, but (as the great Gretzky says) you miss 100% of the shots you don't take.

Not all parents are as determined, as resource-rich, as stubborn, as supportive, as teeth-clenchingly bloody-minded as my friend Rachel and me. My heart sinks as I wonder how many young people are giving up on their life's dreams because of what a teacher has said.

Let's be enablers, folks.

I came across this today, courtesy of a link from one of my Twitter friends. I forget which.

For those who use Articulate and like to try to find ways to push the boundaries of what it can and can't do.

I have tried umpteen times to embed the video in this post, but, while I can see it perfectly in preview, it simply refuses to play ball when I publish. Sorry.

Like that will work…not!  Seriously, there are several things that have to line up to get social media working for you (which, if you’ve been paying attention, is the new and only sustainable competitive advantage).  As I discussed a couple of days ago, your learners have to have the skills.  But there are a couple [...]

Those of you who are Facebook friends will know that I lost something during my recent holiday. It was something of no intrinsic value, and yet I treasured it enormously. It was something that has been featured on this blog before. It was the bangle my father gave to me for my 17th birthday.


It was the only truly personalised thing he ever gave me, and the only times I wasn't wearing it were the times it was broken. It occupied pride of place on my right wrist through thick and thin for about 30 years (albeit with the aid of several repair jobs). If you have met me in person, you have seen it, even if you haven't noticed it.

It has been swimming, climbing and jetskiing... and survived it all.

But one day during our holiday, my elder son playfully grabbed me by my wrists in the sea. As I felt it move on my arm, I yelled, "My bangle! My bangle!" It took my son a moment to understand the import of what I was saying, by which time the bangle had fallen off. We could see it clearly through the water, lying at my younger son's feet. In distress, I yelled at him to pick it up for me, but he couldn't see it, and - as I watched in horror - accidentally stepped on it, burying it in the sand.

Although we borrowed goggles from kindly people nearby and spent the next 40 minutes or so searching for it, we finally had to accept that it was gone.

I am unashamed to say that I wept huge, wracking sobs for the loss of it. I felt hollow. I even dreamt that night that it was returned to me. I still keep absent-mindedly trying to adjust it on my arm, and there is a faint tan line where it used to be.

Someday, maybe, someone else will find it: a badly made, shoddily repaired silver bangle cut in the shape of an unusual name not their own.... and it will have absolutely no value to them.

Will they even be able to tell that it had once had enormous value to someone else? Will they know as they hold it that there is a woman somewhere out there whose delight would know no bounds if they were to find a way to return it to her?

So it is with learning. Sometimes we share things in this space that have inspired us, or from which we have gleaned enormous value. Sometimes we wax lyrical about something we have found or made or seen... to a round of utter indifference.

And what of it? Does that diminish the value to the beholder? I certainly hope not. There is space for a wide range of value systems and measures.

As learning providers, we need to be careful not to denigrate certain resources simply because they hold no value for us. One man's meat, as the saying goes...

On our weekly twitter learning fest, #lrnchat, I regularly identify myself as a learning experience design strategist.  I don’t always assume people know what that means, but for that audience I figure they can infer what it means.  However, I think the idea is worth exploring, because increasingly I think that not only is that [...]

The Social Media for Trainers blog book tour begins this Thursday, September 2, with a kickoff post from the Centre for Learning and Performance Technologies' Jane Hart. This will be followed by a constellation of blogging stars from the training and eLearning fields.

Is the book right for you? The publisher asked specifically for activities and ideas to help trainers and instructional designers develop an understanding of social media tools at "eye level": What are they, how are they best used, and how can we use them to extend and enhance current practice?  The book is available from booksellers in North America now, with UK and EU releases due in the next few weeks. Check out the "look inside" feature on Amazon.com to get a peek.

Take a look at the blog book tour schedule  and watch for the posts from my colleagues. Many thanks to them for their help with this project!

More? Follow "Social Media for Trainers" on the book's Facebook page and on on Twitter @SoMe4Trainers  (use #SoMe4Trainers).

Just a quick post to thank everyone who attended our first official Tandem Learning Innovation Community event. We hosted the event on our new Second Life island and I'd encourage all of you who were unable to make it to stop by any time and take a tour. The island is open and public; we'd be happy to share the space with any member of the TLIC who'd like to run a meeting in SL. Feel free to contact me if you'd like a guided tour or assistance in how to work our browser displays.

In all the excitement about empowering learners by providing rich information and social environments, it’s too easy to think that “if you build it, they will learn”.  Yet the evidence is to the contrary. While there are numerous components, including a culture that tolerates diversity and doesn’t punish honest mistakes, one that is easy to [...]

In reflecting on the presentation I gave earlier this week, I realize that I didn’t make it clear that just making it social will make activities lead to better processing.  Of course, my goal was evangelizing, but I reckon I should followup with some clarity.  There are some design principles involved. First, the assignment itself [...]

Another Twitter conversation, another blog post...another thought-stream based out of ADL's Implementation Fest #ifest (Srsly, this is the most a conference has inspired me to write in a long time. A good, and bad, thing...).

So, let's talk about the opportunities and problems with using games for assessment. Especially when you throw the "s" word in there..."standardized" assessment.

Deep breath.

First, I'll preface by saying that games are a natural environment for assessment...in essence, they are assessing your performance just by nature of the game structure itself. Unless, of course, there aren't clear success metrics and you "win" by collecting more and more meaningless stuff (like Farmville)...but that's a whole other topic. So let's assume there are success metrics built into the game and those metrics align with what your learning objectives are. Its logical that by having someone play a game, you'll see how well they know something or know how to do something. Right?

Nothing is ever that easy. There are lots of aspects of game play that depend greatly on how the game was designed. For one, games have an intrinsic layer of cognitive overhead that may not exist in real life. For example, as I've been learning how to play Call of Duty 4, I first have to master the use of my PS3 controller. No, this isn't a learning game, but the same principles apply...it's why real guitar players get irritated playing Guitar Hero...there are skills that you need to develop to play a game, or to be successful in a game, that don't exist in real life or don't mirror the skills necessary to be successful at real life tasks. I think it becomes clear in first person shooter games, where your ability to operate your game controller does not directly translate to being able to accurately fire an automatic weapon in a combat environment. For any assessment, you have to make sure you're not just assessing how well someone plays the game, but how well they have mastered the real skill or content. In using games for assessment, you run the risk of assessing how well someone plays the game, not the objectives you are hoping to assess.

Another issue with games for assessment is the gender differences in how people play games. I'm about to talk about broad generalizations, so bear with me and recognize that some women game like "guys" and some guys game like "girls"...but there are different ways that people approach game environments and those differences do tend to follow along gender lines. Men are bigger risk-takers and explorers; women like to be guided, understand the environment, and follow the rules. Depending on how you design your game, you risk alienating a whole group of players if you don't consider the gender differences in game play. Worse, if you are using games for standardized assessment, you could be putting about half of the people you are assessing at a disadvantage just by the nature of the game design. Given the general acknowledgment that standardized tests are racially and class biased, adding a layer of gender bias in the game design risks making the concept of "standardized" even more meaningless.

Do I think games can be used effectively for assessment? Yes. Look at surgical simulations, flight simulators...close approximations of performing tasks in real life. Research has proven that successful performance in these simulated environments correlates to successful performance at the actual tasks. Where you can mirror game performance to real performance in this way, I think games are a brilliant and useful measure of assessment. But without careful design, thoughtful reflection on what the game environment adds to assessment, and what the trade-offs are with other forms of assessment, we risk creating another assessment environment that falls short of measuring true capability, potential, or performance.

Coming out of the #ifest Twitter stream, I once again heard how America's Army was a shining example of how games had been used to improve recruitment efforts. I posed the question...has it improved recruitment of women at the same rate it has improved the recruitment of men? So far, all I've heard back is *crickets*.

For two weeks, I have been looking for data or research on America's Army that mentions gender as a research parameter, but so far, I've found nothing. If you know of any research, I'd love to see it. My hypothesis? Recruitment of women was not as greatly improved after they played America's Army. If that's the case, what does that say about the relative value of recruiting women vs men into our military?

What makes a game successful? Is it ok for public institutions (government, schools, etc.) to measure the success of a serious game without looking at differences in outcomes along the most basic parameters (gender, class, race)? Is it ok to say a game is successful in achieving its goals if we don't consider those issues as part of the discussion?

I'm tired of hearing the marketing spin and the hype around how games can change the world if we're not even asking the most basic questions about WHO games are changing and HOW they are changing them. You won't find a bigger advocate of games for learning and as a vehicle to raise awareness and support behavior change. But not all games are created equal. We have to be vigilant and constantly questioning our design to ensure we're achieving the outcomes we seek. Ignoring questions of gender, class, and racial bias in serious game design makes me question the motives of the design itself and the motives of those promoting a game's "success."

As always, I welcome anyone's comments who can prove me wrong...

Twitter not quite right for your organization? This came up in #lrnchat last week, and in a Twitter discussion yesterday. Here are tips mostly from Aaron Silvers (Twitter: @mrch0mp3rs) on using microblogging in the enterprise:

-Remember, the practice is more important than the tool. This gives flexibility to change tools later on.

-Having said that: Choose the right tool in the first place.

-Make sure someone is a registered admin. Don't do this with no one in charge.

-If you're using a free account, do your org a favor and link to digital files in these microsharing tools instead of uploading into them.

-There ARE reasons why email works. Use the right tool for the task.

-You want leaders to contribute consistently -- even if it's just once a day, a reply to an employee.

- Write up the "rules" or expectations for your boss person to distribute. Fear is often not knowing what to say.

-Give examples of the kinds of things to use it for to get people acclimated/started.

-With any new communications medium, patience and consistency are keys to adoption. Modeling how to use is important.

-Start w/ a core group, and make sure at least one big manager is involved and posting daily.

And from @ldennison: if you're bringing it into the organization, you're the person responsible for it.

See also: Comparison of Microblogging Tools

In a conversation with my ITA colleagues (we keep a Skype channel open and conversations emerge daily), we revisited the idea that there’s a higher perspective that needs to be highlighted: social media is a business engine, both internally and externally!  Jane Hart’s been helping clients with social media marketing, and this has been an [...]

Yesterday, I talked about how social added to formal learning.  Today I want to extend social learning learning to informal learning.  When I talked about the value social adds to formal, it was about processing the information in richer ways, to help facilitate learning.  However, the value proposition for informal learning is different. We can [...]

We are pleased to announce the first, official Tandem Learning Innovation Community event scheduled for Friday, August 27th, 2010 at 9:00 am SLT/ 12:00 pm EST. We’ll be hosting an open house on our new Second Life island. The official networking event, led by Koreen Olbrish (SL: Nina Sommerfleck), will be an informal discussion of community topics, including:

• New technologies for the TLIC to explore
• Major challenges in technology adoption for the community to address
• TLIC at DevLearn 2010 – call for interested parties to be showcased

Please let us know if you plan to attend by emailing Jedd Gold via linkedin or at jedd.gold@tandem-learning.com. We will be sending out the SLURL the day before the event to everyone who RSVP’s.

If you haven't yet joined the Tandem Learning Innovation Community, you can request to join here.

We are looking forward to seeing you there!

In thinking about the benefits of adding social to formal learning for a presentation I’ll be giving, I realized that the main reason to extend social activities into formal learning is for the additional processing it provides.  While there is processing individually, social interaction provides more opportunities. The types of learning processing that matter for [...]

When you look at the changes going on in society, and the implications for business, you realize that there are some significant changes going on.  This isn’t news: things are moving faster, we’re having less resources available, our competition is more agile, the amount of relevant information is increasing, customers are more aware, the list [...]

Even though I'm attending through the Twitter stream, ADL's Implementation Fest #ifest is getting me fired up about some learning technology industry issues that just can't be explored in 140 characters. For example, yesterday there was some lively conversation around the usefulness of learning tools.

And I, in a rash statement, said that most learning tools suck.

But let me clarify, because there can be a broad definition of what a learning tool is.

For me, a learning tool is not what I use to design learning experiences (those things might include pen and paper, whiteboard, PowerPoint, Visio, etc.). A learning tool is NOT a reference tool like Wikipedia. Wikipedia is an information portal where you can go, read, and maybe learn something new...but it was not designed as a learning experience. It does not facilitate learning, even though it can enable it. Can you learn from a reference tool? Sure! But good reference tools have good user experience design, not instructional design, making it a reference tool and not a learning tool. There IS a difference.

A learning tool, to me, is something that you use to develop a learning experience. In other words, a tool that allows you to "design" a learning experience and output it into "Voila!" a learning experience. Input = content, output = training. And here's why I think most learning tools suck.

Most tools limit what you can design intrinsically in their functionality. Let's take PowerPoint. What you're going to get is slides. Pretty didactic. Maybe a little video embedded, some nifty animations...but you're not going to get much in the way of learner interaction.

But now I'm going to ask you a question...have you ever learned in a workshop that was guided by a PowerPoint slide? Have you ever been in a learning environment where PowerPoint was the primary learning tool, but the content, activity, discussion actually taught you something? I'm going to guess yes. Maybe you've even been lucky enough to be in a session guided by PowerPoint that made you do something differently when you left. You know what that is? GOOD DESIGN. It's not the tool. Its how you design learning experiences that facilitates learning, not the tool that you use.

So what makes a learning tool "good"? Openness. Flexibility. Interoperability with other learning tools and reference tools.

What makes a tool bad? One that dictates design. I could list some specific examples, but I'm betting you know what they are. Online learning development tools would be a great place to start.

One of my favorite quotes from yesterday's Twitter discussion was from John Campbell @jpcampbell :

what's ur expected output from tools? Learning Content? Why ask the architect to output a house?

Which is my point exactly. Instructional design and learning technology development are two different skill sets. Instructional designers are the architects and technology developers are the builders. You shouldn't build a house without an architectural plan, nor should you expect your architect to go ahead and put hammer to nail to bring his plan to life. There's an essential relationship here that too many organizations neglect to recognize, instead hiring IDs to build their training content using some rapid development tool. Most organizations are guilty of this in someway..."Put together a PowerPoint - led workshop!" "Import our workshop content into a virtual classroom!" "Create an Articulate module!" "Make video clips accessible from a smart phone!" This isn't a fault of the tools, its a lack of awareness of the importance of design. As an industry, we should NOT be designing learning experiences dictated by what tool you have (if you're a hammer, everything looks like a nail) but by the appropriate format to support the goal, supported by the appropriate design for that format. Instructor-led, game-based, online, mobile, print...they are ALL good formats when they are appropriate for the content, designed appropriately, and appropriate tools are used to develop them.

And that's why I think most learning tools suck...because they neglect to recognize the difference between design and development, and the default tends to be development at the expense of design.

[Note: This originally ran on Training Magazine’s former “Training Day” blog on 2/12/2010]

Discussion of objectives in training could be a topic for a book all by itself, but lately I’ve run across 2 excellent examples of problems with learning/performance objectives. They provide a good basis for looking at just a couple of common problems.

Example 1: One summer afternoon my friend Jo left her son, 5-year-old Max, in the care of his grandmother. While Max was napping Grandma found a dead rattlesnake in the yard and thought to herself, “This is a good time to teach Max about snakes.”

Her objective: “Max will understand about snakes.”

So when Max awoke from his nap Grandma took him outside and said:
“See, Max, this is a rattlesnake. Some snakes are very dangerous so you must be careful if you are ever near one. They can be hard to see.” Using a hoe, grandma moved the snake into high grass, then onto a bed of pine straw, to show Max how the snake’s colors tended to blend with the setting. Grandma talked about being careful when running around outside barefoot, not bothering or teasing snakes, and taking care when playing near places snakes might be found, like fallen logs or warm rocks.

At the end of Grandma’s lesson she said, “So, Max, do you understand about snakes?”

And Max looked up at her and said,

“Oh, yes, Grandma. I love snakes.”

In the example with Grandma and Max, the problem was an objective too vague: “He will understand “ can be interpreted in more than one way, which is exactly what happened, and Max did not understand in the way Grandma meant him to. This is a common problem in compliance and policy training, where it’s more usual than not to see objectives like, “Learner will know the policy”, “Learner will understand the rules regarding unlawful harassment”. And regarding Grandma, well, as we say here in the American South, bless her heart. She did intend to help Max “understand” (learning) but she didn't specify actual performance. She tried to make the snake training meaningful and engaging. She did not read PowerPoint slides to Max. She included important information (they are hard to see in the ground cover) and offered some helpful tips (don’t tease). But the training did not accomplish what she’d intended.

I’ve seen the opposite problem as well: Objectives (and performance this time, not just "learning") so detailed and specific that the real point of the thing is lost. Which brings us to Example 2: A contractor charged with developing online tutorials on the new employee timekeeping system listed the desired performance objectives (below).

At the end of the training, the employee will be able to:

• Log on and navigate to the employee section of the portal

• Record and review time

• View time statements

• Display leave quota overview

• Generate leave requests

• Access system help resources

• Assign charge object numbers

• Report premium pay hours

The objectives were certainly detailed and specific. The contractor had thoroughly delineated desired performance. After weeks of tedious wordsmithing, next-level management finally signed off on the objectives. Senior management likewise approved of the plan. Everyone involved agreed that, yes, these are the outcomes we’re after.

Several million dollars later the training was launched, and several weeks after that the new time sheet software “went live” to 30,000 workers. And the critical problem with the tutorials quickly, and loudly, and in a most dramatic way, became evident. The list of objectives had not included:

At the end of this training, the employee will be able to
complete his or her time sheet.

[This is not to oversimplify the other problems here, including the evidence that no one ever thought to ask even one potential learner to try the material out, or that much of the training content, like charge object hours, was relevant only to a fraction of the target audience.]

So: Before developing the instruction don’t just write objectives. Write the right objectives. What is this person really supposed to do back on the job? What does “understand” mean, and what evidence will show you that understanding has occurred? Devotees of Bloom’s taxonomy will argue that learner performance like “listing” and “describing” can constitute what he called ”enabling” objectives. That may valid, but they should not be the only objectives: Employees are rarely asked to “list” or “describe” anything, so it’s critical to move on to behaviors closer to desired performance, not just knowledge. And: Enabling objectives are easy to write, and to develop bullet points for, and to develop training around, and to write a quiz to assess. If you feel the training really must address these, fine, but be sure to push past them on to things that more closely resemble real performance. In my train-the-trainer course I don’t want my learners to describe strategies for engaging learners, I want them to deliver a piece of instruction in which they demonstrate the ability to apply those strategies. It’s more work for both learner and me, and much more time consuming, but it moves us far closer to the actual desired performance. And it makes the training worth doing.

Think Goldilocks. Not too little, not too much. And remember in developing objectives to keep an eye on the rock-bottom performance goal: Don’t get eaten by bears.

Other problems with training objectives? I asked Twitter training/elearning/ID folks and here are some of their answers. Perhaps we’ll expand on some of these in a future column.

• Gina Minks @gminks, EMC: “When objectives relate to what someone wishes the performance was, even though that may be a fantasy.”
• Jeffery Goldman @minutebio, Johns Hopkins Healthcare LLC: “Not setting them at all, not measuring whether they are met in the final assessment, and not providing content to meet objectives.”
• Guy Wallace @guywwallace, EPPIC, Inc: “Objectives are not systematically ‘derived’ from solid analysis of ideal performance/gaps & are best guesses.”
• Kevin Bruny @row4it, Chesterfield County VA Government: “Once used for design and communicated in training, we tend to forget about them and never return to validate.”
• Kara DeFrias @californiakara, Intuit: “People get so wrapped up in objectives they forget to take time to make the actual learning meaningful & engaging.”

--JB

I've been following the ADL Implementation Fest #ifest stream on Twitter today and some of the conversation with my PLN has sparked some thoughts, maybe perspective, on how, or where, I see the government being able to lead the way in training. And, what prompted me to write this post, the ways in which its misdirecting its energies.

First, let me say, there are some great examples of people in government doing things the right way. Just from my immediate experience, Dr. Alicia Sanchez, who works for DAU, is the games czar who is helping integrate gaming into their curriculum. Mark Oehlert, also at DAU, is integrating social media technologies to support learning and knowledge management. Judy Brown at ADL is an industry recognized expert in mobile technologies and how they can be leveraged for learning. (Just realized, ironically, that these three will also be showcasing their knowledge at DevLearn 2010. You should go.) These three people, who happen to be people I know and respect, understand the unique positions they hold, and their opportunity to leverage technology for innovative applications. In short, they recognize that they have the chance to DESIGN really cool applications of existing technologies within the government and talk about how these projects are helping to improve learning, collaboration, and communication.

What I'm hearing out of iFest (so far...its the first day...;) is that the focus is still really on what technology can do for you and what technology initiatives ADL has been focusing on. To which I say...REALLY?!?! Sigh.

I don't need or want government agencies to fancy themselves technology companies. They aren't a start up, nor are they Microsoft. In short, there are companies who actually do that. And those companies need to make money doing it, which means that they need to build things that the market needs (even if the market doesn't want it...that's a totally different thing...).

What I'd love to see is that agencies within the government start looking at what REALLY helps support learning...good design. I'd love if they saw themselves as master implementers, not builders. Our government has tons of people that need training, its got tons of money and resources...why not leverage it for those things? Try innovative solutions. Experiment with design. Conduct research to establish best practices. That's what the learning technology industry needs. The government could provide this...it could LEAD this. But for the most part, its not.

If a technology is needed, the market will push it because that's what the market looks for: meeting unmet needs to make money. I'm tired of hearing about how a technology the government is developing is going to solve some problem.  Let's face it, even Google has struggled with implementing innovative technologies (see: Wave, Lively) and that's their business...its what they do to make money...and they are arguably the best at it.

I'm hoping that as I hear more at iFest that its focused on design. Fingers crossed. If not, its an opportunity lost...