Elnet News Blog
The latest news on the Elnet Network
Sophie Carter is and Learning and Development Consultant at Challenger. She has worked with staff and management to create and embed change through developing learning strategies including systems development, team development, instructional design, facilitation and enhanced measurement and accountability for learning within the business.
We caught up with Sophie ahead of her presentation at the upcoming ElNet Sydney congress, to talk elearning and put 5 questions to her…
1. What will be the biggest challenge for online/elearning practitioners in Australasian organisations in the next 2 years?
Separating out the hype from the useful. The pace of development and change in this area is super-fast – but identifying what’s going to be genuinely useful, and spending time looking at the ‘how’ of making new tools, methods, toys valuable to a learning experience will be key. On top of identifying the what and how, being more flexible with learning generally will be a big challenge I think we’ll get to see more of in the next couple of years.
2. A common oversight in elearning project management is...
Thinking it’s just like any other project! ‘Managing’ a learning initiative is unlike any other project a business can get involved in. Being realistic about that from the beginning is definitely important.
3. Can flick and click elearning be effective?
Only when the learner knows exactly what they want to get out of that particular activity, and why. Quick, learner-oriented pieces are effective when the learner decides they’re useful.
4. What are your top 2 Must read/subscribe elearning resources?
ASTD is great. I also listen to the Design Matters podcast with Debbie Millman. It’s not directly e-Learning related, but I always find it useful in keeping me thinking outside the box, and equal parts focused on process and output.
5. If you were given $1million dollars to set up any kind of online learning resource/project what would it be?
Something open-access and peer based that provides unobtrusive learning opportunities in the workplace. Without the clunky design of imposed learning structures on unsuspecting innocents!
Sophie Carter will be presenting at the upcoming ElNet conference on: “Free is Good, Freedom is even Better”… Rethinking project management and learning design to involve managers, employees and SMEs in the end-to-end learning experience. E-learning projects are no longer the domain of the learning expert, and can actually be enhanced by increasing the level of involvement other stakeholders have. We’ll explore how to shepherd their experience.
Register for the Sydney congress on the 14th of June here http://www.elnet.com.au/index.php/component/dtregister/?Itemid=0&eventId=35&controller=event&task=individualRegisterContinue reading
eLearning in medical education is evolving, as medical specialists conservatively engage with current educational technology. However significant improvement to medical education could be achieved with greater use of technology.
While politicians celebrate the opening of new medical schools and new doctors graduating in Australia, six medical programs were struggling to find heads of medical education in 2012. Hu et al. (2012) ask“Where is the next generation of medical educators?” anddetail reasons for the shortage of medical educators including the fact that the medical educator’s path for doctors is not lucrative or prestigious.
In the world of medical educators, issues include: a shortage of medical educators in Australia; the isolation of some medical educators; the lack of a prominent Australasian medical educators association; an absence of compulsory qualifications for medical educators and a lack of a defined career path for medical educators. Hu et al. (2012) recommends creating a specialty: a Medical Education Specialist with compulsory qualifications. I advocate that this includes training in elearning.
Medical educators need to provide quality educational outcomes. The Australian Medical Council (and others) provide guidelines on accreditation of medical schools including standards for fellowship training, assessment and continuing education.
Medical educators have a wide variety in backgrounds, teaching experience and qualifications. There are medical educators who are doctors and those who are academics or educators. Neither group of medical educators have a recognised ‘medical educators’ qualification- partly due to the lack of specific courses available. Huwendiek S, et al. (2010) report on the results of an international web survey on the backgrounds of medical educators: “68% come from medicine and 12% from education, basic science, psychology and other health professions.” Those with formal qualifications include 16% with a Master’s degree and 7% a PhD.
A structured training path with a recommended qualification for medical educators should include elearning experience and education, especially considering the medical elearning opportunities in Australasia. In specialist medical colleges, many educators without medical degrees are the quiet providers of training and assessment. These educators support clinicians (as subject matter experts) to deliver medical training with educational technology.
To assist in resolving some issues in medical education in Australia, medical schools should look to elearning, for improved access to medical education, to deliver education to greater numbers of doctors and to assist delivery of rural and remote medical training. Additionally medical educators should connect in a community of practice to build a presence of Australasian medical elearning educators by forming an online Medical Education eLearning Specialist association.
Existing associations of medical educators are:
· AMEE the Association for Medical Education in Europe (AMEE) http://www.amee.org/
· Academy of Medical Educators http://www.medicaleducators.org/ in the UK
· FRAME Federation of Rural Australian Medical Educators www.ausframe.org
· Medical eLearning Educators Network (MeEN), supported by government funding for the specialist medical colleges only.
Confidentiality and an aging workforce in some medical specialties appear to be a barrier to uptake of elearning in medical colleges. Learning Management Systems (LMS) and webinars are in use in most specialist medical colleges, with webinars proving very popular as doctors don’t have to travel. The technology requirements are low and accessible to most doctors where they have internet access. Hospital firewalls can prove a hurdle as can remote internet locations. For some rural medical colleges, teleconferences and emailed PowerPoint presentations provide a reliable, fail safe training delivery method. Doctors working 60 hours+ a week don’t appreciate spending 10 minutes with technology problems at the start of tutorial, every week.
As learners, doctors appreciate intuitive applications, with minimal software installation, one platform with one password and little time required to learn the software. ePortfolios have been embraced by some medical colleges but the time required to learn Mahara or Pebblepad is a deterrent. Some larger colleges have moved to online exams requiring complex organisation. Online facilitation of discussion forums, wikis or other innovative methods is frequently dependant on and limited by, busy senior Fellows of the colleges generously donating their time as medical educators and subject matter experts.
As medical educators build elearning expertise, the use of innovative technology such as Point of View Cameras will improve training access, availability, effectiveness and delivery. Development of an Australasian association of specialist medical educators with ICT could provide formal and informal learning opportunities, professional support, peer support, a defined qualification, a career pathway, and a clearer identity for medical educators in the 21st Century.
Hu, W., C., Y., McColl, G., J., Thistlethwaite, J., E., Schuwirth L., W., T., and Wilkinson, T. (2013). Where is the next generation of medical educators?Med J Aust, 198 (1): 8-9. doi:10.5694/mja12.11654 https://www.mja.com.au/journal/2013/198/1/where-next-generation-medical-educators
Huwendiek S, Mennin S, Dern P, et al. (2010). Expertise, needs and challenges of medical educators: Results of an international web survey. Medical Teacher, 32:11, 912-918. http://informahealthcare.com/doi/abs/10.3109/0142159X.2010.497822
The theme for our upcoming Sydney congress is "Innovative eLearning Project management" and we assembled some of this topics key leaders from around Australia to present their thoughts on the design, delivey and implementation phases of successful eLearning project management.
We caught up with Alison ahead of the ElNet Congress in Sydney to learn a few lessons of our own.
Alison Bickford is the Director and Principal Consultant for Connect Thinking and a local Sydney elearning Guru!
1.What will be the biggest challenge for online/elearning practitioners in Australasian organisations in the next 2 years?
Having the time to immerse and play in the evolving learning technology space, and plan for the significant changes occurring in connectivity, mobility and personalisation.
2. Flick and click elearning can be effective - true or false + why?
True, if it’s part of a greater content strategy of short and concise briefings or ‘how to’ topics. Video is great for this.
3. If I knew 10 years ago, what I know now about elearning, I would have..
... made more ‘just-in-time’ performance support videos (assuming I had a searchable platform that would enable people to access videos easily).
4. Top 2 Must read/subscribe resources?
- Site Visibility Internet Marketing podcast – the principles of developing and disseminating content as an internet marketing strategy holds true for any internal L&D content strategy.
- ASTD magazine (American Society of Training and Development) – most of the articles are well written and remind me that the world of L&D is an amazing place.
5. If you were given $1million dollars to set up any kind of online learning resource/project what would it be?
A YouTube-like enterprise video platform with mobile access on any device.
You can hear more from Alison May 30th at the ElNet Sydney Congress when she speaks on the topic of “Horses for courses. Defining project purposes and pedagogy to fit “. Register here…Continue reading
The theme for the upcoming ElNet Workplace Sydney Congress (Thursday 30 May 2013) is Innovative eLearning Project Management. Naturally, this has reminded me of one particular episode of The Simpsons (Season 2, Episode 15: "Oh Brother, Where Art Thou?")
To refresh your memory: In this episode, "The Homer" (aka: "The Car Built for Homer") was a car designed by Homer Simpson when his half-brother, Herb, gave Homer a job at Powell Motors.
Homer took charge of the project after Herb encouraged him to follow his gut when it came to designing the car that American consumers would want to buy.
The result of this project is shown below, a strange looking car with lots of whistles and bells, that cost so much to develop, and was so expensive to buy, that Herb’s car company went out of business shortly after.
Now the segue to eLearning…
When designing eLearning courses some of your clients may be Homers. They may have a wish list of what they want in their course, and it may end up looking like a Homer.
The scary part is there is a little Homer hiding in every client. They may have strong opinions and insist on the look and features of their course. When more than one client is involved in a large project they may all insist on the look and features of their sections only, without considering the project as a whole, or the application of learning theory, or the needs of their end users .
So how to avoid Homer designing your course?
The linchpin between yourself and your client is your Project Plan. This document includes the who, what, when, where and why of your project – and importantly what your project will look like. It's a formal document that is agreed upon prior to you even turning on your computer.
It may take a few meetings to work out your Project Plan, but this time is well spent and is something to refer to and start waving when Homer starts wanting to add extra whistles and bells.
The project plans that I use include the following 8 sections:
1. Project Overview - Details of the requirements of the project
2. Project Sponsor - Details of the department/section/business contact that you are making the project for
3. Project Team - Project Coordinator (yourself) and SME contact details
4. Project Specifications - Timeframes for each project phase (e.g. Design, Development, Testing and Publication)
5. Project Design - Different designers will have different ways to show the design of their course: flowcharts, storyboarding, outlines etc. This may vary from course to course and sometimes I use a hybrid to show different areas of a course
6. Reporting Requirements - Details such as:
- Who will receive project reports?
- How often will these reports be provided?
- Who will attend update meetings?
- When will these meetings occur?
7. Project Administration - Details such as recording the project on a corporate database, and creating folders to store your project artefacts on network drives
8. Approval - Project plan approval from relevant stakeholders
The moral of the story:
The more time you spend developing and fine tuning your Project Plan with your clients, the less chance your course will turn into The Homer.Continue reading
Recently I’ve been enrolled in a few Coursera courses - both as a participant and as a lurker. Coursera courses are known as a MOOCS: Massive Open Online Courses. An average course can have thousands of students enrolled - a challenge in terms of effective course design, facilitation and assessment. Online reviews of MOOCS such as Coursera seem quite polarised, and as a participant in these courses I’ve been thinking about the elements that I’ve liked and want to carry over into future courses that I design, as well as elements I would like to improve.
What I like
The majority of MOOCS are delivered via video presentations. Video allows participants to speed up, pause, slow down or jump to different areas of a presentation, a big benefit for global demographics. The option to download presentations for use on other devices is also useful. Two great ways to maintain interest and attention in video presentations have involved maintaining strict recording times, and peppering presentations with interactive questions. The better presenters know to chunk their presentations to be no more than 10 minutes in duration, and have good justification for going over this limit. I’ve also liked the idea of peppering presentations with a few non-assessable interactive and reflective questions that students must respond to before the presentation moves on at certain points.
What I’d improve
The quality of presentations varies from course to course. Presenters may not be the most dynamic individuals, however this is often highlighted when the basics of video production rules are not adhered to – testing location, lighting, voice and clothing on camera before recording final takes.
What I like
The majority of Coursera courses include two tools that socially involve participants - Forums and Meetups. With courses delivered to such large audiences, Forums present an opportunity to gain first hand feedback from participants. Facilitators are able to respond quickly to topics that gain popularity such as errors in quizzes, or popular requests and feedback.
What I’d improve
There is the potential for MOOCS to empower participants and remove traditional instructional ‘paternalism’ (where the instructor is wise and students not so wise). Meetups are an interesting idea, with participants organising to meet up with fellow participants in their cities. Normally this is a social event (known as a ‘pissup’ in Australian), however there is potential for meetups to be organised better. If provided with guidelines and direction, meetup organisers could run their sessions as constructive workshops, summaries of meetups could be posted online and instructors could use outputs of meetups to structure future lessons. If organisers were provided with a virtual space, this could also be achieved virtually.
What I liked
For me, 100% online courses occupy a busy space, which means it’s easy to forget that you’re enrolled in them unless you set up a regular routine as soon as the course starts. Regularly scheduled assessments help in setting up routines around which participants can create a study schedule. An assumption with designing online courses is that participants will complete their assessments after they view all the course content. I like to see assessments used as teachable moments. The reality is, unless you have a justified reason to lock down your course navigation, assume participants will attempt their assessments at any time. This is not necessarily a bad thing.
There is a temptation in designing online courses to have only multiple choice (multiple guess) question assessments. This is not ideal, but better if there is feedback built into the quiz once students submit their responses. Allowing multiple attempts at quizzes further allows students to go back and revise material before attempting another quiz that is drawn from a large pool of questions.
What I’d improve
The challenge of designing a course to be delivered to thousands of students is trying to assess qualitative responses. Peer reviews are the answer, providing reviewers are given a clear rubric and ideally a training session in how to assess using the rubric. I would however build in an appeal process if participants felt they had been reviewed unfairly. This would be simple enough to achieve, by re-routing an appealed assessment to other randomly selected peer-reviewers. As courses are often recorded ahead of time there is a missed opportunity for lecturers to use the analytics from course assessments to structure their lessons. For example, trends in questions where participants had the most incorrect first attempts could be reviewed at the commencement of the next lecture.
Potential for MOOCS?
What I like
MOOCS are great starting points for people wanting to receive a well-structured, free introduction to subjects that interest them. Participants who wish to delve further into their subject areas can take advantage of recommended articles, books and resources. Participants also have access to their course materials once they’ve finished their courses.
What I’d improve
Clearer formal study pathways in each country would be useful if you would like to gain formal qualifications in the field that you studied. A possible alumni forum could help with members of each country posting course options that they are considering undertaking.
Version 0.95 of the Tin Can API specification was released recenty and is getting eerily close to the official 1.0 release, still scheduled for early next year. Just to clarify, it looks like Tin Can is going to be renamed the Experience API (expAPI) but I'll continue referring to it as Tin Can since that's what the industry is still calling it. Regardless of this, if you've been hearing a lot about Tin Can or the next version of SCORM, you might be wondering how this will impact eLearning and your organisation in general. The official website offers a wealth of information, but I'll try to highlight the important parts with a lot less acronyms.
Tin Can is the successor to SCORM, SCORM allows eLearning providers to produce content that can store information in a uniform way and can be moved to different platforms without modification. SCORM achieved this by tying developer's hands behind their back and forcing them to fight their way out of Alcatraz with just their pinky toe. That's a slight exaggeration, but it does mean that a lot of what people want in an eLearning course can't be done whilst remaining SCORM-compliant, and as it turns out, most people would rather have this flexibility than a pain-free migration of all their eLearning content to a new platform (which might never occur anyway).
Tin Can aims to rectify these problems and for the most part they have. By removing the requirement for content to be hosted on the LMS, it's now possible to produce trackable activities that can exist anywhere - on an external server, a learner's computer, a mobile device, even in the real-world. Tin Can isn't just for eLearning (although eLearning will still benefit from it), it also has the ability to track and report on actual face-to-face training or physical activities that take place (provided some system exists to communicate it).
This already offers immense benefits by removing some considerable technology restrictions, but Tin Can has taken it one step further and provided a means for reporting data that can be as fine-grained or coarse as you wish. At the bare minimum, an activity can report that it's been completed which brings us to almost SCORM-level tracking, but it can also drill down and report on the very specifics of a course. It's possible to produce a piece of eLearning that will report every time a question has been answered, what answer you entered, how long it took to answer it, and how many times you attempted it. Then there's also collaborative learning, where a group can undertake an activity and can report the group's progress as a whole.
But you don't really care about that, you want to know what it means to your company and how it will increase your baseline. If we take this detailed tracking and apply it over a series of activities and different learning paths, we can produce a series of analytics that monitor an employee's learning progress and compare it to their performance on the job. The concept is called Job Performance Correlation and it means you can look at performance trends, identify weak training programs that are resulting in poor job performance and start making in-roads to correcting them.
"As we start to aggregate these streams across an enterprise, or even across an industry, we can start to identify the training paths that lead to the most successful outcomes. Or, conversely, we can identify the training paths that are leading to problematic outcomes. Now we can determine the effectiveness of our training programs and measure ROI."
At Aspin, the Tin Can API or the Experience API is a big milestone on our roadmap. We have plans to take advantage of these benefits and deliver them in our eLearning courses, we want to be able to deliver courses that better track a learner's progress and can determine which areas of a course a learner is struggling with, and conversely, which areas they blitzed. We can then collate this data, identify common areas of weakness and feed it back into our eLearning development cycle to produce a more effective resource for the learner.
However, this is only a starting point, the Tin Can API will open a lot more channels for continuous improvement that we're already looking into and it's something you'll hear more about soon.
Coursera has announced an expansion of partner universities from 16 to 33 including the first Australian, the University of Melbourne. The media hype around the appeal of Massive Open Online Courses, MOOCs, is generating dynamic discussion through channels such as the University online newsletter The Conversation.
Athabasca University in Canada is offering a MOOC for anyone interested in finding out more about MOOCs and participating in one at the same time. The course started on 12 September and runs for twelve weeks. http://open.mooc.ca
The key issue for learners, and employers is how and when providers will recognise knowledge and skills gained online in open courses by implementing alternative and flexible pathways to accreditation.
Providers are experimenting, needing to find out the costs of supporting learners, producing courses and paying staff. The fear of reputation damage associated with online learning and a few "Diploma Mills" has also made University executives and Council members cautious about moving too fast.
The key driver for being an early provider is to be in a position to deliver if the market proves to be resilient. MOOCs might become the dominant source of revenue in international education and training.
Tony Bates has one of the best blogs in the education sector and it is guaranteed he will substantiate his arguments and provoke great discussion. I loved this recent post by Tony given the current hype about MOOCs, and the role of instructors.
I follow the higher education sector but would love to know how workbased eLearning professionals would respond to his questions.
I would also welcome assistance in building ElNet's resources with links to discussions and articles that provide insights to getting the balance right in workplace based eLearning.
Check out his web site, the feed came from:
I had a strange dream last night. I was in discussion with an editor from a publishing company about the draft of a new book I was writing. As in all dreams it wasn’t clear what was going on in the discussion but then I realised he wanted me to change what I was writing to make the case that computers can replace teachers in higher education. He told me that his CEO and a number of CEOs from other companies all thought this was the right way to go, and were trying to influence the market to accept this. I was so upset that I woke with a start.
Now you could say I shouldn’t drink tequila before going to bed, but this dream was not at all unrealistic in the light of events over the summer.
No, they really ARE trying to get you
Let’s start with xMOOCs and automated marking and peer review to get around the awkward point that one instructor cannot provide adequate feedback to thousands of students. No problem: a combination of big data collection and analysis and multiple-choice testing will solve most problems, and the ones that it won’t solve will be solved by dumb students marking less dumb students.
Then there’s the Republican Party of Texas whose election platform contains the following (p. 12):
Knowledge-Based Education – We oppose the teaching of Higher Order Thinking Skills (HOTS) (values clarification), critical thinking skills and similar programs that are simply a relabeling of Outcome-Based Education (OBE) (mastery learning) which focus on behavior modification, and have the purpose of challenging the student’s fixed beliefs and undermining parental authority.
(It also reads as if they oppose clarity in sentence construction, but that’s another matter.) However, if you can’t teach critical thinking skills through automated computer-based teaching, get rid of the requirement for critical thinking. Brilliant!
This was followed by the California State University system deciding to outsource online learning to a commercial publisher.
Then it turns out that the California two year college system has undergone nearly $1 billion of cuts since 2008, resulting in a waiting list of 470,000 students who cannot get into classes. Talk about creating demand for automated courses.
Still in California (do they have too much sun there?) Stanford University has just created a new Vice-Provost for Online Learning, who turns out to be a computer scientist (as are all the people heading up Coursera and Harvard/MIT’s EdX). Who needs someone with expertise in teaching for positions like this?
Lastly, in Canuckistan, the Ontario government is looking for more ‘productivity’ from the post-secondary institutions, and is asking how online learning can lead to improved productivity. In this case, that’s a good question; it’s the answers people may come up with that scare me.
Do we really need teachers in post-secondary education?
At least these developments are forcing an examination of something that most of us have taken for granted – so let’s examine it.
Here we need to look carefully at the language we use. One thing that struck me when I emigrated to North America nearly 25 years ago was that in Britain, those that delivered teaching in universities and colleges were called lecturers or professors. In North America they are called instructors. Obviously, lecturers lecture, professors profess and instructors instruct. But we talk about university or college ‘teaching’.
Now having a background in primary school (k-7) teaching, I always associated ‘instruction’ with a didactic form of teaching, where the instructor determines what content will be learned, delivers information, and the student gobbles it up and is tested on how well he or she has ‘understood’ and ‘remembered.’ This may be a fair description of a lot of post-secondary education, but in my mind it isn’t and never was ‘teaching’. To develop critical thinking skills, professors did more than lecture: they discussed and talked with students, marked assignments and gave detailed feedback. The tried to help learners learn. In other words, they taught.
Yes, this was possible 50 years ago because we had an elite system, and few students per professor. Now we have, especially in North American Tier 1 public research universities, very large classes and many students per professor (made worse in first and second year by tenured professors focusing mainly on research and graduate education). So we have fallen back almost completely on ‘instruction’ rather than on ‘teaching’ in undergraduate programs. But, given the demands of a knowledge-based society, what we need is less instruction, more teaching, and in particular a different quality of student learning.
Instruction is easier to automate than teaching
If the focus is on a didactic model of instruction, then it does become easier to automate. Choose the ‘best’, i.e. most knowledgeable, professor in the field, record their lectures, and set multiple-choice computer tests and assessment, with automated feedback, and bingo, you can teach millions with the same teacher. If you knew nothing about education, like the Republicans in Texas, this would be an ideal way of avoiding grown-up things like paying taxes or tuition fees. As they say, if you can be replaced by a computer, you should be.
However, whether this will produce graduates with the skills and knowledge needed in a knowledge-based society is another matter.
What computers find difficult in teaching
Let’s define productivity in educational terms: it’s achieving the same or better quality outcome in learning at less cost. By definition, it means doing things differently (sometimes called innovation). If we want better quality outcomes then we also need to define outcomes. I put high value on, yes, critical thinking skills, evidence-based decision-making, independent thinking, self-management of learning, responsibility, and ethics.
To successfully achieve such learning outcomes, learners either have to be incredibly self-motivated and already highly knowledgeable (i.e. already well educated), or they need an environment that supports the development not just of these outcomes, but also the development of their thinking and decision-making. This requires fostering or supporting their motivation to learn, dealing with gaps in knowledge or lack of learning skills, providing timely feedback, and above all providing guidance, criteria and direction to ensure that they meet the necessary standards to operate effectively in the real world. This is what I call teaching, and much if not all of it is difficult or impossible to automate.
It should be remembered that a very behaviourist form of computer-based learning, called ‘programmed learning’, existed before the Internet. For over 30 years computer scientists, working in the field of artificial intelligence, have been trying since then to improve on ‘programmed learning’, which failed to deal adequately with the development of cognitive thinking skills beyond the level of comprehension and memory. Despite substantial research investment over many years, AI has proved to be less than successful in the teaching domain. It is very hard to replicate the complexity of a skilled teacher who has to deal with many different variables and factors in real time. It is not just about processing speed and data management, but also about building relationships with students, making intuitive judgments, and being able to handle qualitative issues such as beliefs, values, and the personal feelings of students. Computers are not good at this (and nor are many ‘instructors’, to be fair.)
I first became excited in online learning in the late 1980s, when the Internet was being developed, because it offered the possibility of communication at a distance, a problem that was not well managed in distance education at that time. Thus the computer was not replacing the teacher, but providing teachers with tools they could use to teach even students who were remote. In the early days of the Internet, online teaching was called deliberately ‘computer-mediated communication’ or CMC, to distinguish it from programmed learning. Now the computer scientists in xMOOCs are trying to drive us back to programmed learning.
Online learning can improve productivity, but not through automation (and it ain’t going to be easy)
It’s no good cutting costs if you don’t get the desired outcome. The best example is the construction industry. Cheap construction often has expensive consequences. So we have to be sure that if we are seeking increased productivity, the outcomes are at least as good, if not better, than before the intervention.
The main ways that online learning can improve productivity are as follows:
And with the return of real students and real ‘instructors’ next week (at least in Canada), maybe reality will return and my nightmares will go away.
1. Anyone else share my paranoia? (about computers replacing computers – anything else, see a doctor)
2. Do you believe we should replace teachers (or instructors) with computers? What are your reasons?
3. Can online learning improve productivity in post-secondary education without getting rid of most instructors?
4. Can you recommend a good doctor?
You’ve designed a new eLearning course and have the luxury of running a trial group to provide feedback. The feedback that you receive sounds something like this:
'It was pretty good, I felt a bit lost in some parts, I forget where though'.
'All good, but I couldn’t find the final quiz'.
What were they thinking! Surely you can elicit better quality feedback than this to improve your course…and you can.
Think Aloud protocol - try to get into your users’ heads
One technique that I like using is Think Aloud protocol. This is a technique that aims to help you understand the thought processes of users as they complete your eLearning course. For eLearning designers this is a very useful technique because you receive first-hand real time feedback about your course that you normally don’t receive through feedback happy sheets.
The Think Aloud protocol is a technique that involves getting a user to Think Aloud as they work through your course - including whatever they are looking at, thinking, doing, clicking on and feeling. You are present during this activity as an observer and your job is to take notes of everything that the user says, without attempting to interpret their actions and words.
How do I use Think Aloud protocol?
I find the technique particularly useful when you are making major design changes to a course such as a new user interface or course template that will affect several courses. While participants work through an online course, Think Aloud protocol allows users to explain the method that they use to navigate through and complete the course. Importantly, the technique also reveals any areas of difficulty that they encounter in the process, which you can later review and improve.
Here is a basic guide to using Think Aloud protocol:
1. I recommend using a sample of at least 5 users - ideally from your target group of participants. Book a mutually beneficial time to meet with these users.
2. I prepare my resources, including the course/online resource that I want to trial, screen capture software, and pens and paper to take notes.
3. Not everyone is used to talking about their thoughts, so I have a small Think Aloud activity to help familiarise participants with the procedure. This activity, for example, might involve me asking participants to describe to me all the rooms of their home.
4. Before starting the activity it is important to tell the user that:
- You are testing the course, not the user, and that any difficulties they experience are not their fault, but the fault of the course.
- The activity is voluntary and they can stop at any time if they become uncomfortable.
- They may ask questions at any point in the process, but you may not answer all of them.
- You will not tell them when they have completed the online course/resource; they must determine this on their own.
5 I then record the screen capture software and get participants to start the course while taking notes about what they say. If I see them getting quiet at any points I remind them to keep talking with questions like 'what are you up to now', or 'what are you thinking about now?
6. Once the activity is finished I debrief and thank the user, transcribe responses and review the screen capture software recordings. The responses that you receive will inform improvements to your eLearning course/resource.
Lessons learnt from using Think Aloud protocol
One important lesson that I’ve learnt from applying think aloud protocol is to never assume anything about your course design.
In particular, DON'T ASSUME that your users:
• Will navigate through your course the way that you’ve designed it
• Will read all instructions
• Will read all content
• Will complete all activities and assessment items
• Will interact with your course elements as designed
• Won’t get lost in your course
Think Aloud protocol avoids you becoming an ivory tower eLearning designer - that is, someone who designs eLearning without any feedback from the end user. Though it can take some time to administer, the data that you receive can help improve your course design by getting a bird's eye view of the user experience.
On a recent flight back to Sydney from an excellent holiday in Queenstown, I was pleasantly amused by the Air New Zealand safety video… You may have seen it? It was a humorous cartoon style and took a different approach to any others I’d seen before. It got me wondering – does this count as e-learning?
There are certainly some fairly critical desired learning outcomes. Granted, these days many of us are fortunate to travel enough to be familiar with the safety procedures, so we may not actually be ‘learning’ anything new anymore. However for people flying for the first time, or those who travel infrequently there is important ‘learning’ to take place. And you’d have to think there is some kind of legal obligation on the airline to guarantee the message gets across!??
So is this e-learning? And what makes something e-learning anyway? Surely not every video we watch, by virtue of being electronic and informing us of something new – constitutes ‘e-learning’!?
The International Review of Research into Open and Distance Learning recently published a paper, Building an Inclusive Definition of E-Learning: An approach to the Conceptual Framework following an international project, based on the participation of experts around the world, seeking to agree on a definition of e-learning (also summarise by the Australian Policy Online site ).
That study reached the following ‘preliminary definition’ (but recommended further review):
“E-learning is an approach to teaching and learning, representing all or part of the educational model applied, that is based on the use of electronic media and devices as tools for improving access to training, communication and interaction and that facilitates the adoption of new ways of understanding and developing learning.”
This definition is pretty all-encompassing, and would probably include the safety video. What framework do you use to define e-learning? And how would this video stack up against your measure/criteria? I also wondered - do they test the effectiveness of these safety videos? Possibly do emergency simulations and see how much information people recall from the video?
When looking up the video online I actually discovered that Air New Zealand have a history of a-typical safety videos on YouTube. They probably have a slightly bigger budget that the average Australasia e-learning practitioner, but you might find some good inspiration… http://www.youtube.com/airnewzealand
ElNet Committee Member
So you need to train your staff in corporate policy? Often this training looks something like this:
1. Providing a booklet for staff to read (that you know they won't)
2. Ensuring staff sign that they have read the booklet (when you know they haven't)
3. Waving that signed piece of paper at the staff member at some point in the future when they make a mistake (to prove that they didn't read it)
Read this, sign what?
You can't 'train' staff in corporate policy by getting them to read policy. The reality is, not many people read policies for the sake of reading them. You can however make training more relevant by getting staff to apply policy to different scenarios that they might encounter in the workplace. In eLearning this is where case studies and interactive branched scenarios can become effective, e.g. "Tim and Mary are deciding between X or Y. What do you think they should they do? Why? Which parts of policy Z helped form your decision?"
Tim & Mary vs the bottom drawer
Designing a great course is all well and good, however the challenge lies in staff being able to cross that bridge between what they learn in a course (the make-believe world of Tim's and Mary's), and what they need to do in the workplace (the "real world'). One way to do this is to embed relevant information into everyday procedures, so that when staff encounter something that relates to corporate policy, they have information that can help them form a decision when they need it, and not in a booklet at the bottom of their drawer.
I was reminded of this when I read the following article last week - Banks' $2 fee has big effect http://www.smh.com.au/business/banks-2-fee-has-big-effect-20120809-23x13.html. To summarize the article: when we now use a "foreign'' ATM machine a ''point-of-sale prompt'' reminds us that we will be charged $2 for the privilege of using a foreign ATM. Since the introduction of this prompt:
- there has been a reduction in withdrawals from foreign machines from half of all ATM withdrawals to 40 per cent.
- Amongst senior citizens the proportion of withdrawals has fallen to less than 10 per cent.
- 10 per cent of people also confess to cancelling a transaction in process at least once in the past month.
What makes you cross the road to save $2
The display messages are short, clear and personal. They elicit commitment through an accept/cancel button, and most importantly provide us with relevant bank policy information just when we need it, and they reinforce this message every time we use a foreign ATM. By embedding the policy information into the transaction, customers receive the message when it is most relevant, and not in a product disclosure statement booklet (which they never read).
When it comes to eLearning, we start to realise that our messages don't need to include whistles and bells to get your point across, or superlative and erroneous information, we just need to provide the right message at the right time. This is where being involved in developing informal learning tools is imperative as an eLearning designer.
Free the content from your course
There is a grey area not being fully explored by eLearning designers, and this involves breaking away from designing online courses, and taking advantage of the "ATM opportunities" which informal learning offers, and communicating messages to our participants in real time, as compared to designing a formal course where staff need to remember how to cross the bridge between their course world and the real world. Corporate systems training provides a perfect opportunity to explore this area. As e-learning designers gain more experience in the field, we come to realise that our skills aren't necessarily just useful in creating online courses, but also in the design of organisational support mechanisms that embed learning. For a brief intro to what I see as the different levels of embedded learning check out my previous blog: http://www.elnet.com.au/index.php/news/blog/entry/if-80-of-corporate-learning-is-informal-what-are-elearning-designers-doing-about-this
I see the ability to be able to troubleshoot and search for relevant information online as a vital skill. There hasn't been a week where I haven't searched Google or YouTube to find out how to troubleshoot a technical or software issue, or find out how to build something better or differently in my courses. I call these my just-in-time learning tools - they provide the information that we need just when we need it.
Have-the-time learning (but want it free?)
If you want to learn about the theory behind eLearning the information online can vary in quality, structure and consistency. Putting together and making sense of these disparate pieces of information can be challenging if you don't have a sound foundation to build on. This is where formal courses can be useful.
Before you fork out lots of $ and enroll in a formal course in eLearning, it would be worth your while to see what is out there for free. Below are some options worth considering:
Coursera - https://www.coursera.org/
An exciting site that I've come across is Coursera, which offers courses from the top universities, for free. Once enrolled you're able to watch high quality lectures, complete interactive assignments and collaborate with a global community of students. You even receive a certificate at the end!
At the moment there are 116 courses in Coursera. Of these there are a few courses that are either directly or indirectly related to eLearning. Some that may interest you include:
- Human-Computer Interaction
- Design: Creation of Artifacts in Society
- E-learning and Digital Cultures
- Fundamentals of Online Education: Planning and Application
- Game Theory
- Creative, Serious and Playful Science of Android Apps
- Social Network Analysis
Another two sites worth a mention are Udacity http://www.udacity.com/ and Edx https://www.edx.org/ . Their course offerings aren't as large as Coursera; Udacity currently offers 11 classes and Edx currently offers 7 courses, however they are still worth a look - in particular Udaciy's Web Application Engineering course and Edx's 6.00x Introduction to Computer Science and Programming for all you Sheldon Coopers out there.
These sites present great opportunities for anyone interested in developing their knowledge and skills in eLearning, or any other interest for that matter. I particularly like that they allow for collaboration and provide a hinge on which to build your ideas and interests from. For eLearning designers they provide an insight into the enormous potential for online educational delivery, and a learning delivery model that provides accessibility to everyone around the world with internet access.
At some point in time you would have been in a training course where someone confidently talks about their own particular ‘learning style’. Learning style comments often sound very similar to horoscope conversations:
Learning style comment: I have a very visual learning style so that’s why I learn best by seeing pictures and images in my courses.
Horoscope comment: I’m a Virgo and we’re very visual people so that’s why I learn best by seeing pictures and images in my courses.
What does the research say?
Kratzig & Arbuthnott (2006) found that people’s intuitions about their learning styles may be incorrectly attributed. In particular, such styles may indicate preferences and motivations rather than inherent efficiency at taking in and recalling information through specific sensory modalities.
Coffield, Moseley, Hall, and Ecclestone (2004) found no evidence that matching instruction to an individual’s sensory strengths was any more effective than designing content-appropriate forms of education and instruction. Bloomer and
Bloomer & Hodkinson (2000) argued that learning styles are a minor factor determining how learners react to stimuli and that the effects of contextual, cultural, and relational factors play a much larger role.
Many people and organisations have blindly accepted the concept of learning styles as fact. Elearning designers may be wasting their time in designing courses and assessments that cater for individual learning styles, the overall design of which may in turn be ineffective. Conversely, learners who pigeon hole themselves into simplistic categories may be avoiding many learning opportunities and hindering their learning ability. I’d like to think that as learners we are more adaptable and complex than just a few simplistic ‘learning style’ categories. In short, sometimes it hurts to learn, and this can be good for us because we are experiencing something new. Whenever you’re having trouble learning something, don’t use your ‘learning style’ as an excuse.
Bloomer, M., & Hodkinson, P. (2000). Learning careers: Continuity and change in young people’s dispositions to learning. British Educational Research Journal, 26, 583–597.
Coffield, F., Moseley, D., Hall, E., & Ecclestone, K. (2004). Learning styles and pedagogy in post-16 learning: A systematic and critical review (Report No. 041543). London: Learning and Skills Research Centre.
Kratzig, G.P., & K.D. Arbuthnott. (2006) Perceptual learning style and learning proficiency: A test of the hypothesis. Journal of Educational Psychology 98, 238–46.
All eLearning designers have been tempted at some point in time to use a stock image for the sake of filling up space on the page. Any business related eLearning course that you develop just isn't quite the same without a stock photo depicting out-of-work models being paid to look like office workers. We can, however, easily forget about the purpose of adding graphics into our courses, and what value they actually bring to a course.
What factors influence graphic effectiveness?
Dr Ruth Clark was a keynote speaker at this year's International Conference on E-Learning in the Workplace that I attended in New York. She said that both the under-use and over-use of visuals can fail to deliver the potential of graphics to support learning. I tend to agree with this as I’ve been a participant in both text heavy and image heavy online courses, and the experience in both types of courses is equally negative in their own different ways (compare spending a week reading War and Peace vs. a week spent watching the Kardashians).
Dr Clark proposes three main factors that shape graphic effectiveness:
Factor 1: The Instructional Goal - Whether your graphics match the goal of your learning. These goals can be broken down into three main areas:
- Supporting procedural (routine task) performance
- Supporting principle-based (non-routine task) performance
Factor 2: The Learning Landscape - This consists of a combination of factors that influence your graphics, such as participant prior knowledge about the content and their spatial ability. It also includes the practical aspects that influence your graphics production, such as project timeframes, budget, organisational standards, guidelines and available resources.
Factor 3: Features of the Graphic - This includes the functionality of the graphic - both its communication and its psychological functionality: can the graphic effectively communicate its intended message?
What to take from this?
ELearning designers can spend so much time and effort trying to extract content from subject matter experts that it’s easy to forget about thinking critically about the graphics that they're placing into courses, both their function and ultimate effectiveness. First impressions of courses with lots of graphical whistles and bells may receive positive reviews from clients, however these may ultimately hinder the learning of participants. Conversely you don't want to burden your participants with heavy text screens that they most likely won’t read.
You’re on the right track if you actively try to strike a balance between the graphics that you incorporate into your courses, your learning goals, your participant needs and whether your participants are able to successfully make the link between your graphics and the communication message you’re trying to get across.
In short, don’t be lazy with your clipart!
If you want to know more about the use of graphics in eLearning, I've tracked down 2 of Dr Clark's articles online at:
I recently had the opportunity to present at the 2012 ICELW (International Conference on E-Learning in the Workplace) in New York.
Tourist hints for New York
- $100 notes aren't accepted everywhere (in case they're fakes)
- ''Straight up'' means you want your double scotch without ice
- Chipotle Mexican Grill outlets are great value
My presentation was on designing for knowledge transfer in online learning environments through analogical encoding. Below is a very brief summary of this concept:
We're good at remembering surface features but not underlying structures
Earlier this year I met a guy on the street who thought his car had a flat battery and asked to connect his jumper leads to my car battery to try to start his car. On the surface it looked like his car had a flat battery and our novice diagnosis was confirmed because the car made the usual flat battery struggling noises when trying to start. The jumper leads didn’t work and we realized we may have applied the wrong solution. This was confirmed when I realized that headlights shouldn’t work with a flat battery. Novices are great at remembering the surface features of things that they have learnt (such as the sound of a car struggling to start with a flat battery) but not the underlying structures (such as seeing the car as a system made up of several interconnecting parts). When trying to solve problems we often try to look for a one-to-one correspondence between the problem we’re trying to solve and similar problems from the past. The problem is, we're not always right when it comes to the solutions that we apply to problems.
The Inert Knowledge Problem
Successfully accessing our inert knowledge in solving problems can be challenging as memory retrieval can depend largely on surface similarity between objects, while the accuracy of transfer depends critically on the degree of structural match between the new and existing information. This relates to what can be called the inert knowledge problem - a failure to access prior examples that would be highly useful if retrieved (such as turning on the lights of a car before assuming the battery was dead). When learning through the use of cases, learners cannot be counted on to spontaneously draw appropriate comparisons between the cases, even when the two cases are presented in close juxtaposition.
Analogical encoding is a technique that proposes to help learners apply inert knowledge to new situations. Analogical encoding is a simple concept; however it’s something that we may not be considering when designing online courses, particularly those that involve the use of cases. Analogical encoding involves learners being asked to compare examples (such as comparing two cases) to reveal common structures, regardless of the content of the domains being compared. The understanding of the common structures can then be applied to a new situation, target problem or issue.
Where Comparison is useful
Analogical encoding is particularly promising for learning environments involving novice problem solvers who often focus on surface features, such as similar characters and settings, rather than similarities in relational structure. It is also promising for learning environments where participant knowledge and skill levels may vary, and facilitator feedback may be limited, as can be the case in online learning environments. In learning design involving cases it may be assumed that learners will solve problems through the mapping of existing knowledge to new information. This expectation however may not be directly or indirectly built into the design of the learning environment, and as a result learners may miss relevant structural relationships.
Where can online learning environments help?
The majority of past research in analogical encoding has been conducted via paper-based experimentation. Online learning environments however enable various features and functionality that could potentially benefit analogical encoding activities and are not available through a paper-based medium. Hypertext allows learners to actively compare examples to reveal common structures, as well as conduct efficient searches within the text. Multimedia, such as video and interactive graphics, can provide a richer and more personalised experience for learners, and use of online interactivity may also assist in helping learners develop active relationships with the material.
Want to know more about analogical encoding?
A good start would be to check out the following articles:
Gentner, D., Loewenstein, J.,&Thompson, L. (2003). Learning and transfer: A general role for analogical encoding. Journal of Educational Psychology, 95, 393–405.
Gick, M. L., & Holyoak, K. J. (1980). Analogical problem solving. Cognitive Psychology, 12, 306–355.
Last week's Elnet's Workplace Congress 2012: Going Mobile and Supporting Learning provided a great opportunity to mingle with fellow eLearning folk and listen to some great presentations. What I liked best was the great cross pollination of ideas from participants throughout the day.
One theme that really stood out for me is that the quality of technology and availability of online applications in our homes is often leaps and bounds ahead of what is available to us in the workplace. During the conference I heard amusing stories about organisations that still insist on using outdated browsers and outdated flash players (it seems that IT staff keep themselves employed by running compatibility testing that can take a few years). There were lots of stories about organisations that either partially or fully lock down their internet so that staff need to hide in the toilets to check the internet on their mobiles, even for legitimate work use. A related theme was the disparity between the types of online services that we use at home and at work, including social networking and Web 2.0 applications. The growth and benefits of web 2.0 applications can't be ignored by organisations - just have a look at http://www.go2web20.net/, a great web applications index site to gain a sense of what's out there.
The reality is we now have a new generation of employees that have never known what it was like to live in a world without the internet. Schools are now starting to embrace BYO Technology and Devices (BYOT and BYOD): http://www.smh.com.au/technology/technology-news/smarter-use-of-home-devices-20120527-1zcry.html and the expectation of this generation of future employees will be a great impetus for change at work. A great quote from the conference comes from Anders Sorman-Nilsson: 'Change doesn't care whether you like it or not - it doesn't need your permission'.Continue reading
As a follow up from the Sydney workplace congress, I asked the presenters if they could compile a blog post to summarise their presentation and to link in the take home messages for mobile learning within the workplace. Here is the contribution from Alison Bickford from Connect Thinking.
Link provided below
Thanks Alison for a wonderful presentation.
Facebook shares anyone?
Recently General Motors killed a $10 million Facebook advertising campaign citing that the service had not delivered effective results. This got me thinking about my experiences with car advertisements and whether the design of the advertisements could be partly to blame. When viewing news sites on my iPad, nothing loses my attention more than a car advertisement popping up on the screen. They all seem to have a similar format: a picture of a car driving around a picturesque scene, a catch phrase to try to tickle my emotions, a list of features that I'd expect all modern cars to have, and a paragraph written in very small font with terms and conditions that end up doubling the price of the car. Pretty much it's the same ad that appears in magazines, so why would I want to click on a car advertisement when I know I won't learn anything, won't be entertained and won't be engaged in any way?
Same Same but Different
When the TV remote control was invented, advertisers quickly realised that they needed to capture people's attention better than they had in the past. With online advertising, advertisers have even less time to entice us because we can click or swipe off ads when they appear, or decide to not click on them at all. We need to be aware that technologies like Facebook are still quite new, and companies are still trying to find out how best to leverage from them. Sorry advertisers, but converting a print ad to html just doesn't work for us anymore.
Thanks Don Draper, but how does all this relate to mobile learning?
The tricks of the trade that were effective in designing eLearning courses in the past may not work with mobile learning. When designing mobile learning we need to be aware that the courses that we design shouldn't just be transferred to online mobile format; we need to ensure that they are transformed. When transforming a course for mobile delivery, certain principles apply:
1. Design for the device: If you have a mix of participants using mobile phones and tablet PCs ensure your CSS can distinguish between the two
2. Personalise: Keep your content relevant to your participants and their environment
3. Mobile contexts: Design for learners on the move. Learners will leave the content and return to it at different times and at different locations
4. Blend: Consider blending your mobile and non mobile technologies to add value to your overall course content
5. Simple and intuitive: Mobile learners are distracted learners, don't let your design add to the noise
You know you have been successful in transforming a course for mobile delivery when you receive positive qualitative and quantitative feedback through surveys and reporting tools. Courses are never truly complete because the feedback that you receive informs improvements for the next group of participants that enrol in your course.
The Elnet Workplace Congress 2012 - 'Going Mobile and Supporting Learning' - will be held on Friday May 25th in Sydney. With this theme in mind I wanted to explore some ideas on augmented reality.
Augmented reality enhances our perception of reality through a live, direct or indirect, view of a physical, real-world environment whose elements are augmented by computer-generated sensory input such as sound, video, graphics or GPS data. For designers of training this should be an exciting time, however the technology isn't yet mainstream and there still seems to be reluctance for designers to jump in the deep end with augmented reality.
Firstly what are the types of Augmented reality?
There are three main types of augmented reality:
- Marker-based: a two dimensional barcode type image that you point your mobile device at (see the example I photographed below from String's augmented reality showcase)
- Marker-less: which typically use the GPS of your mobile device
- Hardware:such as Google's proposed augmented reality glasses
But I'm right brain!
From talking to different eLearning designers, one assumption they have is that you need to be a programmer to make anything augmented a reality. This may have been the case when the technology was new, however now there are lots of non-programming augmented reality software for you to use. This is great news for right brain designers who like to keep an arm's distance away from left brain programmers!Think outside the 'real' box.
Think outside the ‘real’ box
Another issue with augmented reality technology being relatively new is that designers may assume that the technology is still novel, and have trouble conceptualizing where it can benefit their current training delivery. A solution for this is to immerse yourself in examples of augmented reality. You really do need to think outside of the box and envisage possible applications for the different augmented reality whistles and bells, and how they could add value or improve current training and business objectives. Imagine an induction course where new staff could take a tour of their workplace through augmented reality. They could stop at different checkpoints, read, watch videos, take pictures, regroup and discuss their findings. Imagine being able to train your customers in new products via their mobile devices, so that they could view available colours and see what your products would look like in their homes. The opportunities are endlessly exciting.
The reality of augmented reality
So don't be like that friend you knew in the late nineties who refused to own a mobile phone. Dip your toes in augmented reality, have a play of some examples and download a few free software trials. The reality of augmented reality is that the technology exists, it's available now, it's getting easier for designers to utilise and it won't go away in a hurry. From a business perspective your outlay is minimal because the chances are your target market already owns a mobile device. Even that friend of yours …hopefully!
See you at the conference.Continue reading