Measuring Learning in Workplaces

How do you solve a problem like measuring learning? In schools, the traditional answer has been to give students a test. I’m a firm believer that tests have a time and place within education, but not necessarily in all the times and places that they’re often used. They’re great for giving students the opportunity to see what they know and what they need to work on, giving teachers the chance to see what they need to go over again, and giving schools a feel for how their students are doing. Tests are even great for giving students a deadline to focus on – I know I work better with clear deadlines, especially just before those deadlines. But how good is a test as a measure of learning? As Bernie Sanders knows well, tests often cause people to study only the test material, neglecting other opportunities for growth. So, tests are good for measuring a bit of what a person has learnt, but not everything. And what about adults in the workplace? If you are a company trying to understand how well your workers are learning, can you just give them a test? Is that a good way to understand if they’re learning?

This question of how do you know if adults are learning in a workplace was an issue that I encountered during my PhD when working with energy companies. In the energy sector, an accident can mean somebody loses their life; so a lot of resources, time, and money are invested in safety and learning from incidents. Accidents and almost-accidents are investigated to understand how something similar could be avoided in the future. These insights are generally summarised and sent out in an email blast so that all workers can learn from the mistakes of others. But how could a company know if people are actually learning from these summaries of incidents? Would a test be useful in this scenario?

It could definitely have a place, but it’s likely that any test would only make people study the details of that particular incident, which isn’t really the point. Ideally, workers should be learning by reading about an incident, its causes, and then thinking about which bits apply to them. As anyone who has ever seen Bloom’s Taxonomy of Learning can tell you, applying ideas from one place to another is a more difficult type of learning than simply memorising facts (I acknowledge that not all tests are about memorising facts, but it’s tough to write a really good one!). So a test could be useful, but it’s probably not enough on its own.

I’ve now finished my PhD and I still don’t have the ultimate answer for how you can measure learning in workplaces. But there are two ideas from outside education that I think could be helpful. The first is the idea of an index. Environmental researchers work with indexes a lot to try and put numbers to something that’s difficult to quantify. A habitat suitability index, for example, will strive to come up with a single number that represents how suitable a location is as a habitat for a particular species, therefore inferring how likely you are to find that species there. This is done by quantifying different metrics based on location variables, such as geographic surroundings and climate, and on species needs, such as food and shelter availability.

Something similar could be useful for learning. Instead of placing all your faith into a single number, like a test score, chose a couple of different metrics and create a learning index. Inputs could include amounts of time dedicated to discussion, motivation of workers, amount of jargon in slides. Just like in the life sciences, companies could put a number to each of these, to get an index of how conducive to learning the materials and methods the company uses are, and therefore how likely it is that learning is happening in the workforce.

The second idea that I think has some use in this context is bricolage. What’s important for learning will depend a lot on what it is you’re trying to learn and why. As a company, you want to limit time and energy spent collecting, quantifying, and analysing lots of different sorts of data to measure learning (it’s a full-time job for many researchers!). Bricolage is the idea of selectively bringing together the materials you already have to solve a problem. Rather than designing the perfect solution to a problem, you create something from what you have. As my US friends would say, you MacGyver it.

I think this could be a really useful approach for learning. Think about what’s important for creating a suitable learning environment; think about what data you already have that gives you insight into those important factors, and create an easily updated baseline. Especially in a world where every click and hover on your computer can be recorded, this method becomes increasingly feasible. There are likely to be important things that are not captured in the numbers you have available, but at least it’s scalable and a starting point to know where to look a bit deeper. The trick is to not get drawn into the trap of believing your data tells the complete story and represents reality.

Measuring intangible things like learning is not a problem unique to education. Outside of education, there are several approaches used that, perhaps if modified, can offer insight into how to look at what is happening and adjust accordingly. Maybe one (or a combination!) of these approaches can have the final say on whether people are learning.


What we have learnt about tackling COVID-19 so far?

The British government currently faces a tough question: should they prioritise the economy and keep businesses open, or should they prioritise safety and enforce a strict lockdown? Their ultimate responsibility is to the well-being of the people, but does that mean protecting the UK residents’ current health or their future prospects? It is a tough question, clouded in uncertainty.

              Part of the issue is the limited amount of data. When 1000s of people were dying early in the pandemic, there was a moral imperative for the British government to do everything in their power to save lives. Now they find themselves once again in the situation of needing to curb the rapidly increasing rates of COVID-19. To stop the spread of the disease, the government will try to follow scientific advice. But it is difficult to know what prevention measures are the most effective, as when governments move quickly to save lives they put several measures in place at the same time. Would wearing masks alone be enough to stop the spread if everyone did it, or do restaurants have to control the space between their patrons as well?

              At this point in history though, there are tools to pick out the best approaches. Based on just the data of this current pandemic, the government could look across at other countries to learn which measures have been successful. Regions such as Hong Kong and Australia have stemmed the flow of COVID-19 with swift surveillance techniques, quarantine measures and social distancing. The Global Response Index, created by foreignpolicy.com, shows that countries which implement measures quickly, make contingency plans early on, and have effective testing strategies have fared best. These are endeavours the British government should focus on, especially building testing capacity which, 6 months into the pandemic, is still lacking compared to many countries.

While building testing capacity sounds like a daunting task, this is an area where there are lessons to be learnt from others. Africa as a continent has struggled to obtain PPE that has been bought by wealthier countries, but international co-operation via the African CDC has enabled what has been available to be distributed effectively. African countries have also demonstrated incredible innovation with their resources. Ghana has used pool testing to increase their testing capacity (i.e., pooling samples from multiple people for a single test and only conducting individual tests when a positive result is found). Likewise, Senegal has created a test for coronavirus that works in 10 minutes and costs only 1USD.

Another lesson comes from Vietnam, which had a highly-effective communication strategy. The basic measures of washing hands, wearing face makes, and social distancing are very effective if everyone follows them. However, after 6 months of restrictions people are starting to fatigue and take small risks here and there. At the start of the pandemic it was very easy to walk around the supermarket as everyone was conscious and keeping their distance. Two days ago I for the first time asked someone at the supermarket check-out to keep their distance as they encroached on my space. But it wasn’t the first time I’d wanted to ask someone. As people fatigue the government could look at some of Vietnam’s methods for communicating and reminding people of the dangers.

My PhD looked at how energy companies learn after incidents. Something that is spoken about often in that context is risk normalisation. This concept is that is that when you work next to something dangerous everyday without experiencing any incidents, you start underestimating the danger involved with taking risks. You assume if you take a small risk here or there nothing will probably happen. Energy companies are very aware of this, which is one reason they invest so much in communicating about safety and incidents. Vietnam’s communication strategy probably had a similar effect.

There is a human aspect that must be factored into whatever the government decides to do. Many have gone 6 months following the rules, without actually experiencing any personal losses to the disease. Today the total number of people identified as having had COVID-19 in the UK is 618,000, which is less than 1% of the population. Even fewer people have had severe symptoms, and even fewer have died from it. People are starting to normalise the risk of getting COVID-19, seeing it as something that others in the news get, but not something they as an individual are likely to suffer from. But as the most recent outbreak in the US shows, those who do not show the virus enough respect are likely to become super-spreaders. The government needs to invest in its communication strategy and consider what guidelines are going to work given the current mindset of people. The support bubble system is one of the best parts of the government’s current regulations. If people have their basic needs for connection met, then they’re more likely to be able to follow the rules. This is something that the Dutch government understood early on.

The current British government continues to face incredibly difficult decisions. They must listen to the advice of scientists and follow the example of countries that deal well with coronavirus. Increasing testing capacity is definitely a must. However, the fatigue of people in the UK cannot be ignored when deciding how much to enforce with fines and how much to trust that people will act for the greater good. British people do care about others, but after 6 months of little consequence for the majority, it is easy to start taking small risks as there will most likely be no negative consequences. Am I really likely to spread coronavirus from one embrace with my parents? The problem is, if everyone in the country takes those small risks, then the virus will be spread quickly.

In the end, an investment in testing capacity and marketing campaigns might be the way to get around the need to decide whether to chose between people’s current health or future well-being. People want to do the right thing, but after 6 long months small reminders via marketing can help tip the balance when people need to make those split-second decisions about taking small risks. Larger testing capacity could also help people to connect in a safe way. Imagine that we each had one test that we could use every month. Before meeting my parents for a BBQ we could all take a test to elimate the risk. Instead of thinking about how to force people to do stay away from each other, the government needs to think about what people are experiencing and provide practical advice and resources to support people through this difficult time.

Is Finland really an educational paradise?

I am lucky to have had first hand experience of several of the world’s most highly rated educational systems: as a primary, secondary and university student in the UK, a university exchange student in Singapore, a teacher in Japan, and a university student in Finland.

 

PISA is undoubtedly the most recognised national ranking for educational systems, and was what originally made Finland’s educational fame. Although the most recent versions of PISA have seen Finland fall to around 12th place in Maths, 5th place in Science and 4th place in Reading, Finland still has an amazing reputation for education. Whenever you hear of Finland’s educational practice it is always in relation to things like this video that was popular on Facebook (it’s a heavy handed treatment of school but makes some valid points). Finland is touted as an educational paradise where students out achieve other nations because of radical ideas like shorter school days or no homework.

 

So is it true that Finland is an educational paradise? Spending 2 years on the international Master’s programme at the University of Jyväskylä gave me some insights into what it is that Finland does that works, and what is perhaps over hyped a bit.

 

One of the first things that normally appears in an article from the US about Finnish education is the lack of homework, the shorter days, and the larger number of breaks. This is true to an extent. One of the great things about Finnish education is that the amount of homework and lessons in a day starts out relatively little but increases as a student gets older.  Especially for younger kids there is far less than scheduled time than in some other countries. Harris Cooper and colleagues conducted research into how much homework is good for students and found that for elementary students the only real benefit could be developing good study habits, so 10-20 minutes a day for young children is fine. Finland takes this to heart. Allowing breaks through the course of the day also aligns well with what we know about the ability of humans to concentrate in general. The Pomodoro Technique is a popular way to get more work done for adults, splitting the working day into 25 minute work sprints followed by 5 minutes breaks. If this is an effective way for adults to work, then it makes sense that younger students would find it harder to concentrate solidly for an hour and would need a break after each lesson.

 

However, is this shorter day what really makes the difference in Finland? In a previous blog I mentioned that getting into teacher education in Finland is incredibly difficult, and this is probably one of the largest contributors to Finland’s high quality system. A very good read for those looking to know more about what makes a real difference in a classroom is John Hattie’s Visible Learning. Hattie took over 800 summary articles (a.k.a. meta-analyses) to try and understand what has the biggest effect on students. The biggest impact seemed to be a quality teacher, so it follows that Finland’s teachers needing to go through a competitive selection process alone could be what propels it to near the top of the table. However, there are several other elements to the teaching profession that aid Finnish teachers, such as continual professional development and more freedom over what they teach than in other countries. The relatively free reign of teachers is perhaps only made possible due to the fact that students have few large assessments through the course of their education. As far as I know smaller tests are still a part of the educational ethos, but seen as a way to give feedback, rather than the ultimate goal. A society that trusts and respects teachers is likely to produce quality education.

 

This can also be observed in Japan, where teachers are seen as civil servants (as each teacher is in fact a government employee). Currently PISA is dominated by Asian countries, with Singapore topping the tables in Maths, Science and Reading. I believe that at least part of the reason for that is because of the status afforded teachers in society in many Asian countries. However, PISA isn’t everything, and if I had a child I would rather send them to a school in Finland than to a school in Japan. The Japanese system has some fantastic points, not least the sense of responsibility and community that it ingrains into its students. Every member of every class has a role of some sort, whether it is being in charge of delivering school lunch to their peers or recording the homework from English class. This sense of each person having their place and a role to play is great, I think it really helps students to learn the importance of responsibility. The difference, however, comes in the number of hours that are dedicated to education. Students in Japan work long and hard, usually spending a couple of hours doing club activities after school, before heading to a cram school to study some more. Korea I hear is pretty similar. The high suicide rates in Japan and South Korea due to huge pressures placed on students are well known, and I would guess have a lasting effect on youth. While I was living in Japan I remember my parents visiting me and one of their first questions was ‘why are there some many students wearing uniforms on the weekend?’ Education is a lifelong pursuit, but I don’t believe that enforced formal education should be a person’s life. As Mark Twain famously said, “I have never let my schooling interfere with my education”.

 

One element of Finland’s education success that I haven’t heard spoken about often is a cultural factor. Finnish schools are free, in fact it is for the most part illegal to charge tuition fees. Finland is also one of the countries in the world with the smallest gap between the rich and the poor. This, combined with such high entry requirements for teachers, means that schools across the country provide relatively consistent education. As I previously wrote about, there are a number of advantages to ensuring that students are not divided by income with the rich attending better schools.

 

So, as Finland appears to fall in the PISA ranks will it lose its reputation as an educational wonderland? I think not, but as many countries develop their own education systems it may not shine quite as brightly. There are certainly many things it could learn from other systems as well, such as the community spirit of the Japanese. Really there are no ideal education systems, as each country must strive to create something that will help them to develop and prepare their societies for their own unique situation. It is certainly a good system, nonetheless, and one where many others could find good ideas.

Education for all, and for all a good night

A month or two ago it was accidentally let slip by the British government that they intend to bring back grammar schools. This has been a controversial decision so I wanted to consider write a bit about them. However, I feel it important to note that I am personally against grammar schools, so this blog post should be read with that in mind.

Grammar schools are a type of school in the UK that have traditionally selected students based on their grades. There are currently some grammar schools in the UK, but since 1998 there has been a ban on the creation of new ones. This ban is what the government is proposing should be lifted. The government certainly seems to have its heart in the right place when lifting the ban on grammar schools. The schools that the government wants to create will take the best and brightest at the age of 11, and, regardless of economic background, will provide them with a high level of education. The intention of grammar schools is therefore to enable social mobility and ensure that those from less fortunate backgrounds have the same opportunities as those from wealthy backgrounds. Enabling social mobility and striving for equality sound like great goals, so why am I against grammar schools?

While I support the sentiment behind grammar schools I believe that the Matthew effect will come into play in this kind of arrangement, meaning that the goal of enabling social mobility is unlikely to actually happen. The Matthew effect is an interesting idea: those who have much will continue to get more, and those who have little will get even less. Malcolm Gladwell’s book Outliers gives great examples of this in action. The basic idea in society is that those who have had an advantage will be presented with more opportunities, which in turn will become even more advantages and lead to greater opportunities. This explains, for example, why British footballers tend to be born in September. The British school system begins its academic year in September so those born in this month are the oldest of their peers, and are almost a year older than the youngest. When you first enter school this extra year makes a huge difference in terms of physical strength and coordination so those born in September are at a natural advantage in sports at that age. This then allows them to join the football team, where they will receive additional training. Due to the additional training the gap between them and the younger members of their peer group becomes even wider, and makes the older students more attractive for selection of regional teams, where they will receive even more training, which in turn increases the gap.

While grammar schools are intended as a way to lubricate social mobility I predict the reality to be quite different. Those who have come from more fortunate backgrounds are likely to be better at taking tests, and therefore more likely to be able to enter a grammar school. Likewise, those who come from more fortunate backgrounds are more likely to have parents who can afford to send them to additional preparation classes for the grammar school tests, which is a huge industry already in Japan and Korea where many high schools and universities have their own entrance exams. An argument against this is that the 11+, the test that grammar schools will use to select pupils, is an IQ test rather than a test of knowledge and test taking ability. Except that it has been shown that those from lower socio-economic statuses perform poorer on IQ tests from ages as low as two, and the gap increases as the children age.

An argument could be made that at least it is better than doing nothing, where there would be no pathways for intelligent children from poorer families to receive a top class education. However, this doesn’t appear to be the case when you look at which school systems are doing well. McKinsey & Company produced a report in 2017 entitled “how the world’s best performing school systems come out on top”, where they looked at what factors made the best school systems in the world so effective. Their conclusion was three things seemed to encourage high quality schooling: make sure the right people become teachers, train teachers well initially and continue developing them through their careers, and monitor students to allow interventions when someone starts to fall behind. In my opinion grammar schools will actually go against some of these principles.

One of the things I disagree with most about grammar schools is that it will cause good teachers to be drawn to grammar schools, where the students are likely to be higher performing. Currently the salary for teachers, along with the long working hours and the status of the profession in general, mean that the best of the best are not often attracted to teaching. There are many amazing teachers in the UK, but there isn’t enough competition to ensure that only those who have a passion for teaching enter the profession. In Finland, for comparison, getting into teacher training is about as competitive as getting into MIT, so the status that teachers have in society is very high. By ensuring that all teachers are high quality means that the level of teaching at all schools is good. Japan and Korea, on the other hand, have an interesting mechanism where teachers are randomly moved every couple of years to different schools, potentially even to a different city. This is quite an extreme system, but it does ensure that teacher quality is roughly even across schools.

With relation to monitoring students and helping them when they fall behind, this sounds like an area where grammar schools could be a positive. By separating the brightest and smartest won’t that leave teachers to adjust the level of classes to the students, ensuring that less students are less behind as they’re at more suitable levels? I am divided on this and am eager to see what happens in the future. However, is the separation of students of different backgrounds not depriving kids of one of their greatest forms of support: each other. Many studies, for example those by Rupert Wegerif and Neil Mercer, have shown the power of students supporting each other and building many skills by doing so. If you take this approach to education, that students should support each other and develop collaboration skills, is it not beneficial to have as diverse a classroom as possible? After leaving schools students might find themselves in a whole range of environments so should they not learn to communicate with people from a range of backgrounds? Furthermore, I’m sure that there are many students who don’t perform well on the 11+ who have talents in particular areas that could benefit others in the class who have theoretically higher IQs overall.

The question then becomes if not grammar schools then what? In an ideal world I would love to perform an experiment to compare what would happen if the government invested the money it intends to spend on grammar schools instead on teacher salaries. Would that increase the status of the profession and draw more people to the job? Or would investing money in more teaching assistants to provide targeted help for those who need reap better results in the long term? It is easy to criticize the government’s decision to remove the ban on grammar schools without proposing alternatives. In the end the educational system is a complicated beast where results of policy changes cannot easily be predicted with 100% accuracy. That does not mean, however, that policies should be made without looking at what has worked in other countries and what research has suggested.

The World’s Hardest Language

Although learning languages at school was never something that I enjoyed I have since taken quite an interest in them. While living in Japan I undertook the task of learning the local language, gaining conversational ability. Since leaving school I’ve also taken classes in Mandarin, Finnish and Spanish. It’s well known that some languages are more difficult to learn than others, and there are a lot of lists out there on the Internet acting as guides (for example herehere and here). Mandarin and Japanese always seem to in the top with Finnish not far behind. While I definitely agree that for a native English speaker these languages are more difficult to learn than Spanish, the reasons given often centre around grammar and the difference in concepts compared to English. Personally I found Japanese to be far easier to learn than Finnish, for many reasons beyond just how different the language is structurally to English. Here are five considerations of language difficulty besides grammar.

1. Your Goal

The Finnish language was once described to me as being an upside-down pyramid. When you begin studying it the grammatical structure is likely to be drastically different to your own language (unless you are Hungarian or Estonian), and the vocabulary is probably also going to have very little connection. It’s very difficult at first, but then when the rules and vocabulary are learnt it’s possible to perfect. Japanese on the other hand is the reverse, when you initially begin to learn it doesn’t take long to be able to have basic conversations. The grammar is different but logical, and there are very few exceptions or irregular words. To master Japanese, however, is incredibly difficult due to the need to perfect honorific language, a task which even native speakers struggle with. The different writing system present in Japanese and Mandarin are also often presented as a reason for its challenging nature. The question though is what would you like to achieve by learning the language. If your aim is not perfection but communication then Japanese’s easy pronunciation and regular grammar mean you can speak well relatively quickly. Mandarin’s tones are a challenge, but the grammar is some of the most straightforward of any language. If your aim is not perfection, but rather oral proficiency then both these languages are likely to be easier than even more familiar languages such as German.

2. Motivation

Motivation is well known to have links to educational outcomes, after all if you have no motivation then you will certainly not progress much. It is a complicated issue though. While there are some obvious motivating factors dependent upon a person’s individual circumstance, the need to talk to a partner’s family or promise of work advancement, for example, there are some less obvious factors that can help maintain motivation while learning a language. Interest in the target culture is a primary consideration before embarking on learning a language. Culture and language are closely tied, so if you don’t particularly like a certain culture then their language is unlikely to hold your attention for long. I am constantly surprised by people who tell me that they want to learn Chinese because of China’s global economic importance, but have no interest in ever going to China. Your real life experiences with a language will be what make or break your motivation with regards to studying it, if you don’t want to visit a country that speaks a certain language then there are probably better choices.

A similar thought for those considering which language they should study is ease of meeting real life speakers. I’ve been toying with the idea of learning Esperanto, after studying Spanish it looks incredibly easy to pick up, but in the end it would be difficult for me to ever use in realistic settings so I have decided not to. One of the best things about studying Spanish and Japanese is that I can encounter them regularly in my life at home, as well as desire to go to the countries that speak the languages. Spanish is one of the most widely spoken languages in the world, and is often used in US TV shows. Japanese tourists are some of the most enthusiastic in the world and can often be met abroad. Besides having an extensive movie and TV production business, Japanese is often heard in pop culture as a language of business men, ninjas or Yakuza. One of the biggest factors that I’ve heard cited for a lack of motivation for studying Finnish by my peers is that only 5 million people speak it natively. Whatsmore, of those 5 million people a very large percentage have functional English, meaning that even when you do meet a Finn, Finnish is unlikely to become the lingua franca.

3. Directness of Target Culture

The time will eventually come when learning any language when you need to interact with native speakers. Suddenly being asked to communicate with people who aren’t aware what set phrases your textbook included is intimidating, and yet so rewarding if successful. So what affects if this transaction is successful? Your skill in the language is an obvious factor, but so too is the ability of the person you’re talking with to guess that your utterance of ‘Have orange, apple, banana?’ means that you would like to know if the shop has any fruit.

While there are several ways of classifying cultures on different scales, one of the the most influential is that of the high- and low-context societies based on the work of anthropologist Edward Hall. This concept can be seen well in English by comparing the US and the British culture. As this Buzzfeed article illustrates, the British often say things and expect their conversation partner to understand a hidden meaning, while this is rarer in the US. The US would be described as a more low-context society than the UK as the literal meaning of the words are more likely to contain all the information needed, regardless of the context of the conversation. East Asian societies are often very high-context where much information is left up to a person to infer from a situation. While this may sound frustrating to those from a low-context society, it means that people from these cultures are more used to placing less importance on specific words and using context to provide clues as to what their partner might be trying to communicate.

I witnessed a good example of this when I first arrived to Finland, a very low context society. I stood outside a restaurant with two Finns and a Chinese friend at lunch time. The friend was trying to communicate in Finnish that we should go and eat by saying ‘let’s go for food’ in Finnish. His pronunciation of the Finnish word food, ‘ruoka’, however was not great and the Finns seemed to have no idea what he was saying. I knew perfectly what he was saying, and from context it was obvious that this was what he was trying to communicate. However, the Finns had to ask in English what he was saying. If perfection, or something close to it, is needed to communicate in a language then the language becomes very challenging.

4.Availability of Material

Everyone has different ways that they like to learn, and your preferred study methods are likely to vary depending on what is your eventual goal. The more popular your language is then the more likely it is that there will be study material to match your desires. I am a huge fan of both Duolingo for Spanish, and make use of a lot of the pre-made flashcard decks available for Anki for Japanese. Finnish has resources available online, as well as in textbooks. However, these materials are not nearly as extensive as those for many other languages. Variety is important for keeping motivation high, so this is a very practical consideration when deciding if you should begin studying a language.

5. Success of Others

While taking a slightly different perspective than the other entries in this list, the success of others in a similar position to yourself is another thing you might want to think about when starting to study a language. The planning fallacy is a concept that Daniel Kahneman talks about in his book Thinking Fast and Slow. The basic idea is that people always underestimate the amount of time that they predict a task will take to complete. One way that he suggests of getting around this is by looking at how long it has taken others to complete similar tasks. This is often a much better estimate than if you think about each of the steps that will be necessary and how much time it will take. A similar approach can be useful to thinking about your success when it comes to learning a language. When I lived in Japan a lot of the foreigners I knew who were living there spoke excellent Japanese. In Singapore the number who spoke Chinese was considerably less, however I have heard that those who actually live in China often develop a decent command of the language. While I know many foreigners who live in my city in Finland the number who speak Finnish well is small. A person’s environment is a huge influence on what he or she is likely to achieve, and so the achievements of people in similar situations to yourself is a sensible consideration when thinking about potential language success. Despite this, however, every person’s situation and internal motivation is different, added to which no language is impossible to learn. Studying a foreign language is a useful and enjoyable activity for most, and the process itself may be enough of a reward regardless of resulting language proficiency.

The Placebo Effect and Education

How do you measure learning? What do you want a participant to walk away with after attending training? These are questions that educational researchers have been considering for a long time. Often when researchers want to see how effective some kind of educational intervention has been they will give students a test. It’s quantifiable and so easy to compare results before and after an intervention, or between an experimental group and a control group. Indeed tests are a favourite way to assess many things, from individual knowledge to the standard of whole country’s education system. While many dislike tests, and teaching solely to a test can hinder growth rather than encourage it, there’s no doubt that the ability to recall facts, rules and information is useful. Good tests even include problems that require candidates to apply what they know and show understanding. So is this the best way to assess progress?

A paper that I often like to refer to is Dunlosky et al.’s (2013)1 meta-analysis that compares ten of the most popular revision methods employed by students, and how effective they actually are. To give a sneak preview it seems that periodically testing yourself on material is a much better way to learn than reading your notes three or four times the day before an exam. However, if you were to ask how confident students felt after using the techniques then they would probably feel more comfortable after re-reading their notes. Their ability to recall data might be lower, but as testing yourself means you’ll make mistakes it’s easy for a student to feel like they haven’t learnt much, despite each wrong answer being a great learning opportunity that will stick in their head! On the other hand, when re-reading notes everything seems familiar and so a person’s confidence grows. The problem with this is that as everything seems familiar people don’t actually pay close attention. It seems obvious that this is a false confidence, and so the methods that actually improve recall ability are definitely better, right?

Recently I was reading Ben Goldacre’s book Bad Science and came across the chapter on the placebo effect. In an experiment where patients with pain were given sugar pills and aspirin it was shown that the sugar pills had some effect, although not as much as the actually aspirin. Nothing too surprising there. What I found very interesting, however, was the fact that both the sugar pill and the aspirin were more effective if taken from a branded box than a plain black one. The mind has a powerful effect on the way our bodies work. In educational research there is a popular theory called Social Cognitive Theory, a collection of ideas popularised by a psychologist called Albert Bandura. The theory has many parts but one central portion is self-efficacy – a person’s confidence in their ability to successfully complete tasks. How people behave is directed linked to their beliefs about what the outcome of their actions is likely to be. This applies to educational contexts as well, if a person is confident that they will be able to learn, then they are more likely to learn. One of my favourite quotes on learning, from Frank Herbert’s sci-fi novel Dune, nicely sums up this idea:

We can say that Maud’Dib learned rapidly because his first training was in how to learn. And the first lesson of all was the basic trust that he could learn. It is so shocking to find out how many people do not believe that they can learn, and how many more believe learning to be difficult.

Your confidence in what you have learnt influences your emotions, which in turn influence how receptive you will be to learning. Self-efficacy is therefore not a placebo effect, the fertile mindset it creates for learning is genuine. If confidence is not considered then motivation can become a serious problem, and over an extended period of time learning can be hindered. What self-efficacy does share in common with the placebo effect is that the perceived value of your experience affects how effective outcomes are.

As an example of this in practice I’d like to offer my experiences learning Finnish. Our classes were excellent, designed for those living in Finland, who would often encounter situations where they don’t understand much around them while their language skills are still in a fledgling stage. The teachers employed a lot of good tactics for placing students into uncomfortable situations, trying to teach them the skills to survive and function. Nonetheless, this is not the way that many are used to studying a language. In schools grammar is normally given a central importance, and exercises start from basic sentences with perhaps only a couple of new words. When you have years to develop a language this is a fair way to approach its development, but it develops few strategies for those who need to have some competence almost immediately.

In my opinion people learnt a lot from the course. They could quickly survive in a restaurant conversation, or ask for directions. Perhaps most importantly strategies were developed to help students guess what is happening in situations where they don’t understand much. People’s opinions at the end of the course were, however, that they had learnt little. Most of the class exercises had placed them in situations where they had felt uncomfortable and therefore they felt they could not cope. Objective learning and perceptions did not align in a sense. After the course many people decided not to continue studying Finnish as it was too difficult. If the teachers had perhaps prepared people for this style of study, altering people’s expectations of what they should be able to do, then more might have continued studying, eventually attaining a high level of skill.

In a world where most knowledge is searchable on a mobile device, confidence at finding and applying knowledge becomes more important. Applying knowledge in unfamiliar situations is a disconcerting experience though, and so developing confidence simultaneously with competence will become increasingly more important in training.


1Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4-58.