Holland, a Country of Contrasts and Professionals

Simulation games involve the recreation of various situations within the framework of a specially designed environment. There are many different types of games, but most come in the form of computer simulations used in the fields of medicine, flight training, business, and politics. Simulation games are unique in that they are used mostly in practice; that is, they are constructed and only then is one able to see whether these constructions work or not. To learn how to take a scientific approach towards this topic, Natalia Isaeva, Junior Research Fellow and Doctoral student at the Institute of Education went to the Netherlands to study at Maastricht University.

Background

One of the advantages of HSE’s full-time advanced doctoral programme is that it offers you the opportunity to go on an international internship. To be honest, I was initially somewhat sceptical. It didn’t seem like I would have enough time as an intern to really do something great or truly learn about the subject matter. I also doubted that it would help me with writing my dissertation. This is why I left for the internship mostly hoping to get the experience of living in another country, as well as to meet professionals in my field who might be able to help me in the future.

One of the advantages of HSE’s full-time advanced doctoral programme is that it offers you the opportunity to go on an international internship

My dissertation research is about simulation game technologies in evaluating educational programmes. The problem with these games is that in Russia, it is mostly practitioners, such as training and consulting agencies, as well as corporate universities, who are making developments in this area. HR specialists actively use simulations to teach and assess personnel as well. But at the same time, there are almost no reliable theoretical studies that look at the process of simulation development and validation from an academic or evidence-based perspective.

How ‘They’ Do Things

Two years ago, Professor Jeroen van Merrienboer came to the HSE Institute of Education. He is well known for being one of the creators of the 4C-ID approach to education design, and he is a researcher in teaching and learning, as well as an international expert and consultant in modern educational technologies. Professor Merrienboer spoke about his experience with designing and using simulations at the 5th International Conference on Managing Differentiation in Rapidly Changing Higher Education Systems: Challenges and Opportunities. I was really impressed by his presentation and had almost no doubt that my aim would be to go to Maastricht University for my internship.

This is how I found myself at the Maastricht University Department of Educational Development & Research in the small medieval town of Maastricht. This is one of the best institutions in the world that works in educational programme design. In my opinion, there are two reasons behind the department’s success. Firstly, the Department of Educational Development & Research works on developments in the field of medical training, and as everyone knows, it is particularly in medicine and the military that the majority of innovations are made. Secondly, they are very evidence-based and can turn what we might understand to be an accidental effect into an entirely new teaching technology.

For example, when we – that is students – prepare for an exam, we concentrate on the rumors we’ve heard about the instructor. If we know that you don’t really have to study much to pass a certain professor’s exam, then we aren’t likely to try very hard. But at Maastricht, they conducted an entire experiment on the matter. Before an exam, they showed one group of students a video of former students saying that in order to pass the exam, you just need to learn a set of so-called ‘protocols’ (step-by-step instructions for doctors). As for the second group of students, they were shown a video of the instructor saying that it goes without saying that students must know the protocol, but also that the instructor would evaluate students on critical thinking and their ability to think outside the box. As you might have guessed, the first group scored significantly lower on the exam than the second group. The results of the experiment were so impressive that they made instructor videos an essential part of the programme.

Academics

My programme at Maastricht turned out to be very intensive. Professor Merrienboer compiled a busy schedule of meetings with him and his colleagues for me and by the end this list had almost doubled in length. Everyone tried to suggest some of their colleagues who might be useful for me, and one meeting was oftentimes not enough to discuss everything with some of them.

It turns out that one of the main components in training doctors is teaching them to be leaders and evaluating their leadership skills through simulations

My first meetings took place in the Department of Educational Development & Research. Before going on the trip, I spent a lot of time reading Professor Merrienboer’s articles and books, and I tried to get a good grasp of his unique model for structuring training programmes. Despite the fact that his books use very clear language, I wasn’t able to fully grasp all of the concepts. His model ended up being too complex, and it became clear that I was lacking certain necessary knowledge. But at our first meeting, the professor and I talked, and he laid everything out for me really well by telling me a wonderful story of how a training programme was developed for hair stylists. ‘Just imagine,’ he said, ‘I spent nearly two months in the salon and listened to the types of demands and requests that people came to the stylist with. A man with short hair wants his hair trimmed with clippers, one woman wants her hair dyed one colour, another wants highlights, another dyed and cut, and yet another woman with curly hair wants her hair dyed, styled, straightened, and laminated…’

When all of these types of requests started repeating themselves (it’s worth noting that nearly 1,500 requests were collected), they stopped observing and began sorting the tasks into three categories ranging from simple to the most complex ones requiring the stylist to employ several competencies at the same time. After this, special assignments were developed that involve varying levels of instructor participation. First the instructor would demonstrate several simpler assignments, such as using the trimmer to cut hair, and then the students would be given the opportunity to try it themselves under instructor supervision. In the end, the students had to be able to demonstrate their abilities independently without the instructor’s help.

Once the students had mastered simpler skills, they moved on to more complex tasks like dying and cutting at the same time. The same system was used, however, with students going from full instructor support to complete independence. This all culminates in a final assessment that the student completes only after successfully making it through each step of the training process. Among other things, this model allows students of varying levels to begin the training programme from different stages depending on their starting knowledge. As you can see, the model is a competency-focused approach, which is unique for us. Professional assignments lie at the model’s core, and these assignments are things that a professional would actually encounter in their work.

A similar design was used when training doctors in the Netherlands. That is, the programme focused on real professional problems, and the results of this type of training changed the healthcare system as a whole. Everyone has probably encountered a situation where they are first sent to one doctor and then to another and then another after that… Each doctor might prescribe you a different medicine, and the patient goes from one room to the next trying to piece together all of the advice they’ve received. In the Netherlands though, all of the doctors meet the patient in the same room, but instead of looking at separate problems like sore eyes, a sore throat, the heart, etc., they look at all symptoms and ailments as a whole.

From talking to Jan-Joost Rethans, the Director of Maastricht University’s Medical Education Programmes Department, I found out that the field has undergone some pretty radical changes over the last 20 years. Professor Rethans carried out an interesting experiment for his dissertation. In a simulation, the same group of medical students worked with patients who were specially trained and understood that this was a training game. The next time, there were patients who were part of the experiment, but they didn’t know it. It was anticipated that the students with the best results in the training exercise should have clearly demonstrated this in a ‘conditionally real interaction.’ But the hypothesis was not proven, and the results ended up being completely unpredictable.

This experiment served as the beginning of a large-scale project at Maastricht University on fine-tuning these types of educational programmes so that students can turn the results they achieve during training exercises into real-life success in their job.

I had the opportunity to discuss my research with 15 experts. As a result of my trip to Maastricht, I had a clear idea of what and how I should do further, and how to turn the applied ability to design games into a science-based professional activity

Professor Merrienboer and Professor Rethans talked about the role simulations play in contemporary educational programmes, as well as what the potential of these simulations are. But the entire time, I was troubled by one thing – all of the examples they gave were connected with professions where the results could more or less be seen and measured objectively. Was the haircut straight, was an x-ray analysed correctly, etc. What about things that are harder to measure objectively, like leadership skills? The answer to this question was found unexpectedly in medicine by Uli Strauch, Chief Anesthesiologist-Intensivist at the Maastricht University hospital.

It turns out that one of the main components in training doctors is teaching them to be leaders and evaluating their leadership skills through simulations. During an operation, for example, especially if the operation doesn’t go according to plan, it is critical that the doctor be able to work on a team and ensure that the work of the entire group of specialists is organised smoothly. I had the pleasure of not only listening to some interesting stories, but also sitting in on one of these kinds of simulations at the Simulation Centre.

The Centre has fully reproduced operating rooms with all of the necessary equipment, as well as special mannequins and robots capable of imitating human activity. There is a separate room for instructors where they are able to change various parameters to create special crisis situations. They are in turn managed by a single instructor who sits in the operating room and specifically tries to throw wrenches into the work of the team. What I valued most was the advice I got from Professor Strauch on developing and validating assessment instrumentation based on the simulation model, including the development of a conceptual map, scorecards, and a simulation scenario.

So, I had the opportunity to discuss my research with 15 experts. As a result of my trip to Maastricht, I had a clear idea of what and how I should do further, and how to turn the applied ability to design games into a science-based professional activity.

In addition to that, I managed to visit Amsterdam, to see Rembrandt’s ‘The Night Watch’, and to participate in a traditional Dutch carnival, the main events of which take place in Maastricht. I also enjoyed meeting the family of Jo Coenen, a renowned Dutch architect, and they revealed to me some secrets of Dutch life.