Neural Network for Assessing English Language Proficiency Developed at HSE University
The AI Lingua Neural Network has been collaboratively developed by the HSE University’s AI Research Centre, School of Foreign Languages, and online campus. The model has been trained on thousands of expert assessments of both oral and written texts. The system evaluates an individual's ability to communicate in English verbally and in writing.
Individuals interested in assessing their speaking and writing proficiency are invited to complete a series of tasks, including writing an essay, answering interview questions, and composing a monologue. Completing the tasks takes no more than 60 minutes, and the test is accessible on any digital device. Results, along with brief comments, are available within 30 minutes.
This format enables a representative assessment of the range of skills most commonly used in oral and written communication. When evaluating responses, the neural network considers over 45 parameters developed by experts at the HSE School of Foreign Languages. The result is determined through a comprehensive assessment of lexical, grammatical, and phonetic/stylistic aspects of speech, as well as characteristics such as coherence, logical presentation of thoughts, and skills in description, reasoning, and narration.
Based on the results of the tasks, a final score is calculated, and a language proficiency level is assigned, ranging from elementary to upper-intermediate. The proficiency level scale developed at the HSE School of Foreign Languages and integrated into the neural network aligns with all international language testing systems.
'An AI model capable of evaluating oral and written tasks with open-ended responses remains a rarity in language education. It can be described as a cutting-edge category of AI-based assessment systems, as the neural network has been trained to identify a variety of complex indicators of speech behaviour to determine candidates' levels of communicative competence,' according to Ekaterina Kolesnikova, Head of the HSE School of Foreign Languages.
The model has been trained on thousands of expert assessments of oral and written texts produced by actual participants in academic communication. The process of ongoing training and refinement of the model continues. Currently, the neural network is in its beta stage, undergoing testing and calibration for further integration into the process of assessing student work as part of independent English proficiency examination. It is planned for the neural network to function as one of the experts.
'The project to develop AI Lingua is an example of adapting large foundational models for a specific applied field—education. We have come all the way from testing various scientific hypotheses to implementing models in the final product, which is available as an online service for self-assessing English proficiency. Our solution is distinguished by its ability to simultaneously evaluate not only text responses but also audio speech, producing a language proficiency level based on a comprehensive set of criteria. To achieve this, we studied a substantial number of open-source solutions and subsequently conducted additional training on works analysed by professional linguists,' explains Alexey Masyutin, Director of the AI Centre.
The HSE University's team also notes that there are plans to develop models for other foreign languages. This will assist foreign language learners in conducting self-assessments of their knowledge, receiving timely feedback with the necessary level of detail, and tracking their progress in language proficiency development.
'The integration of AI solutions into the language skills assessment process fundamentally transforms the approach to language learning, making it more convenient and user-friendly. We are developing a tool that not only records the current level of knowledge but also dynamically tracks progress, offering everyone the opportunity for regular and objective self-assessment,' emphasises Sergey Roshchin, Vice Rector of HSE University.
The beta version of the English proficiency self-assessment tool is available for free with a promo code sent via email upon registration.
See also:
A New Tool Designed to Assess AI Ethics in Medicine Developed at HSE University
A team of researchers at the HSE AI Research Centre has created an index to evaluate the ethical standards of artificial intelligence (AI) systems used in medicine. This tool is designed to minimise potential risks and promote safer development and implementation of AI technologies in medical practice.
HSE Researchers Develop Novel Approach to Evaluating AI Applications in Education
Researchers at HSE University have proposed a novel approach to assessing AI's competency in educational settings. The approach is grounded in psychometric principles and has been empirically tested using the GPT-4 model. This marks the first step in evaluating the true readiness of generative models to serve as assistants for teachers or students. The results have been published in arXiv.
‘Philosophy Is Thinking Outside the Box’
In October 2024, Louis Vervoort, Associate Professor at the School of Philosophy and Cultural Studies of the Faculty of Humanities presented his report ‘Gettier's Problem and Quine's Epistemic Holism: A Unified Account’ at the Formal Philosophy seminar, which covered one of the basic problems of contemporary epistemology. What are the limitations of physics as a science? What are the dangers of AI? How to survive the Russian cold? Louis Vervoort discussed these and many other questions in his interview with the HSE News Service.
HSE Scientists Propose AI-Driven Solutions for Medical Applications
Artificial intelligence will not replace medical professionals but can serve as an excellent assistant to them. Healthcare requires advanced technologies capable of rapidly analysing and monitoring patients' conditions. HSE scientists have integrated AI in preoperative planning and postoperative outcome evaluation for spinal surgery and developed an automated intelligent system to assess the biomechanics of the arms and legs.
HSE University and Sber Researchers to Make AI More Empathetic
Researchers at the HSE AI Research Centre and Sber AI Lab have developed a special system that, using large language models, will make artificial intelligence (AI) more emotional when communicating with a person. Multi-agent models, which are gaining popularity, will be engaged in the synthesis of AI emotions. The article on this conducted research was published as part of the International Joint Conference on Artificial Intelligence (IJCAI) 2024.
HSE University and Yandex to Host International AI Olympiad for Students
The HSE Faculty of Computer Science and Yandex Education are launching their first joint AI competition, Artificial Intelligence and Data Analysis Olympiad (AIDAO), for students from around the world. Participants will tackle challenging tasks in science and industry and interact with experts from HSE and Yandex. The winners will receive cash prizes.
Winners of the International Olympiad in Artificial Intelligence Admitted to HSE University
In mid-August, Bulgaria hosted the finals of the first International Olympiad in Artificial Intelligence (IOAI) among high school students. The Russian team demonstrated excellent results, winning gold medals in the scientific round, silver medals in the practical round, and coming first in both rounds overall. This year two members of the Russian team were accepted into the programmes of the HSE Faculty of Computer Science.
Artificial and Augmented Intelligence: Connecting Business, Education and Science
The history of AI research in Nizhny Novgorod dates back to the 1960s and 1970s. Today, AI technologies, from voice assistants and smart home systems to digital twin creation and genome sequencing, are revolutionising our life. Natalia Aseeva, Dean of the Faculty of Informatics, Mathematics and Computer Science at HSE Campus in Nizhny Novgorod, discusses how the advancement of AI connects science, business, and education.
HSE University Leads the AI Alliance Ranking
The AI Alliance Russia has released a new ranking of Russian universities based on the quality of education in the field of AI. Similar to last year, HSE University has joined the leaders in A+ group alongside MIPT and ITMO. A total of 207 universities from 69 Russian regions participated in the ranking. In 2024, over 35,000 students were enrolled in AI-related programmes at these universities.
Reinforcement Learning Enhances Performance of Generative Flow Networks
Scientists at the AI Research Centre and the AI and Digital Science Institute of the HSE Faculty of Computer Science applied classical reinforcement learning algorithms to train generative flow networks (GFlowNets). This enabled significant performance improvements in GFlowNets, which have been employed for three years in tackling the most complex scientific challenges at modelling, hypothesis generation, and experimental design stages. The results of their work achieved a top 5% ranking among publications at the International Conference on Artificial Intelligence and Statistics AISTATS, held on May 2-4, 2024, in Valencia, Spain.