Learning Analytics, current challenges

 

We live in a stage where technology stands as one of the main drivers of social change. The advances in recent years have brought about profound changes in several areas of society and not a few authors talk about the dependency relationship between digital transformation and social welfare or economic progress. Without coming to value these opinions, what is certain is that technological development appears closely linked to the last generations (eg Y generation, Z generation) and recent social phenomena such as the so-called fourth industrial revolution and the knowledge society.

However, if we are talking about the engines of social change, education must be placed in the leading position. After all, education modulates the progress of the human being and the way in which he sees and understands the world. From the technological perspective, in Gradiant we know of this importance and we work to play our part in the educational improvement. Not surprisingly, the main objective of the organization is to improve our environment through technological innovation of ICT and, is there a better way to boost the growth of society than to work on improving teaching-learning processes?

 

«In particular, the eLearning work line mission is to facilitate learning and improve educational performance through the creation, use and management of technological processes and resources».

 

This is the first of a series of articles to analyze the relationship between technology and education, technology improvements in different educational domains (school, higher education, corporate), the latest trends in educational innovation or the situation of our environment in this framework. All this from the point of view of the bets and technology solutions we develop in Gradiant.

Learning Analytics

In the literature we can find several definitions of Learning Analytics [1]. Thinking about the approaches followed in our projects, and being pragmatic, perhaps the definition of Johnson, Adams, & Haywood 2011 [2] is one of the most successful:

 

“Learning analytics refers to the interpretation of a wide range of data produced by and gathered on behalf of students in order to assess academic progress, predict future performance, and spot potential issues. Data are collected from explicit student actions, such as completing assignments and taking exams, and from tacit actions, including online social interactions, extracurricular activities, posts on discussion forums and other activities that are not directly assessed as part of the student’s educational progress. The goal of learning analytics is to enable teachers and schools to tailor educational opportunities to each student’s level of need and ability. Learning analytics promises to harness the power of advances in data mining, interpretation, and modelling to improve understandings of teaching and learning, and to tailor education to individual students more effectively. Still in its early stages, learning analytics responds to calls for accountability on campuses across the country and leverages the vast amount of data produced by students in day-to-day academic activities.”

 

We are, therefore, talking about a process of measurement, collection, analysis and interpretation of data that allows us to take advantage of advances in several disciplines (e.g. information sciences, sociology, computer science, statistics, psychology, learning sciences, educational data mining) in order to make the most of the vast amount of data that is produced in educational activities and, from them, to adapt education to students effectively.

Learning Analytics vs. Educational Data Mining

Within the related or useful disciplines for Learning Analytics systems, Educational Data Mining stands out. This fact, together with its contexts of application, causes the mix of both concepts. Seeking to make a clear distinction between them, Bienkowski, Feng, & Means [3] establish:

 

“EDM develops methods and applies techniques from statistics, machine learning, and data mining to analyze data collected during teaching and learning. EDM tests learning theories and informs educational practice. Learning analytics applies techniques from information science, sociology, psychology, statistics, machine learning, and data mining to analyze data collected during education administration and services, teaching, and learning. Learning analytics creates applications that directly influence educational practice”

 

Besides, in a more schematic way, they define a series of differences:

• Learning Analytics does not emphasize reducing learning to individual components. Instead, it tries to understand complete systems and support administrators and educators in decision-making
• Learning Analytics generally does not address the development of new computational methods for data analysis but focuses on the application of known methods and models to solve issues that affect student learning
• Educational data mining focuses on generating automatic and / or system-generated responses while learning analytics seeks the participation of the human being in the process through adapting instructional content, intervention with at-risk students, and providing feedback
• In Learning Analytics social media analytics is used to discover relationships between students and social or «attention» metadata to determine their commitment

These differences seem to us important to highlight the influence that learning analytics solutions in the educational practice look for and the orientation towards the support to the actors involved in this practice. Albert Einstein said: «I fear the day when technology surpasses our humanity. The world will only have a generation of idiots”. Learning Analytics systems do not seek to replace the human being or «surpass our humanity». Learning Analytics support the development of an educational experience in order to be more effective and fit with key actors’ needs.

In Gradiant we propose our solutions around this objective, highlighting the SIMPLIFY project, an intelligent system with a psychometric motor for supporting educators with information from digital contents; or Netex-Gradiant Joint Unit «Smart Lifelong Learning for the industry of the future and knowledge society», more focused on industry and companies goals:

• Research and development of new metacognitive models for learning measurement
• Research and development of training solutions to make easier lifelong learning and competency learning
• Research and development of adaptive instruction mechanisms

Interest in Learning Analytics

At present, research in Learning Analytics is in great health. Perhaps prompted by its inclusion in the 2011 Horizon report [4] as one of the emerging technologies in education, in recent years it has seen a remarkable increase in its study. Thus, in one way or another, it has continued to be reflected in all successive Horizon reports:

• In Higher Education in 2012 and 2013 editions and in K-12 (school education) 2013 and 2014 editions Learning Analytics is referenced as a technology to be closely followed and it is given two or three years to be incorporated into the market (medium-term perspective)
• In Higher Education 2015, it is already pointed out as a trend that accelerates the adoption of new technologies in the medium term (advances in the next three to five years)
• The K-12 2015 edition mentions the importance of the progress in Learning Analytics to advance the trend of using blended-learning or blended learning scenarios (online work controlled by the student combined with practical work in face-to-face classes)
• Higher Education 2016 reappears like a technology of special importance but with a market time adoption of one year or less
• In K-12 2017 edition is a tendency for the adoption of new technologies in the classroom for the next three to five years

 

“The NMC Horizon Project is a global ongoing research initiative that explores the trends, challenges, and technology developments likely to have an impact on teaching, learning, and creative inquiry. Founded in 2002, it uniquely provides a cross-sector view of disruptors in higher education, K-12, academic & research libraries, and museums, with NMC Horizon Report editions that focus on each”

 

In addition, in last five years successful initiatives related to the dissemination and research have been launched, such as:

• International Conference on Learning Analytics & Knowledge (LAK), which will hold its eighth edition in 2018
• SoLAR Projects (SoLAR), Learning Analytics Community Exchange (LACE) or Predictive Analytics Reporting Framework (PAR)
• Learning Analytics Summer Institutes (LASI), organized by SoLAR (Society for Learning Analytics Research)
• Journal of Learning Analytics (JLA)
• Seminars on Learning Analytics from University of Michigan (SLAM)

More informally we can also weigh interest by using Google’s tool to explore the most popular search topics on your browser. The following image shows the growth in «Learning Analytics» searches as a topic and as a search term between 2012 and 2017.

Evolution of Learning Analytics (theme) & learning analytics (search term) searches. 2012-2017. Google Trends. October 2017

Challenges in Learning Analytics

Despite being an emerging research field, as we have already mentioned, important advances have been made in recent years in this area. However, like any change derived from technology development, Learning Analytics must still face certain important challenges, which we highlight [5]:

1. Building robust connections to the learning sciences: The process of understanding and optimizing learning requires a thorough understanding of how it is produced, how it can be supported, and how important factors such as identity or reputation are.
2. Development of methods to work with different data sets to optimize learning environments: looking for lifelong learning in more open and informal environments, students need support to take their learning away from typical LMS. This situation requires a shift to more complex and heterogeneous data sets that may include mobile, biometric or affective data.
3. Focus on the student: it is necessary to focus more on analysis related to the needs of the student than on those based on the needs of the institutions. Thus, it is crucial to extend learning success criteria beyond grades and persistence to include variables such as motivation, trust, enjoyment, and achievement of goals.
4. Development and application of a clear set of ethical standards: the collection of student data and its subsequent processing must be consented to and properly justified. At present, there are no standard procedures for reporting on data collected, consenting or limiting its use. For this reason it is necessary to define clearly and concisely the rights of the students and the responsibilities of the entities that use Learning Analytics processes.

To a greater or lesser extent, Gradiant have been working on all of them in recent years:

• We have researched connection with the science of learning to propose solutions that help the understanding of the learning process, e.g. detailing educational experiences, content usage analysis, catching common process errors (using Frequent Pattern Mining), identifying core users (using Social Network Analysis), etc.
• In the methods to work with different datasets, we can highlight, among others, the implementation of (1) ETL (Extract-Transform-Load) tools to support different types of sources; (2) event transformation tools to a common format (xAPI); (3) client (JS) libraries to capture events in content or client platforms; and (4) browser extensions to capture events on any web source.
• Focusing on the student, we have developed algorithms that can characterize her based on learning dimensions (e.g. enpowerment, social, performance, effort, engagement) and, around them, to propose activity recommendations, group work recommendations and / or intervention alerts either for the direct use of the student, or as support to the teacher in his task of defining the strategy of action in situations of risk (eg failure of school, reduction of intensity in the educational process). In addition, we have always spoiled the representation of such information and the adaptation of it to its recipients.
• Without having developed a set of ethical standards or a policy to that effect, we have put the focus on documenting what kind of data are captured and what is the use that is made of them in each case. Of course, the legal obligations regarding the processing of personal data have also been respected.

Closely linked to these, we can point out certain technological needs or challenges:

1. Interoperability: if we previously commented on the obligation to consider and interpret heterogeneous and ubiquitous data sources, from a more technical point of view we can point out the need to build interoperable systems to capture information from different sources.
2. Big Data Challenge: Both the consideration of heterogeneous datasets and the volume of this type of information itself requires an improvement in the computational efficiency of the algorithms used in its collection and processing.
3. Information collection and analysis strategy: the data that is maintained and processed in the system must be correctly aligned with the questions to which it intends to respond with an appropriate modelling.
4. Security: in line with the need to preserve the privacy of students and regulate the ownership and use of the data generated in their learning process, it is necessary to establish mechanisms that ensure the exchange of information in the different learning environments and their storage.

We can also highlight some examples that reflect our approach for the different challenges:

• In terms of interoperability, collection and analysis, we have focused our solutions around the xAPI specification, learning standard with strategies to structure and store the interactions in a learning experience. On the one hand, by its own condition of standard, it allows us to face with guarantees the construction of heterogeneous systems that capture data from diverse sources. On the other hand, it gives us some guidelines for storing and retrieving the data in a unified form and, from here, to align the proposals of the analysis modules based on this form of storage.
• To address the challenge of Big Data, from a pragmatic point of view, for the construction of the modules of analysis we have reviewed and used different frameworks with support to the processing of large amounts of data.
• Finally, with regard to data security, in some of our projects we have put forward anonymising and de-identification strategies to preserve the privacy of the actors involved in educational experiences. In addition, as previously mentioned, the current legislation on data protection in the areas of application of the projects has been taken into account.

 

---

Author: Agustín Cañas, Head of eLearning in Services & Applications Department at Gradiant

---

 

References

[1] Chatti, M., Dyckhoff, U., Schroeder, U., & Thüs, H. (2012). A Reference Model for Learning Analytics. International Journal of Technology Enhanced Learning (IJTEL) – Special Issue on “State-of-the-Art in TEL”, 318-331.

[2] Johnson, L., Adams, S., & Haywood, K. (2011). The NMC Horizon Report: 2011 K-12 Edition. Austin, Texas: The New Media Consortium.

[3] Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics: An Issue Brief. Washington, D. C.: U.S. Department of Education.

[4] Horizon. (s.f.). NMC Horizon reports. Recuperado el 06 de 10 de 2017, de NMC website: https://www.nmc.org/publication-type/horizon-report/

[5] Ferguson, R. (2012). Learning Analytics: drivers, developments, and challenges. International Journal of Technology Enhanced Learning, 304-317.

 

SIMPLIFY project

SIMPLIFY es un proyecto de I+D financiado por la Secretaría de Estado de Investigación, Desarrollo e Innovación, con expediente nº RTC-2015-4329-7 a través de la convocatoria Retos-Colaboración del Programa Estatal de Investigación, Desarrollo e innovación orientada a los retos de la Sociedad, en el marco del Plan Estatal de Investigación científica y técnica y de innovación 2013-2016.

 

UMI Netex-Gradiant

Subvencionado por la Agencia Gallega de Innovación mediante el Programa Unidades Mixtas de Investigación 2016, cofinanciado por el Fondo Europeo de Desarrollo Regional (Feder) en el marco del Programa Operativo Feder Galicia 2014-2020. Número de expediente IN853A- 2016/03, Resolución de la Axencia Galega de Innovación el 17 de xunio de 2016.

Apoyado por la Consellería de Economía, Empleo e Industria de la Xunta de Galicia.