Artificial Intelligence and User Experience in reciprocity: Contributions and state of the art

• Content-based retrieval is a technique used in web search to retrieve information based on the similarity of content. It involves analysing the content of a web page, such as the text, images, videos, and audio, to determine its relevance to a user’s query and provide relevant search results. For example, content-based retrieval has been used in an intelligent mobile application to retrieve music pieces from digital music libraries [75]. Another example involves using agents for feature extraction in the context of content-based image retrieval for medical applications [76].

• Collaborative filtering uses data from other users with similar search behaviour to recommend search results. For example, a model that represents a user’s needs and its search context is based on content and collaborative personalisation, implicit and explicit feedback, and contextual search [77].

• Hybrid filtering in web search is a method of combining two or more different types of filtering techniques to improve the accuracy and relevance of search results. This approach can be used to address some of the limitations of individual filtering techniques and provide a more comprehensive search experience. For example, a hybrid filtering mechanism is proposed to eliminate irrelevant or less relevant results for personalised mobile search, which combines content-based filtering and collaborative filtering; the former filters the results according to the mobile user’s feature model generated from the user’s query history, and the latter filters the results using the user’s social network, which is constructed from the user’s communication history [78].

• Machine Learning: Machine learning allows search engines to learn from user behaviour, such as clicks and dwell time and adapt search results based on individual preferences. For example, if a user consistently clicks on results from certain websites, the search engine may prioritise those results in future searches. Machine Learning is used to improve the accuracy of search results. This technique is also used to identify patterns and relationships in large datasets. An example of this kind of approach is the use of machine learning algorithms based on experts’ knowledge to classify web pages into three predefined classes according to the degree of content adjustment to the search engine optimisation (SEO) recommendations [79].

• Deep Learning: Deep learning is a subfield of machine learning that involves training artificial neural networks to learn from large datasets. The goal of deep learning is to enable computers to learn and recognise patterns, classify information, and make decisions in a way that is similar to how the human brain operates. Deep learning is used in web searching to improve the accuracy of image and voice-based search. Neural networks in web search are used in web search engines, ranking algorithms citation analysis [80]. For example, in [81] a deep learning model of Convolutional Neural Network (CNN) is used in big web search log data mining to learn the semantic representation of a document to improve the efficiency and effectiveness of information retrieval.

• Natural Language Processing (NLP): NLP is used in web searching to understand the user’s queries in natural language and match them with relevant web pages. NLP techniques include text segmentation, entity recognition, and sentiment analysis. For example, [82] presents a method for translating web search queries into natural language questions. The authors propose a system that uses natural language processing (NLP) techniques to transform a web search query into a question that can be answered using a knowledge base.

• Semantic Search: Semantic search uses machine learning and NLP techniques to understand the meaning behind user queries and deliver more accurate search results. This approach involves analysing the relationship between words and phrases and identifying entities in the text. Semantic search and semantic web are two related concepts in the field of information retrieval and web technologies. Instead of relying solely on keywords, semantic search algorithms attempt to understand the meaning of the query and the content being searched, and to provide more relevant results based on that understanding. Semantic search can be seen as an application of the principles of the semantic web to the problem of search. By using semantic technologies such as ontologies, taxonomies, and natural language processing, semantic search can provide more intelligent and accurate search results that better match the user’s intent. Semantic web technology and semantic search are emerging areas of research that have the potential to transform the way we interact with information on the internet. According to [83], semantic web technology enables the creation of machine-readable data that can be easily interpreted by computers, leading to more intelligent and effective search results. In [84] the authors further explain how query technologies can be developed to enable semantic search in the context of the semantic sensor web, where sensors generate large amounts of data that require sophisticated search and analysis tools. Finally, in [85], the authors provide a diachronic study of publications comparing the terms “semantic web” and “web of data” over time and identifying the evolution of these concepts. Together, these works highlight the potential of semantic web technology and semantic search to improve information retrieval and analysis on the web.

• Knowledge Graphs: Knowledge graphs are used to organise and connect information in a way that makes it easier to find and understand. These graphs are used to identify entities, relationships, and concepts related to the user’s query. For example, in [86] methods are proposed for extracting triples from Wikipedia’s HTML tables using a reference knowledge graph.

• Collaborative filtering: Collaborative filtering (CF) is a well-known recommendation method that estimates missing ratings by employing a set of similar users to the active user [89]. Collaborative filtering can be further divided into two sub-types: user-based collaborative filtering and item-based collaborative filtering.

• Content-based filtering: This approach recommends items to users based on the user’s preferences and characteristics of the items. Content-based filtering uses attributes of the items to recommend similar items to the user. For example, in [93] the proposed content-based recommendation algorithm quantifies the suitability of a job seeker for a job position in a more flexible way, using a structured form of the job and the candidate’s profile, produced from a content analysis of the unstructured form of the job description and the candidate’s CV.

• Hybrid recommender systems: This approach combines collaborative filtering, which uses data on user behaviour to recommend content, with content-based filtering, which uses characteristics of the content itself to make recommendations. For example, in [94], a two-level cascade scheme is presented where a hybrid of one-class classification and collaborative filtering is used to decompose the recommendation problem, with the first level selecting desirable items for each user using a one-class classification scheme trained only with positive examples, and the second level applying collaborative filtering to assign rating degrees to the selected items.

• Machine-learning based recommender systems: This approach employs machine learning algorithms as the main underlying reasoning mechanism. For example, in [95] an artificial immune algorithm has been used in a system for music recommendation. In [96] the recommender system uses “musical genre classification” and “personality diagnosis” as inputs to make recommendations, by employing machine learning to classify musical genres and diagnose personality traits.

• Deep learning-based recommender systems: This approach uses neural networks and deep learning techniques to model complex relationships between users and items to make personalised recommendations. For example, an approach of biomarker-based deep learning is used for personalised nutrition recommendations [97].

• Fuzzy-based recommender systems: The presence of several works focuses on managing items’ attributes with fuzzy techniques, strengthening the development of uncertainty aware content-based recommender systems [98].

• Context-aware recommendation: This approach takes into account additional contextual information, such as the time of day, user location, or weather conditions, to make more relevant recommendations. For example, in [99] environmental and social considerations are incorporated into the portfolio optimisation process and in [100] a tourism recommendation system is context-aware and is also based on a fuzzy ontology.

• Constraint-based recommender systems: These systems constitute a type of recommendation system that uses a set of constraints or rules to generate personalised recommendations for users. These systems are often used in situations where there are specific requirements or constraints that must be considered in the recommendation process, such as limited availability of products or services, specific user preferences or characteristics, or other business or operational considerations. For example, in [101] a constraint-based approach has been used in a job recommender system.

• Knowledge-based recommender systems. These systems rely on explicit knowledge about a user’s preferences, needs, and characteristics, as well as knowledge about the product or service being recommended. This knowledge is usually represented as rules or logic statements that are developed by human experts in the domain. Knowledge-based recommendation systems typically do not rely on machine learning algorithms to generate recommendations. For example, in [102] a knowledge-based TV-shopping application is described that provides recommendations based on users’ current input and historical profile from past interactions.

• NLP, voice synthesisers, and animations in recommender systems. These approaches can enhance the user experience and improve the effectiveness of the recommendations by providing more personalised, natural, and engaging interactions. For example, in [103] the incorporation of animated lifelike agents into adaptive recommender systems has been used to help users buy products in an e-shop in a more effective way.

• Matrix factorisation: This is a type of collaborative filtering that uses matrix factorisation to extract latent factors that explain the user-item interaction. The technique involves decomposing a matrix of user-item interactions into lower-dimensional matrices, and then using those matrices to predict missing entries and recommend new items. For example, in [104], the authors propose a lightweight matrix factorisation algorithm that reduces the number of parameters required for training, while maintaining the accuracy of the recommendations.

• Association rule mining: This technique involves finding patterns in large datasets to make recommendations. Association rule mining helps to find out the association between different attributes of the system. For example, if users of an e-bookshop who buy a book A also tend to buy a book B, then the system might recommend book B to users who have purchased book A by saying something like Users who bought the book A were also interested in buying book B. An example of an approach uses association rule mining to uncover hidden associations among metadata values and to represent them in the form of association rules. These rules are then used to present users with real-time recommendations when authoring metadata [105].

• Reinforcement learning: Reinforcement learning (RL) is a machine learning approach that learns by interacting with an environment to achieve a specific goal. The authors in [106] point out that unlike traditional recommendation methods, including collaborative filtering and content-based filtering, RL is able to handle the sequential, dynamic user-system interaction and to take into account the long-term user engagement and that although the idea of using RL for recommendation is not new, it used to have scalability problems in the past.

• Graph-based recommendation: This approach uses graph algorithms to model relationships between users, items, and other entities in the system. The graph structure can be used to generate personalised recommendations and to identify communities of users with similar preferences. For example, [107] presents a comprehensive framework for smoothing embeddings in sequential recommendation models. The approach involves constructing a hybrid item graph that combines sequential item relations based on user-item interactions with semantic item relations based on item attributes.

• Passive help systems require that the user requests help explicitly from the system. For example, UC is a passive help system for Unix users that can answer to users’ questions about Unix commands, using Natural Language [117].

• Active help systems should guide and advise a user like a knowledgeable colleague or assistant. For example, AI has been used to generate hypotheses about users while they are interacting with the Unix operating system so that it may provide spontaneous assistance [118, 119].

• Goal recognition: Goal Recognition refers to the ability to recognise the intent of an acting agent through a sequence of observed actions in an environment. While research on this problem gathered momentum as an offshoot of plan recognition, recent research has established it as a major subject of research on its own, leading to numerous new approaches that both expand the expressivity of domains in which to perform goal recognition and substantial advances to the state-of-the-art on established domain types [120].

• Plan recognition: Plan recognition in intelligent help systems and virtual assistants refers to the ability of these AI systems to infer the user’s plans or goals based on their actions and behaviour. Plan Recognition is a closely related problem to goal recognition, where the premise of the observer is to recognise how the acting agent intends to reach its goal [121]. This involves analysing and interpreting the user’s interactions with the system, such as their voice commands, search queries, and other input, to identify their intentions and anticipate their future actions. Plan recognition is a critical component of intelligent help systems and virtual assistants as it enables the system to provide more personalised and context-aware responses and recommendations, improving the overall user experience. Various techniques and algorithms are employed to implement plan recognition in intelligent help systems and virtual assistants. For example, a theoretical treatment of plan recognition is presented in [122] where the use of an interval-based logic of time to describe actions, atomic plans, non-atomic plans, action execution, and simple plan recognition is described. In another example, plan-based techniques for automated concept recovery are presented in [123]. The authors of [124] note that a recogniser is deemed superior to prior works if it can recognize plans faster or in more intricate settings, although this often results in a trade-off between speed and richness or computation time.

• Plan generation: Plan generation in intelligent help systems and virtual assistants refers to the ability of these AI systems to generate plans or sequences of actions that help the user achieve their goals or complete their tasks. This requires analysing the user’s input and context to determine the appropriate course of action and generating a plan that is most likely to achieve the desired outcome. Plan generation is a necessary component of intelligent help systems and virtual assistants as it enables the system to provide more proactive and effective assistance to the user. Various techniques and algorithms, such as rule-based systems, decision trees, and reinforcement learning, are employed to implement plan generation in these systems. For example, in [125], to achieve the design target of responding accurately to complex users’ queries that involve the interconnection of multiple commands with various options, the system requires a detailed representation of dynamic knowledge about command behaviour and a planning mechanism that can integrate the knowledge into a cohesive solution.

• Error diagnosis in intelligent help systems and virtual assistants concerns the ability of these AI systems to identify, diagnose and help in the rectification of human errors that the user may have made in the context of the problem domain. Rasmussen points out that human errors are not events for which objective data can be collected, instead they should be considered occurrences of man-task mismatches which can only be characterised by a multifaceted description [126]. An empowered AI-system performs error diagnosis by monitoring and analysing the user’s input as well as contextual information and concerning the domain of use to determine if there are any errors or inconsistencies, and in case there are, identifying the root cause of the problem, so as to provide advice on the rectification of errors, which the user may have or have not been aware of. For example, in the domain of a compiler architecture for domain-specific type error diagnosis of programming languages, the authors define error contexts as a way to control the order in which solving and blaming proceeds [127]. Error diagnosis is an important AI-based operation in intelligent help systems and virtual assistants as it enables the system to identify a problem and offer helpful guidance to the user in resolving the diagnosed problem. As an AI method, error diagnosis involves using AI algorithms and techniques to analyse data and identify patterns that can help detect errors or anomalies in a system. This can be applied in various domains, such as healthcare, finance, and education, to improve the efficiency and accuracy of processes. Various techniques and algorithms, such as rule-based systems, cognitive theories, decision trees, and machine learning, are employed to implement error diagnosis in these systems. For example, in [128] pronunciation error detection and diagnosis is achieved through cross-lingual transfer learning of non-native acoustic modeling.

• Knowledge Representation and Reasoning: This AI technique involves representing knowledge in a way that can be easily processed and reasoned about by the system. By using knowledge graphs, ontologies, and other techniques, intelligent help systems and virtual assistants can store and retrieve information in a way that allows for more efficient and effective assistance. For example, in [129] an intelligent help system sets the design target to advise users on alternative plans that would achieve the same goal more efficiently and along the way, a representation of plans, subplans, goals, actions, properties and time intervals is developed to recognise plans and then to advise the user.

• Natural Language Processing (NLP): This AI technology allows intelligent help systems and virtual assistants to understand and respond to natural language input from users. NLP can be used to create chatbots that can help users with their queries and problems as has been the case in intelligent help systems [130] and in more recent virtual assistants [131]. Chin [132] argues that intelligent help systems should not merely react to user input but actively offer information, correct misconceptions, and reject unethical requests, and to do so, they must function as intelligent agents utilizing Natural Language Processing and Plan suggestion situations.

• Machine Learning (ML): This type of AI can help intelligent help systems and virtual assistants learn from user interactions and improve their responses over time. ML algorithms can be used to personalise the user experience and provide more relevant help and recommendations. Supervised learning algorithms can be trained on labeled data to classify instances as normal or faulty, while unsupervised learning algorithms can be used to identify anomalies or outliers in the data. For example, Microsoft Cortana [110] uses machine learning to personalise user experiences, understand user queries, and provide relevant information and recommendations.

• Neural Networks: Neural networks can be used to learn the relationship between the observed actions and the underlying goals or intentions from a set of training examples. Once trained, neural networks can be used to infer the most likely goals or intentions based on the observed actions. Neural networks can also be used to diagnose errors or faults in complex systems, such as manufacturing processes or healthcare systems. By analysing sensor data and other inputs, neural networks can learn to predict when errors or faults are likely to occur.

• Cognitive theories are important for understanding how people learn, solve problems, and make decisions, and can be applied to the design of intelligent help systems and virtual assistants. These cognitive theories can inform the design of intelligent help systems and virtual assistants by guiding the selection of appropriate instructional strategies, the organisation and presentation of information, and the design of user interactions. For example, in [136] a cognitive theory has been employed to achieve automatic error diagnosis for users interacting with an operating system and offer spontaneous assistance.

• Rule-based Systems: Rule-based systems can be used to represent a set of rules that define the relationship between the observed actions and the underlying goals or intentions. By applying these rules to the observed actions, rule-based systems can infer the most likely goals or intentions. Rule-based systems that use a knowledge base and a set of inference rules can also be used to diagnose errors or faults. Moreover, they can be used to represent a set of rules that define the relationship between the initial state, the desired goal state, and the sequence of actions that should be performed to achieve the goal. By applying these rules to the initial state, rule-based systems can generate a plan for achieving the desired goal. For example, [137] describes a rule-based assistant system for managing the clothing cycle and in [138] an ontology and rule-based intelligent patient management assistant is described.

• Fuzzy logic: Fuzzy logic is a mathematical approach to deal with uncertain or imprecise data that are common in intelligent help systems and virtual assistants. It allows for degrees of membership in a set, rather than a strict binary membership. For example, [139] describes an intelligent fuzzy-based emergency alert generation to assist persons with episodic memory decline problems.

• Hierarchical Task Networks (HTNs): HTNs can be used to represent a plan as a hierarchy of subgoals and decompose the plan into a set of smaller, more manageable tasks. By recursively decomposing the plan into subgoals, HTNs can generate a plan that achieves the desired goal. For example, in [140] it is pointed out that integrating task and motion planning through the use of multiple mobile robots has become a popular approach to solve complex problems requiring cooperative efforts, as advancements in mobile robotics, autonomous systems, and artificial intelligence continue to rise.

• Reinforcement Learning: Reinforcement learning is a branch of machine learning concerned with using experience gained through interacting with the world and evaluative feedback to improve a system’s ability to make behavioural decisions [141]. For example, [142] shows how Reinforcement Learning is helping to solve Internet-of-Underwater-Things problems.

• Speech Recognition: This AI technology can be used to enable voice-based interactions with intelligent help systems and virtual assistants. This can be especially useful for users who have difficulty typing or navigating a graphical user interface. For example, the voice recognition and natural language understanding (NLU) capabilities of Amazon Alexa [109] are powered by machine learning algorithms, which enable it to interpret user commands and respond with appropriate actions.

• Computer Vision: This type of AI can be used to enable visual interactions with intelligent help systems and virtual assistants. For example, users could be interacting with a camera that performs image analysis on the face of the user to identify emotions of users while interacting and detect some displeasure on the part of the user which may need attendance. Another example in [143] describes research on detection of road potholes using computer vision and machine learning approaches to assist the visually challenged.

• Adaptive hypermedia (AH) is a field of research that combines artificial intelligence (AI) and human-computer interaction (HCI) to develop systems that can dynamically adapt to the needs and preferences of individual users. AH systems use a combination of user modelling, content modelling, and adaptive navigation to provide personalised information and services to users. More specifically, adaptive hypermedia is an AI method that uses cognitive modelling, machine learning algorithms and other AI techniques to analyse data on users’ behaviour and draw inferences about their respective level of knowledge, preferences, and interests, and then adapt the content and navigation of a website or application to meet their needs. For example in [144] fuzzy logic has been used for student modelling so that adaptive navigation techniques are applied to automatically adapt the pace of tutoring to individual strengths and weaknesses of students. In another instance, adaptive hypermedia are used to adapt tutoring of algebra-related domains to the individual needs of students [145].

• Student modelling is the process of creating and maintaining a model of a student’s knowledge, skills, preferences, and other relevant attributes. The goal of student modelling is to provide personalised support and feedback to students, based on their individual needs and abilities in the domain being taught. It involves the use of artificial intelligence techniques to create and maintain the model. Some of the AI methods used in student modelling include machine learning algorithms, rule-based systems, cognitive theories and fuzzy logic. For example, in [146] an Artificial Immune System-Based student modelling approach is described to adapt tutoring to learning styles that fit the students’ needs. At another instance, student modelling is used to identify areas where learners have weaknesses so that the intelligent tutoring systems (ITS) may provide appropriate instruction to help them improve their performance in the domain of English as a second language [147]. A more comprehensive review of student modelling approaches and functions is presented in [148].

• Learning Analytics. A new research discipline, termed Learning Analytics, is emerging and examines the collection and intelligent analysis of learner and instructor data with the goal to extract information that can render electronic and/or mobile educational systems more personalised, engaging, dynamically responsive and pedagogically efficient [149].

• Error diagnosis is an important function of Intelligent Tutoring Systems that uses AI techniques to diagnose problematic parts of the students’ behaviour or beliefs that lead to errors in assessment tests and reveal misconceptions. The deep error diagnosis frequently requires conflict resolution as erroneous behaviour may be due to many conflicting hypotheses about possible students’ misconceptions. As an example, in an English tutor, a process of real-time evaluation of conflicting hypotheses about students’ mistakes is used to perform a deep error diagnosis. This approach aims to identify the underlying students’ weaknesses that may be causing the errors [150].

• Knowledge Representation is an important component of Intelligent Tutoring Systems that allows the representation of a domain being taught to be used by several AI inference mechanisms to find out the progress or weaknesses of students. For example, in [151], Fuzzy Cognitive Maps are used for the Domain Knowledge Representation of an Adaptive e-Learning System.

• Fuzzy logic. This is a type of AI that has been used in many aspects of Intelligent Tutoring Systems. For example, fuzzy logic has been used to determine the knowledge level of students of programming courses in an e-learning environment [152] and on another instance it has been used for student-player modelling in an educational adventure game [153].

• Machine learning: This is a type of AI that involves the use of algorithms and statistical models to analyse data and make predictions or decisions without being explicitly programmed. Machine learning can be used in intelligent tutoring systems to personalise the learning experience for each student by analysing their learning data, preferences, and behaviour. For example, [154] decribes a machine learning-based framework for the initialisation of student models in Web-based Intelligent Tutoring Systems of various domains, such as mathematics and language learning.

• Deep Learning: Deep learning is a subset of machine learning that uses neural networks to simulate the workings of the human brain. Deep learning can be used in Intelligent Tutoring Systems at many levels and for varying functionalities. For example, in [155] neural networks have been employed to perform visual-facial emotion recognition so that the tutoring system may analyse the face of a student through a camera and understand how the student feels about the lesson so that it may respond in an appropriate manner.

• Cognitive theories: This is a type of AI that is designed to simulate human thought processes, including perception, reasoning, and learning. Cognitive computing can be used in intelligent tutoring systems to provide more sophisticated feedback and guidance to students, based on their individual learning styles and cognitive abilities. For example in [156], student modelling is performed using cognitive theories to identify cognitive, personality and performance issues in a collaborative learning environment for software engineering. In another instance, the individualisation of a cognitive model of students’ memory is performed in an Intelligent Tutoring System, so that the system may personalise the presentation and sequencing of revisions of teaching material depending on when the student is likely to have forgotten the material taught in preceding sections of the ITS [157].

• Decision Theories: Decision theories are a set of frameworks and models used to make rational and informed decisions in the face of uncertainty. These theories aim to identify the best course of action among a set of alternatives, taking into account the potential outcomes and their probabilities.

An example of an intelligent medical tutor that uses decision theories is described in [158]. This tutor offers adaptive tutoring on atheromatosis to users with varying levels of medical knowledge and computer skills, based on their interests and backgrounds. This adaptivity is achieved through user modelling, which draws on stereotypical knowledge of potential users, including patients, their relatives, doctors, and medical students. The system uses a hybrid inference mechanism that combines decision-making techniques with rule-based reasoning of double stereotypes.

• Affective computing: This type of AI focuses on recognising human emotions and generating responses that contain emotion-like feedback from the part of the computer. One way that affective computing can be used in ITS is through the use of emotion recognition technologies. These technologies can detect and interpret facial expressions, vocal tone, and other parts of the users’ behaviour to determine a student’s emotional state. An illustration of this process is the use of a voice recognition system and analysis of keyboard actions to perform emotion recognition of students in an Intelligent Tutoring System (ITS) [159]. The system takes into consideration the recognized emotions and utilizes a cognitive theory of affect to generate emotion-based responses that are appropriate for the learner’s current affective state. These emotions are conveyed to the learner through an animated agent, which mimics human empathy by adjusting the tone and pitch of its voice while providing recommendations for specific learning materials or modifying the level of instruction difficulty. In a different scenario, described in [160], the cognitive theory of affect is utilized to identify the emotions of students instead of generating emotional responses. This approach is implemented within an Intelligent Tutoring System designed as an educational game. Three animated agents are employed to simulate the roles of a classmate, tutoring coach, and examiner, and they provide appropriate responses based on the recognized emotions of the student.

• Mobile and smartphone senses: Mobile and smartphone devices provide a wealth of sensors that can accumulate users’ responses which, in turn, can be exploited for analysing users’ behaviour in the context of an intelligent tutoring system. Mobile devices can provide profound reasoning [161] concerning users through smartphone senses [162]. One such example is the sentiment mapping through smartphone multi-sensory crowdsourcing described in [163].

• Engagement and immersive technologies: Engagement and immersiveness of learners in computer-based tutoring constitute important features that are sought by learning applications to maximise the educational effectiveness [164]. To achieve these there has been a growing emphasis on recognising human emotions in interactive computer-based learning applications, while using multi-modal user interfaces, natural language as well as virtual reality environment in conjunction with AI techniques.

• Educational games and edutainment: The technology of game playing has been blended with AI techniques and together they constitute important parts of educational software in the context of Intelligent Tutoring Systems to provide engaging challenges to students so that they stay active and alert while learning. These educational activities are used to educate and entertain and often titled as edutainment technology. For example, in [165] a software version of the well known game “Guess who” is developed to teach students parts of the English language as a second language. In [166], a 3D game is developed for the purposes of education and fuzzy-based reasoning mechanism is used to ensure that students receive a dynamically adjusted difficulty level of the game itself so that they may enjoy the game and attend the teaching material [167] without encountering difficulties or being bored with the game itself. In another instance, an educational adventure game encompasses adaptive scenaria to accommodate the individual learning needs of each student [168].

• Collaborative Learning: The social context of class involving students with classmates and allowing collaboration is important in the area of Intelligent Tutoring Systems to enhance learning experiences. For example, in [169] an intelligent recommender system for trainers and trainees has been incorporated in a collaborative learning environment in order to recommend to students and teachers teams of students that appear to have complementary qualities for the purposes of educational collaboration in project assignments.

• Social Networks: Social networks may extend collaborative learning in the platforms of social media. For example, [170] describes an Intelligent Tutoring System over a social network for mathematics learning.

• Natural language processing (NLP): This is a type of AI that enables computers to understand, interpret, and generate human language. NLP can be used in intelligent tutoring systems to provide students with feedback on their written or spoken responses, and to help them practice their language skills in a more natural way. In this approach, AI-empowered tutoring systems can use natural language processing (NLP) and machine learning algorithms to understand user input and provide more human-like responses and feedback by achieving a more natural and interactive learning experience. For example, in [171] the focus is on the combination of various deep learning approaches to automatically help students to accelerate the learning process by automatic feedback, but also to support teachers by pre-evaluating free text and suggesting corresponding scores or grades.

• Computer vision: This is a type of AI that enables computers to analyse and interpret visual information from the world around them. Computer vision can be used in intelligent tutoring systems to track students’ eye movements and facial expressions during learning activities, and to use this information to personalise the learning experience. For example, in [172] visual-facial data is collected and analysed for the purposes of group affect recognition in a the context of a class of students.

• Authoring tools for Intelligent Tutoring Systems: Authoring tools can provide user-friendly environments to human teachers so that they may deliver Intelligent Tutoring Systems to be used by their students in the domain of their teaching expertise. Technically they use AI-based techniques that have to be parameterised so that the teachers can author the content and the way they wish their students to be taught by the Intelligent Tutoring Systems. For example, in [173], an authoring tool for mobile intelligent tutoring systems is presented while in [174] a user model the instructor, termed instructor model, is created to help instructors with managing the difficulty level and the content of their teaching material to fit their teaching needs and their students’ learning strengths and weaknesses.

• Human expectations and unexpectedness of AI

• Levels of Trust for Humans

Trust is described by Körber [177] along six dimensions: 1. reliability, 2. predictability, 3. the user’s propensity to trust, 4. the attitude towards the developers, 5. the user’s familiarity with automated systems and 6. general trust in automation.

However, the user’s trust may reach some problematic levels that can affect the reliability of the whole interaction. In this research work, the identified problematic levels of trust or absence of it are the following:

• Black box problem and explainability

Many AI methods such as deep learning are quite efficient but do not provide explanations as to how they have reached their conclusions. For example, in [178] it is argued that when applied to the healthcare domain, machine learning models fail to meet the needs for transparency that their clinician and patient end-users require and that opaque models (1) lack quality assurance, (2) fail to elicit trust, and (3) restrict physician-patient dialogue. According to Carabantes the most powerful artificial intelligence models based on machine learning are often the ones with the most opaque black-box architectures, despite the increasing demand for their use in various domains in computerized advanced industrial societies [179]. Thus, this gives rise to an important control problem for human society. The lack of transparency in the reasoning of opaque AI systems not only undermines users’ trust but also obstructs communication between domain experts who use the AI systems and their clients who require justifications, as well as among domain experts who need arguments to collaborate effectively in a human team. This is because people are skilled arguers, using reasoning both to evaluate and to produce arguments in argumentative contexts [180]. The categories for black box problems faced by users can be classified in the following restrictions concerning human users:

• Levels of depth in terms of human reasoning of AI in the systems

According to Nilsson achieving real human-level artificial intelligence would necessarily imply that most of the tasks that humans perform [181]. According to the Bloom’s Taxonomy there is hierarchy of thinking skills at 6 levels, namely, 1. Knowledge, 2. Comprehension 3. Application, 4. Analysis, 5. Synthesis, 6. Evaluation. The thinking skills of the three lower levels of the taxonomy – Knowledge, Comprehension, and Application – are referred to as lower-order thinking skills, and those of the higher levels – Analysis, Synthesis, and Evaluation, as higher-order thinking skills [182]. Butterworth and Thwaites argue that creativity and imagination are equally important to rational thinking in human development and that the overlap and interdependence between the two make it hard to differentiate them. Usually, the upgrade from low level thinking to higher level thinking has been the goal for human educators concerning human students. For example, in [183] the authors show a chemistry course that provides the opportunity for students to move beyond surface-level thinking to critical thinking as they use quantitative reasoning skills, analyse data, and draw connections between observations and explanations [184]. However, to what degree AI methods can attain these human thinking levels?

• Levels of knowledge

According to [185] different types of knowledge in terms of what knowledge is needed to do a particular type of work or take a particular action and are categorised as surface knowledge, shallow knowledge and deep knowledge. Shallow knowledge is when you have information plus some understanding, meaning and sense-making. Deep Knowledge refers to an extensive, versatile, and flexible comprehension of a particular subject that enables advanced thinking and problem-solving at an expert level.

• Levels of personalisation and user modelling

• Levels of anthropomorphic characteristics

• Degrees of User control

By extending the categorisation of intelligent help systems to passive and active, to span all interactive systems that incorporate intelligence, there can be the following two categories of AI systems depending on who has the initiative of the interaction and thus user control:

• Bias and Fairness

AI systems can exhibit bias due to a variety of factors, including the algorithm used, the input data selected, and hidden correlations within the training data. According to Roselli and colleagues there are three primary categories of bias: those that stem from the translation of business objectives into the AI implementation, those that result from the distribution of samples used for training (including the impact of historical data), and those that exist within individual input samples [188]. Nelson gives examples from the medical field to demonstrate how Artificial intelligence (AI) operating without oversight could maintain prejudice: “Imagine an algorithm that selects nursing candidates for a multi-specialty practice – but it only selects white females. Consider a revolutionary test for skin cancer that does not work on African Americans” [189].

Concerning fairness of AI, Shrestha and Das point out that although, the relevant discussion within the realm of ML and AI is a recent development, fairness problems and discrimination have roots within human society where the unfair treatment toward the minority is documented in the data that has been created over time and therefore, the inadvertent learning and perpetuation of implicit biases is a longstanding issue in ML/AI systems due to historical biases in the data we have accumulated, which can lead to discriminatory outcomes such as an advertisement algorithm showing more high-paying technical jobs to men than women [190].

In addition to bias resulting from training data, AI systems may also be affected by unintended bias that can originate from the algorithms and the developers involved in their creation. According to Borenstein and Howard, the decisions made by developers during the design process not only carry ethical implications but also reveal the developers’ ethical values. However, developers often view ethics as a responsibility belonging to someone else, such as lawyers or ethicists, rather than as an integral part of their own technological work. [191].

Finally, it is possible for bias in AI systems to be intentionally introduced with malicious intent. This can occur when developers intentionally manipulate the training data or algorithms to produce biased results, either to promote their own agenda or to harm a particular group or individual. For example, recommender systems may suffer from being attacked by malicious raters, who inject profiles consisting of biased ratings since recommender systems are widely deployed to provide user purchasing suggestions [192].

• Usability

The adoption of software by human users is primarily determined by its usability, as stated in ISO/IEC 25010 [193]. Usability refers to the extent to which a product or system can be used by specific users to achieve specific goals effectively, efficiently, and satisfactorily in a specified context of use. Usability of all interactive programs is affected by the existence of appropriate feedback and by the ability of the system to perform effective error prevention while users are interacting with the system in pursuit of their goals. However, a lot of important usability factors remain understudied and this is worsening while AI is expanding in the UX. For example, despite the importance of errors in the usability, a recent preliminary study on usability found that errors are among the least commonly considered usability attributes [194].

Certain aspects of usability are particularly important for Human-AI Interaction. Norman pointed out that the ability to control our systems through interactions that bypass conventional mechanical switches, keyboards, and mice is a valuable addition; however, it also presents new challenges, potential for significant errors and confusion, despite its potential benefits [195]. Nielsen highlights the lack of standards and expectations for UI design on iPad apps which means that anything that can be shown and touched can be a UI on the device [196]. Similarly, to these apps, many assistive AI functionalities are evoked without sufficient notification to the user and thus may cause errors and confusion. For example, If the AI-empowered personal assistant is evoked accidentally by the user, it may suggest something that the user did not intend to ask for or may not be interested in, which can also cause confusion, misunderstandings, and errors. In the context of usability in UX in AI, two important factors are feedback and error prevention:

• Privacy and security of data

Privacy and security issues are very important in the UX of AI as all personalisation techniques gather information about all possible aspects of users, to be used to tailor user experiences and fit the needs of individual users. However, the information that has been gathered could contain both factual data and conclusions drawn about users, potentially without their knowledge or agreement. This data could also be exploited, altered in a harmful manner, or unlawfully obtained by third parties with a history of engaging in fraudulent or cyber-criminal activities.

• Ethical Considerations

Ethics of AI studies the ethical principles, rules, guidelines, policies, and regulations that are related to AI. Ethical AI is an AI that performs and behaves ethically [197]. In general, AI ethics encompasses various ethical considerations that aim to ensure AI benefits people, remains impartial, safeguards privacy and security, and operates with transparency and accountability. Ethical considerations are extremely important in some cases in security contexts such as armed conflict, law enforcement, and disaster relief, decisions must often be made with incomplete information, while under the pressure of time and stress. To alleviate some of this pressure, AI systems can provide valuable decision-making support. As a result, such systems will become increasingly vital. Nonetheless, since human lives may be at risk in these situations, humans should retain ethical accountability for any decisions made, regardless of the AI system’s involvement, whether it’s as a decision-making or decision-support system [198].

• Human-Centered Artificial Intelligence: (HCAI) is an approach to developing AI systems that prioritises the needs and values of humans. This approach involves designing AI systems that are focused on enhancing human well-being, rather than just optimising technical performance or efficiency. According to Shneiderman, linking the development of AI systems to guidelines that assure transparency and accountability will enhance innovation, public confidence, societal value, and commercial value [199]. The UX of AI should prioritise human-centered design, which means creating interfaces that are intuitive and easy to use. The AI system should be designed to fit seamlessly into the user’s workflow and provide value without being intrusive.

• Explainable AI (XAI): XAI is an AI system that can provide human-understandable explanations of its decision-making process. In various practical domains, such as medical diagnosis, financial services, legal decision-making, scientific research, and social media analysis, the interpretability and openness of AI models are crucial not only for their end-users and stakeholders but also for the developers and investigators building the AI solutions. For example, researchers are working towards bridging the gap between explicit knowledge-based AI methods, such as Knowledge Graphs, and implicit knowledge-based AI methods, such as Deep Neural Networks, by combining their strengths through Explainable AI research [200]. Explainable Artificial Intelligence (XAI) can play a significant role by providing interpretable and transparent models, which can enhance users’ trust in the system [214].

• Responsible AI: Ethical discussions are a vital foundation but raising the edifice of responsible AI requires design decisions to guide software engineering teams, business managers, industry leaders, and government policymakers [201]. Regarding responsible AI, one potential approach is to integrate ethical reasoning into the AI systems themselves during their development and incorporation. This means designing AI algorithms and systems with ethical considerations in mind, such as fairness, transparency, privacy, and accountability. By building ethical reasoning directly into AI systems, it becomes easier to ensure that they align with ethical principles and values, which can reduce the risk of unintended harm or negative impacts. This approach can also help promote trust and confidence in AI among users, stakeholders, and the broader public [202].

• Prosocial AI: Prosocial AI refers to the development and application of AI technologies with the goal of promoting human well-being and social welfare. This includes the design of AI systems that are transparent, explainable, and trustworthy, as well as those that support ethical decision-making and minimise the risk of negative consequences or unintended harm. By incorporating prosocial principles into the development and deployment of AI, it is possible to create systems that are aligned with human values and promote the greater good. For example, Paiva points out the prevailing view of human decision-making in AI design, which is based on the homo economicus principle, and proposes exploring the creation of AI agents that prioritise prosocial behaviour to cultivate cooperation and contribute to the social good in human settings [203].

• Combination of multiple AI techniques

AI techniques have advanced to a high extent over the years and involve different approaches. It is considered that AI contains, as a subset, all machine learning algorithms, which in turn contain, as a subset, all deep learning techniques, including neural networks.

• Levels of breadth of implementations of AI in the systems

Artificial Intelligence (AI) can be implemented at varying degrees of breadth, depending on the scope and scale of the application. Here are some examples of different degrees of breadth in AI implementations:

• Accuracy, Trust and Explanations

Trust, accuracy, and explanations are key factors in the design and deployment of AI systems. Studies have shown that users’ trust in AI is closely linked to the accuracy of the system’s outputs and the quality of explanations provided by the system [212, 213]. Lack of trust in AI can lead to decreased reliance on automated systems, which can impact the system’s effectiveness. Therefore, it is important to provide accurate and reliable AI models that are capable of providing clear and concise explanations to users, which can build trust in the system [212, 214].

Furthermore, the role of human-in-the-loop [215] is being recognized as a critical aspect to ensure that AI systems align with human values and comply with regulatory frameworks such as the European Union’s General Data Protection Regulation (GDPR) [216]. The GDPR emphasises the importance of data protection, and it is the responsibility of organisations to ensure that their AI systems comply with the regulations. Trust and regulation are important factors in the development of human-in-the-loop AI, which involves a collaboration between humans and automated systems. Solutions for accuracy, trust and explanations include the following:

• Uncertainty quantification: AI systems can be designed to quantify the uncertainty associated with their predictions.

• Human supervision: AI systems can be designed to work in conjunction with humans, who can provide oversight and make decisions based on their own judgement.

• Human-in-the-loop: The importance of the human-in-the-loop approach is gaining recognition [215]. It is considered a critical aspect to ensure that AI systems align with human values and comply with regulatory frameworks, such as the European Union’s General Data Protection Regulation (GDPR) [216]. The GDPR places a strong emphasis on data protection, and organizations are responsible for ensuring that their AI systems comply with the regulations. Trust and regulation are essential factors in the development of human-in-the-loop AI, which involves collaboration between humans and automated systems [212].

• Levels of AI development iterations: An iterative development process can enhance the accuracy and efficiency of AI systems. This involves making incremental improvements over time by gathering feedback, analysing performance metrics, and adjusting algorithms, training data, or infrastructure. By repeating this process over multiple cycles, AI systems can achieve significant performance gains, particularly in machine learning, natural language processing, and computer vision.