User Interface Adaptation based on a Business Rules Management System and Machine Learning

In ubiquitous computing environments, computers are embedded into everyday lives to provide information anywhere and at any time. Pervasive computing assists us in our everyday activities, operating invisibly in the background, regardless of our location or devices. Therefore, ubiquitous computing requires discovering new Human-Computer Interaction methods helping users to easily interact with ubiquitous systems whatever is the perceived context situation. However, designing context-aware applications that are able to adapt to context instability is a recurring problem that requires special attention from researchers in the HCI community. On the one hand, it is not trivial for a designer to specify how UIs should adapt, and on the other hand it is very hard to predict the context of use changes and accordingly to construct adaptive UIs that match users expectations. In this paper, we present a Model Driving Engineering (MDE) technique to design UIs that automatically adapt to the perceived context situation while the designer and eventually the end user still have full control over the adaptation during runtime. This technique is supported by a conceptual framework and a graphical tool that lower the threshold for designers as well as developers to design adaptive user interfaces, modelize context situations, edit adaptation rules and manage the adaptation process.


Introduction
Emerging ubiquitous environments deliver products and services that are more and more sophisticated creating competitive challenges for computer engineers particularly the Human-Computer Interaction (HCI) community. Welldesigned User Interfaces (UIs) could fail to satisfy end-users if they do not cover the unpredictable dynamic variation of the context of use. UIs should be accessed through several devices such as smart phones, notebooks and tablets and used by people with different skills and abilities. This implies the necessity of qualifying UIs with flexibility and adaptability to increase their level of accessibility and accommodate the context requirements.  (2015) focus on a fully dynamic approach where the adaptation behavior is performed at runtime. However, most of these solutions neither end-user nor designer takes part in the adaptation process in order to select relevant adaptation rule or strategy to be performed.
In our work, we adopt a Model Driven Development approach to generate Final User Interface (FUI) adapted to a specific context's situation. In addition, the adaptation specifications are integrated in the generation process of the Cameleon Reference framework (CRF), proposed by Calvary et al (2003), in order to ensure that UI consistency is preserved even when making changes at any level representation and regenerating the interface. We also take into account that the designer should be a full partner throughout the process. Therefore, our solution consists of a modelbased toolkit offering graphic tools for: • generating UIs from tasks level to the final one, • managing context's situation parameters, • managing adaptation rules • launching adaptation process Thanks to these tools, developers have full control over the adaptation process. They can configure the interface components according to the user preferences or capabilities. If the resulted interface does not satisfy the initial specifications, the process can be rerun with different parameters until the result is satisfactory.
On the other hand, Model Driven approaches aim to deal with the complexity of interactive systems and decrease the effort needed to develop UIs. They put emphasis on models with different levels of abstraction which can be obtained successively through model to model and/or model to code transformations. We exploit these benefits not only at design and generation stage but also to achieve context-dependent customization. Therefore, we define models to describe context components (user, platform, environment) as well as rules adaptation. This can be considered as one of the advantages of this work since rules edition was not addressed by the major of the existing approaches. In fact, in most cases, rules are hard coded or not extensible. Besides, it happens that the designer defines various rules for the same context situation. In this case, conflicts can occur and affect the consistency requirements of the UI or even disable other adaptations. By conflicts we mean situations in which adaptation behavior would have no sense following some operations that impact each other (for example, adding and then deleting the same component), or cause an inconsistent state for the system. To deal with this issue, we proceeded to consolidate our solution by a Business Rules Management System (BRMS) that supports the execution of rules and conflicts management, an aspect that has not been deeply addressed in previous research works.
This article presents our approach to handle the adaptation challenge: first, we present DNA, our model-based toolkit to dynamically generate context-dependent UIs. We principally focus on the adaptation stage of the development process and we show how designers can easily define context situations, manage a database of adaptation rules, select rules that are suitable to current situation and finally launch the adaptation process. Second, generated UIs are at the same time adaptive: for any change in the context of use, related models are instantly updated and the FUI is regenerated. Through using DNA we have generated multi-language UIs (Java and HTML5 versions) and we have tested their adaptability based on a runtime context simulator.
The remainder of this paper is structured as follows: in section 2 we analyse some related works. Section 3 discusses our approach and describes the DNA toolkit and shows how it can be very useful for designers. Section 4 illustrates the adaptation process and explains how generated UIs can adapt themselves dynamically to the context variation. Finally, we conclude and outline our future work.

Related Works
Integrating Adaptation in the UI Development Process Regarding adapted UIs to specific context's situations or to users with special needs, most works had integrated adaptation in the development process by transforming models changeable according to the context parameters. Bouchelligua et al (2010) proposes an approach that allows the adaptation to the context of use based on parameterized transformation. Accordingly, each transformation requires as an input a context model and a UI model and generates as an output the adapted one.
This work models context's elements and offers a formal description for UIs models based on Business Process Modeling notation (BPMN). Nevertheless, it supports adaptation only at Abstract and Concrete UI's levels and merely at design time. Besides, no tool is developed for this approach thus far.
Ditto for Zouhaier et al (2015), who proposes a generic solution for developing accessible UIs applications based on parameterized MDE-transformations. From an application model, which provides an abstract view of the user interface, and an ontology model that describes the accessibility context, the system applies a reverse engineering and series of transformations in order to adapt the AUI. Once done, the FUI is regenerated. The solution aims to incorporate the modality of interaction into adaptation process and provide the system with more flexibility to make it possible choosing desirable modality into the UI, but no tools supporting this approach have yet been developed.
Peissner et al (2012) propose MyUI that provides customized user interfaces that are adapted to a broad range of user's needs, different devices and various context conditions by adapting their presentation layout and modalities, and the navigation paths. Controllability is partially covered by the system in the sense that it asks the user to accept or reject the adaptation. MyUI also supports adaptation at runtime through an automated and rulebased UI generation process, which uses bundles of patterns supporting different user's needs or context conditions. According to the user's profile or the device profile, the most suitable pattern is selected and applied. Finally, this work is distinguished by developing a toolkit that supports developers in the creation of adaptive applications. It helps them specify the application properties, explore the available patterns and select those needed for the interaction situation.
Otherwise, in the research study of Miñón (2015), the adaptation process is based on a repository, which stores rules devoted for people with disabilities. These adaptation rules are integrated into the development process at both design time and runtime. At the design time, the designer can select adaptation rule dedicated for the disability in question and the UI abstraction level and launch the generation process. At the runtime, the same procedure is followed but automatically without the intervention of either the designer or the end-user. Additionally, it should be noted that rules conflicts are resolved by ordering them by a granularity level assigned to each of them. By granularity level, authors of this work denote parts of the UI that can be affected by the adaptation such as the entire application, the presentation format, a group of elements or only a single element.
Akiki et al (2016) present Role-Based UI Simplification (RBUIS) as a system for implementing adaptive UIs by applying two types of adaption named feature-set minimization and layout optimization. The first one disables the UI components that are not required by particular end-users. Afterwards, layout optimization adapts UI presentation to fit the context of use situation. This approach being based on a reference-architecture called the Cedar Architecture supports any changes to the environment, the end user's profile, and the targeted platform. The relevant adaptive behavior can be applied on any UI model and then transferred to UI-presenter, which is responsible for rendering the FUI. To resolve the probable conflicts between adaptation rules, a trade-off manager is used to select those meeting adaptation constraints as much as possible. Another important point to consider in this work is providing end-users with the possibility to reverse the adaptation behavior or even choose other possible alternatives.
Within this background, we notice that a common trend in the design of UIs adaptation is to integrate adaptation requirements into the development process and transfer them to the FUI. However, particular importance should be accorded to: 1) a support tool for allowing designers to develop UIs and adapt them with the proposed adaptation technique 2) the adaptation rules design in order to ensure consistency within UI models and avoid eventually conflicts between them 3) the human involvement in the development process.

Designing Rules Adaptation
Existing approaches can substantially differ about adaptation features, but they all agree that rules design and structure are the key to rise to this challenge. However, rules are often hard-coded or designed in ad-hoc manner, making updating or reusing them a very difficult task. This is e.g. the case for Barrera  . Recently, studies have been more interested in modelling rules adaptation in order to ensure efficiency and ease of use. For example, López-Jaquero et al (2009) define rules are defined using a metamodel and specified in terms of the context of use events and the UI model transformations needed to be applied. The rules design is supported by T:XML, a graphical tool that makes it easy to the designers modelling adaptation rules and automatically generating code for the transformation language. Inspired by the AGG tool proposed by Taentzer in 2000, these transformations are specified in UsiXML language and based on graph grammar. Barrera et al (2014) present Tukuchiy, a rule-based framework that generates dynamic UIs while preserving usability criteria. Several techniques of HCI are mapped to adaptation concepts in order to adjust them to different users and contexts. Rules are hard-coded but supported by a conceptual integration model called Runa-Kamachiy wich provides a taxonomy of adaptation concepts describing adaptation nature and process according to the user's profile, its context of use, its tasks and the UI model. According to Miñón et al (2015) aforementioned, the adaptation rules are defined using the Advanced Adaptation Logic Description Language (AAL-DL) that can be applied to UIs described in others MDE languages, consistent with the CRF. Each rule is structured in terms of event, condition and action. It can be performed in both design time and runtime. In the design time, it is directly integrated in the UI by the designer while at runtime, it is triggered by an event. If the condition is satisfied, the action is then executed. Elsewhere, Zheng et al (2016) use a graphical representation to model adaptation rules, inspired from the decision table reasoning. It consists of an adaptation tree structured on two types of nodes: condition node and conclusion node. The first one checks the context conditions whereas the second one represents the action to be achieved. Each path in the adaption tree, from the root to the leaf, presents an adaption rule. Thereby, to make the relevant decision, the adaptation tree splits the nodes on all available variables (context conditions) and then selects the split, which results in most checked sub-nodes.
To conclude, available research and designers' opinion commonly point to at least three adaptation aspects to be considered (i) a conceptual framework for modelling adaptation rules (ii) an adaptation manager for making decision and executing the associated behaviour (iii) a graphical tool for supporting the specification of the adaptation rules by designers even if they lack knowledge about accessibility requirements. Moreover, it helps them to control the adaptation process.
In the following section, we present our solution for adapting UIs to the context of use, which takes into account these three aspects of adaptation.

Solution Overview
The approach proposed in this paper takes advantage of the MDE engineering by applying different types of adaptation on the various abstraction levels of the CRF before deriving the FUI that will be delivered to the end-user. By applying this approach, we are covering almost all the adaptation requirements previously discussed and we are supporting the integration of adaptation rules in the development process of accessible UIs at both design time and runtime. The proposed theoretical architecture illustrated by figure 1 includes four modules relaying to the adaptation system: (1) UIs Models: these modules are specifying the UI at each level of abstraction of the CRF and lead a transformational method allowing progressive generation of the FUI from the task model.

A Metamodel For UI Adaptation Rules
The adaptation model specifies the appropriate reaction that can occur and that has an impact on the UI. According to the metamodel depicted by figure 2, an adaptation rule is structured upon two parts. The left part (the green block) refers to events that can trigger the adaptation. An event may be internal to the system or a change in the contextual situation either set by the designer ("ContextVariable") or detected by a sensor ("SensorVariable"). AdaptationRule (yellow block) contains Meta information about the rule: its identifier, the policy expresses the level of controllability of the adaptation process, the strategy and the priority value. The right part (the red block) specifies the action to be performed and the UI models involved in the adaptation.
We plan various actions in accordance with the selected strategy and the involved UI model. Some of these actions aim to update the task model but most of them act on the CUI model. At this level, we mainly focus on the modalities of the UI in order to provide to the system with the relevant modality according to the end-user's disability. In fact, we think that multimodal processing is preferred by end-users as it offers more natural interaction with the computer. Multimodal interactions are based on CARE properties (Complementarily, Assignation, Redundancy and Equivalence) that deal with modalities combination from the user perspective.
The main advantage of this metamodel is that it can be extended by other classes describing the adaptation features or behaviors. It is also valid at runtime as well as at design time.  The adaptation manager handles the whole adaptation process. It is particularly composed of two main modules: the Storage Manager and the Adaptation Engine.
The Storage Manager: stores and manages the adaptation rules in a relational Database (MS Access Database) to make it easier for the designers to add new rules or update the existing ones. The same rules defined at the design time can be used when needed at runtime. These rules are then sent to the Adaptation Engine that will achieve succeeding steps of the process.
The Adaptation Engine: due to the large amounts of data that may be treated, decision making becomes more and more difficult. Possible conflicts can also occur between some rules that refer to the same context situation but imply different actions. Thus, we find more interesting to use an inference engine that ensures the performance and even the consistency of the adaptation behavior. A comparison study between some inference engines prompted us to choose the Drools Rules Engine (DRE).
Drools is a BRMS that uses the rule-based approach to implement Expert Systems. It can deal with a large number of rules and facts. Using an adapted implementation of the Rete Algorithm, it matches facts and rules in order to infer conclusions, which results in actions. To be read by the DRE, rules templates must be written using a domain-specific language (DSL). Therefore, in order to hide the complexity of the language and make the definition of rules' templates and their maintenance easier even for non-domain experts, we implemented a module that allows their automatic generation. It consists of two types of MDE transformations: model-tomodel transformations that create rules models conforming to our adaptation metamodel, from data retrieved from the and model-to-code transformations that generate the Drools' rules from the rules models. Figure 3 shows an example of a generated Drools rule's template that defines a rule called "UserPref". According to the user's profile, chiefly his age and his background-color preferences, this rule updates the CUI model by modifying the font size of interactors' text as well as the background color of the container (the container can be a frame, a window or a web page, according to the target language of the FUI).

Fig. 3: An Example of a Drools' rule template
As illustrated by figure 3, each rule has a salience value. It is a form of priority where rules with higher salience values are given higher priority when ordered in the Activation queue.
Based on a foreword chaining system, we feed the knowledge base with two types of data: the context's models data that matches with the left hand part of a rule and the UI's models that will receive inferred conclusions. When the adaptation process is launched, the Drools engine loads his working memory by facts and then proceeds by evaluating the rules. Only those that their condition part is evaluated to true are fired. If more than one rule matches a fact, they will be sorted by salience values. If multiple rules have the same salience, the DRE guarantee executes them at the same level.    The context situation describes the current conditions of use of the system. It includes parameters referring to: (5) the end-user of the system: such as the disability type if it exists, the domain experience's level, the mobility state and the current activity (6) the platform used for the interaction: such as the screen size, the battery level and the multimedia equipment (7) the physical environment where the interaction will take place: such as the noise level, the light, the time and the current location. Adaptation rules can be edited according to the defined context situations. Each rule has a priority value and is specified by the adopted strategy, the target UI model, the action to be performed and the CARE property for only actions that will be applied on the CUI model. For example, the adaptation rule "AtWorkRule" shown in figure 7 has 60 as priority level and is triggered when the context situation "AtWork" is selected (see figure 6). As it is based on the "Removing Task" strategy, this rule acts on the Task Model by removing all the optional tasks.

Machine Learning In the Support of UI Adaptation
In general, rule-based systems can be used to perform UI adaptation at design-time and make automated decision about their behavior according to different context situations. They are attractive because they are simple to manage and to understand. However, they face some issues. For example, they are unable to adapt to new data and need to be under expert control. Furthermore, rules cannot be learned or derived from the training data and remained fixed overtime. As result, we can say that these systems will correctly work only if we know all situations under which decisions should be made beforehand, that is at design time. Therefore, we need to adopt a solution that will help them to recognize new context situations and build suitable rules on the fly. Machine Learning (ML) techniques fit perfectly with this issue and seem to be a significant potential to convey the UI adaptation at runtime. here faced to a multi-label classification problem where we expect to predict more than one class labels at the same time. At this stage, the user preferences are introduced into the prediction process. The action of the system will be selected among actions relatives to the retained rules by the KNN algorithm according to their preferences scores. Each action (class) is associated with a score for every user individually, which represents the users' individual preferences for adaptation behavior. For example, if k=5 and in 3 of most similar rules the action x is chosen and in 2 rules the action y is chosen, then in this case, the user's preferences score will be a decisive criterion for choosing the suitable action of the system. As a simple example, following (figure 8) is a spread of colored points (green, orange, blue, yellow) referring to four possible adaptation actions that can be performed by the system. For a new expected action (black point), the most suitable class label (action) for the current context is determined by looking up the classes of the 5-nearest neighbors (k = 5). Here, two orange points and three blue points are the nearest. Since the orange points refer to a preferred action for the user e.g. have the highest score, then the decision would be orange (the red-framed point).

Fig. 8: The KNN classifier
More formally, let us define rules according to "condition action" paradigm. Thus, a rule is formatted as: if <attributes> then predict <class>, where <attributes> is a conjunction of selectors referring to the context's parameters and <class> refers to the adaptation action. Selectors are represented by (ati,vi) where ati ∈ AT (AT is the set of attributes) and vi ∈ D is its value. For example, the attributes "Disability", "Experience" and "Activity" are related to the user profile. Each attribute has a set of (nominal or numeric) values (e.g. the "Experience" value ∈  The objective of this algorithm is to achieve user satisfaction by getting subsequent adaptation results, which will be closer to its preferences. More precisely, user satisfaction can be reached when the system's adaptation behavior mismatches the user's expectations. User preferences are reified as different weights of the classes so that a class with higher weight will more probably be satisfied. Different methods of rating such as Ordinal rating (Star-rating), Interval rating (numeric value) or Nominal rating (like relevant, "irrelevant" can compute user preferences. In our work, we adopt the second method. Our approach allows users to implicitly interact with the learning models in order to ensure that practice adaptation results do not deviate from their preferences. Therefore, we have modeled the adaptation learning problem as a multi-label classification problem where several adaptation actions can be considered by the system but only the one that simultaneously is major voted and have the highest score referring to user preferences will be performed. To validate our approach, we have implemented this algorithm and compared its performances with those of the SVM and the AdaBoost algorithms as they recommended for multilabel classification problems. First experimental results show that our approach can achieve increased prediction performance with respect to precision, recall and F-measure. Figure 10

A Case Study: Accident Report
To evaluate the efficacy of DNA to support the adaptation of generated UIs at design time, we will set up here an adaptation scenario and we will explain its results through a case study. The scenario is the following. Mr. Salah uses his car every day to go to the office. This morning, he hadn't paid attention while driving and unfortunately, he got into a car accident! His car was damaged and he needs to report that to his insurance company. Fig.  11 shows the interface he used. Other customized versions were also delivered to some users to fit more their profiles. The illustration below gives you an idea about the application interface once adapted to Mr. Salah Profile (see figure 5 for Mr. Salah Profile).  Three actions of adaptation are illustrated by the figure above: 1) according to the Mr.'s age, labels take large sizes. Therefore, scrollpanes are added to provide a scrollable view of the interface 2) background color and font color are customized according to his preferences 3) due to his low level of expertise, tooltips are added giving him some help to fill in the form fields.
Otherwise, to prove the validation of our approach, we have also tested the execution of some adaptation rules. We give below the rules descriptions and the respective resulted UIs (Fig. 13 a), b)).

Rule 1
Id: "AtWork" Condition: the user location is "At Work" Action: remove all optional tasks Description: the adaptation engine will browse the Tasks Model in order to remove all the optional tasks.

Rule 2
Id: "ScreenSizeRule" Condition: small screen (smartphone) Action: replace N interactors by 1 Description: the RadioButtons respective to the "Number of Injuries" and the "Damage" are replaced by two ComboBox having the same options.

Rule 3
Id: "DrivingRule" Condition: the user activity is "Driving" Action: add vocal/audio interactors. Description: according to this rule, the adaptation engine will assign to each input interactor a vocal input recorder and to each output interactor an audio output component.