Combining IoT and Users’ Profiles to Provide Contextualized Information and Services

Introduction

In the 1970s the mainframe was a resource shared by many people, and in the 1980s that concept changed and the era of personal computers began. But in the early 1990s, computational advances such as increased processing and storage power, size minimization, interconnectivity, increased battery capacity, wireless networking, and software technologies allowed the computer to be an integrated part in everyday life. In short, an era has come where technology is present and seamlessly integrated in the accomplishment of the most diverse tasks. Weiser (1991) called this new paradigm ubiquitous computing. Later, Lyytinen and Yoo (2002) report that the capabilities of computational devices allow for environmental information.

According to Weiser (1993), the integration of computational devices in our daily lives, and the mobility factor of entities (people, devices, services), implies several changes in the infrastructure level. Traditional operating systems are not ready to work in a dynamic environment with thousands of devices, which can come and go at any time. With this new computational paradigm, a set of issues are spelled out in the literature, such as user mobility, connectivity, device dynamism, service discovery, as well as the need to interconnect the physical environment with the computational infrastructure (Murphy, Picco, & Roman, 2001).

As technological developments are miniaturizing technology and allowing technologies that are able to process data and perform input and output actions cheaply, it has become possible to soak technology into physical spaces. This reality gave rise to the concept of technology embedded in physical spaces. Alongside the technological evolution of physical spaces, the technology that people carry with them on a daily basis has evolved substantially. The evolution of the mobile phone that gave way to the smartphone has made it possible to have, in a compact and everyday device, a panoply of technologies that can revolutionize the real use that can be given to a smartphone and a whole new world of possibilities to develop ubiquitous information systems.

This paper proposes a conceptual model for the creation of intelligent indoor spaces, capable of allowing new approaches of communication between the physical space and the users of that space. Thus, the article begins by giving an overview to the concept of smart space and reviews the main technologies and techniques of indoor positioning. Next, the proposed model is presented and discussed, ending with conclusions and final considerations about future work.

Smart Space Concept Overview

The interconnection between physical and virtual environments is a feature present in our spaces and leads to the idea that all physical space when it contains embedded with some kind of computational capabilities will produce safer, more efficient, or simply more exciting activities. According to this view, physical environments can be equipped with visual, auditory and other detection systems, interactive devices can perceive and react to users, leading to the concept of intelligent environments (Cook, Augusto, & Jakkula, 2009; Cook & Das, 2007; Kindberg & Fox, 2002; Youngblood, Heierman, Holder, & Cook, 2005). And addressing all of these constraints and complexities as applications develop entails added concerns for the developer and the user. And in the future, according to Costa (2003), heterogeneity in ubiquitous computing systems will not disappear. And the issue of research is not so much about integrating the resources, services, and other entities that are required for the application, but about planning an appropriate integration, considering the current context. For Castelli et al. (2011), the system must be able to dynamically perform this task. However, there are still research questions stated in the various articles (Costa, Yamin, & Geyer, 2008; Roman & Campbell, 2001) as the adaptation of applications to different execution contexts.

Application development, using traditional methodologies, a priori presupposes a set of requirements agreed upon by the stakeholders for the system and then implemented by software engineers. In contrast, these new spaces work in open environments where users and devices move in and out freely and informally, and each user has different expectations (Sousa et al., 2011). And according to Kindberg and Fox (Kindberg & Fox, 2002), the natural and intensive collaboration between computational elements and components of the physical world denotes a space. This is a requirement that is not considered when developing traditional applications where the execution context is limited and known a priori. But in the ubiquitous environment, physical integration is a common feature in all approaches, and where entity availability is highly dynamic, and its characteristics are not usually known a priori. However, it should be noted that each project presents approaches / solutions to achieve a set of requirements in a given domain, and assuming a set of assumptions. Although new technologies and new scenarios are presented, both in academia and in commerce, the main objective, and according to Cano (2010), does not necessarily fit in the development of new technologies, but in finding ways to integrate technologies and plan for appropriate integration.

The concept of smart space is explored and investigated in different areas. The article by Dmitry et al. (2016) presents the museum as a space imbued with technology that interacts with users. While the traditional museum has an information system that serves as an electronic archive or catalogue and as a rule, this information and system has been developed from a traditional perspective where only accredited users have access to the information, and visitors access some information and indirectly, today, vision changes dramatically. Exposures are transformed into computational objects, providing information about themselves or even interacting directly with users and other objects as users move from passive agents to active agents. Cities are also portrayed in the literature as intelligent spaces because they integrate technology in order to exploit the information and knowledge generated to improve planning and public services offered to citizens (Jose, Manuel, Marxjhony, & Maribel, 2019). And the authors point out that applications must consider that the context changes continuously, and that multiple other system requirements must be considered. Smart home is a term adopted by industry as a priority that recognizes the growing importance of technologies in the home context (Korzun, 2016), but Korzun notes that there are few targeted methodologies that take into account the study of user experience.

Associated with the creation of smart spaces is the concept of user detection. Automatic user presence detection and recognition empowers an intelligent system, making it able to respond more assertively even without explicit first user interaction. Therefore, a review is made of the main concepts of indoor positioning (i.e., technologies and techniques).

Technologies and Techniques for Develop Indoor Positioning Systems

The technology solutions within the positioning systems are widely referenced in R&D. Depending on your various goals, different accuracy needs are required. In the context of smart space implementation, we may also encounter disparate situations depending on whether the purpose is only to detect if a given user is in a given physical space (e.g. in a room) or on the contrary to understand their exact position within a given physical space.

In this context, a brief review is then made of the technologies most commonly used for indoor positioning (which are currently discriminated against) and the most important positioning techniques.

Indoor Common Technologies (Useful for Positioning)

The diversity of technologies available for indoor positioning is, nowadays, significant. Each one of them uses different kind of signals to sense the presence of a person or device in a space. Among the most used ones are the radio frequency signal based, such as Wireless-Fidelity (Wi-Fi), Bluetooth, Radio-Frequency Identification (RFID) and Near Field Communication (NFC). Others use infrared signals, sound (audible or ultrasound), image (including scenery recognition or mark recognition, e.g. QR code), or resort to mobile devices to use their inertial and magnetic sensors.

The choice of the technologies to use in an indoor positioning system will depend on several factors, including accuracy, cost, complexity, scalability, privacy and real-time requirements.

Wi-Fi as a technology is used by devices to communicate in a wireless local area network (WLAN) and is based on the IEEE 802.11 protocol family. With some differences among countries, the Wi-Fi standard operates at frequency bands (usually 2.4 GHz and 5 GHz) that do not require licensing, like in Bluetooth standard. It is one of the technologies that enables the development of an indoor location system (Kavanagh & Krishnan, 2005).

Bluetooth is also a technology for wireless communication, used in Personal Area Networks (PAN), and based on the IEEE 802.15.1 standard. It is used to connect personal devices such as peripherals, mobile phones as well as computers. Since Bluetooth is a radio frequency technology, it is possible to use Bluetooth as a technology to develop an internal location system (Park et. al., 2019).

Aiming the Internet of Things (IoT) and the deployment of a wide range of possible applications, a new Bluetooth technology was designed, the Bluetooth Low Energy (BLE), an evolution of the classical Bluetooth, but not backward-compatible, using the same 2.4 GHz radio frequencies but a simpler modulation system. As expected by its designation, the main feature of this technology is the low energy consumption, allowing considerable longevity of energy autonomy of the transmitters/receivers. Due to this feature, BLE is a good option for the development of an indoor location system. It is designed for low-flow information exchange and remains idle most of the time, making the technology more energy efficient. Several companies developed localization systems using Bluetooth Low Energy beacons as emitters (Heydon & Hunn, 2012).

Positioning Techniques

The information obtained from the diverse strategies could provide absolute, relative or proximity positioning. Following, we provide a summary description of the main positioning techniques.

Trilateration and Triangulation are techniques that evaluate the position of the user through reference points and geometry proprieties. The reference points are wireless receivers that compute the differences between the signals that each one receives from the user. In trilateration, positioning is achieved estimating the distance of the user to each of three receivers. Using more receivers will increase the precision and the technique shall be called multilateration. On the other hand, in triangulation the user’s location is estimated by computing the angles of the receiving signals, and it only required two receivers (Thomas, & Ros, 2005).

In triangulation, it can be used the Angle of Arrival (AoA) of the signal to each receiver, using directional antennas as far apart as possible so that a 360° angle can be formed. Based on trigonometric calculations, it is possible to determine the user’s location (Arafa, Jin, & Klukas, 2012).

In trilateration/multilateration, there are some possibilities to compute the distances. One is measuring the propagation time of a signal determined by the difference between the Time of Arrival (ToA) measured by the receiver and the instant the transmission was made by the sender. This approach requires accurate synchronization of sender and receiver clocks, and even a slight offset in clock times affects the calculated value (Liu. Hui. Et al, 2007).

Another one is the Time Difference of Arrival (TDoA), based on the time difference in which the signals reach multiple receptors, or the time difference in which two or more signals of different propagations speeds, such as Wi-Fi and Bluetooth, reach the same receiving device.

And, since in wireless communications the signals sent by a transmitter will be attenuated along their path to the receiver, proportionally to the travelled distance, the received signal power can be used to estimate the distance between those points. The Received Signal Strength Indicator (RSSI) is a relative measurement of the received signal power and is available in wireless networking cards. Since the ratio RSSI/distance is not linear, it is usual to combine this method with fingerprint location technique, which consists of building a signal strength map on each location where the system can estimate the user’s location. This technique is divided into two phases: one offline phase, which includes the measurement of signal strengths for each location and the creation of a table of collected points, filled with an average of the several values measured at each location; and one online phase, that deals with real-time signal strength measurements, depending on the user’s location. The user’s location is estimated by comparing the average values taken previously, in the offline phase, with the readings collected in the online phase. The best match between the readings obtained in the online phase and the averages of the offline phase readings result in the estimation of the user’s location (Le Dortz, Gain, & Zetterberg, 2012). The greater the number of measurements taken in the offline phase for each location, the better the system accuracy will be and the smaller the error in estimating the location of the user.

Dead Reckoning technique estimates the current position of a user based on the movement made since his previous known position. The movement can be represented by a vector of a given direction and magnitude and those values can be computed by a combination of inertial and magnetic sensors (e.g., accelerometer, gyroscope, magnetometer). One issue of this method is that positioning errors will be cumulative and, due to this, it is commonly combined with other methods (Yen & Ke, 2017).

Proximity Detection technique depends on several reference points (e.g., beacons) spread across the interest area that, when detected by the user, allow him to know that his location is around a given reference point. The aim of this method in not the exact location of a user but an estimation of his presence around a determined number of key points, as many as the beacons placed.

A combination of this methods can be used to minimize the drawbacks of each one or to improve the positioning accuracy.

Brief References for Indoor Positioning

Indoor positioning systems have a wide range of applications and have been the subject of considerable number of research papers, with different solution approaches and targets. Research on this subject has seen a remarkable growth lately. Just to get a sense of it, searching on academic journals papers on the subjects “indoor navigation”, “indoor positioning” or “indoor localization” on an online research platform (EBSCOhost) shows almost the double of results published in the last three years, from 2015 to 2017, than before that. However, there is still no universal solution for indoor positioning and navigation.

There are several different approaches and the more effective solutions for real-time indoor navigation systems are achieved combining several technologies and techniques (Li, Wang & Liu, 2015). Around us, there is an increasing number of devices that can help to get a better positioning, like wireless access points, Bluetooth beacons, radio-frequency (RF) tags and Internet of Things (IoT) devices. Smart environments will have more and more IoT devices that will help in positioning. Turgut Aydin & Sertbas (2016) say that the essential requirements of constructing a smart building are knowing and determining the location. It’s expected, then, that positioning accuracy will increase with the rise of connected devices. Gang & Pyun (2019) present a smartphone-based system for location services aiming three different applications: indoor social marketing, indoor navigation and tracking and indoor entertainment. The design and implementation proposed for the system combines several methods and technologies, like Bluetooth beacons, Geomagnetic field, Inertial Measurement Unit (IMU) sensors and smartphone cameras. Another hybrid solution was presented in Gomes, Sousa, Cunha, & Morais, (2018). The main goal was to contribute to a flexible and scalable location system, where the precision could be chosen and progressively deployed, from a reasonable accuracy using the existing Wi-Fi infrastructure up to a high accuracy using visual and NFC tags. The application range would be quite comprehensive and, when fully available, would be beneficial to special user groups, as the blind and visual impaired people, a group of people whose indoor positioning and orientation is of great importance, and with some research and prototypes developed for that purpose (Mansilha, Gomes, & Cunha, 2011).

Next, we propose a model that combines a multilateration based positioning system and a Radio Map system. This proposal for indoor positioning interacts with an intelligent information delivery system based on multiple aspects linked to the system and the user, using input and / or output mechanisms embedded in physical space.

Proposed Model

The proposed model main goal is to convert common spaces into intelligent and interactive spaces that communicate with users by presenting information based on the user’s location, the number of users located simultaneously in each space and the interests of different users, respecting the privacy rules of each user. Figure 1 presents the proposed model for the creation and use of smart spaces.

Fig. 1: Proposed model for support smart spaces according to users’ positioning and context

To be able to use the system, the user must have a device with Bluetooth or Wi-Fi technology (e.g.  smart band or smartphone) and a full profile registered in the user’s profile database, that will have the device MAC address. The user’s profile, having several data about the user’s personal information, interests and personal privacy policies, once the system used, will register all the interactions made by the system to each user. This allows that a historical for each user can be continuous created for improving the intelligence of the system. As soon as the Sensor & Core System starts to detect MAC addresses devices, The MAC address list will be sent to the “Intelligent Engine” module for start the algorithm:

1. The Sensor & Core System starts the MAC detection based on a process, running on RaspberryPi single-board computers spread in the various existing physical spaces, that scans Bluetooth and Wi-Fi devices. After detecting all Bluetooth and Wi-Fi devices, the user database to understand if the detected MAC addresses correspond to the users registered in the system.

1. If users are identified, the same system starts to determine the positioning of each of the registered detected devices (i.e., users) to determine their positioning, the system uses an approach based on multilateration and Radio Map techniques.

1. For identified users, the system crosses their interests, location and privacy policies. After this verification, the Intelligent Engine will analyse, from the information set in the information repository, which information should be sent to the Actuators System and how that information will be made available (through which output device).

1. The Actuators System will output the data (e.g., using a display embedded in the physical space, using audio mechanism or sending information for the users’ smartphone). In some cases, the user can input new data and /or interact with the actuators devices (e.g., user’ smartphone, or touch screen display). This action will send new information for the Sensor and Core System controller. The system will register the information sent and all interactions that have been made by the user.

Delivering contextualized information is crucial to the success of smart spaces. Still, a proper contextualized system must analyse several aspects and interests. Our model, the Users Profile & Data Repository, presented in Figure 2, has four main sub layers, two for each perspective – the user and the system – that store data about the users’ personal information, users’ privacy rules, system information and system rules. Combining all these four sub layers is essential for that the system response can be suitable and does not break any mandatory restriction (e.g., the user privacy).   

Fig. 2: Users Profile & Data Repository Main Components

The suitable system response will be the intersection of the triple combined analysis of restrictions about user detected in a physical space interests, preferred actions according the system interest and the users’ privacy policies, the system preferences and rules, as presented in the above equation.

Being: U – User Interests; A – System Intended Action; P – User Privacy Rules; S – System Rules

System Response = { (U₁, U₂, … , U_n ) Ù (A₁, A₂, … , A_n ) Ù (P₁, P₂, … , P_n ) Ù (S1, S2, …, Sn) }

The model application and operation could be synthetized in the above examples, that are only main examples and not exhaustive ones, of benefits of use:

• Personalize a certain physical space with visual and / or audio capabilities and deliver information according to the users’ intersected interests, when several users are in the same physical space – public delivering approach.

• Delivering information to a giving users’ device (e.g., smartphone) – private delivering approach rather than using a “public device” that several users could look.

• Optimize the information delivered according to the user’s degree of interest and knowledge about one information topic (e.g., different pieces of information about a painting in a museum for a user that is an expert on art or a user that only has passion for art).

• Optimize the format of the delivered information according to users’ special needs, concerning usability issues.

• Enable a more interactive and immersive experience between users and the physical space that can assume a merging perspective with the user.         

Even that smart spaces seem an unavoidable vision of the future of physical spaces, there are important reflections that should be made by researchers and the IT sector, concerning to the balance of smart space opportunities powered by IT evolution and users’ real needs and users respect for privacy. A challenge that, in our opinion, will not be an easy task.  

Conclusion and Final Remarks

Creating smart spaces allows organizations to reengineer their strategy of communication and, to users, new and more immersive experience as it becomes possible to obtain more suitable information and a more contextualized experience. In this context, organizations like, for example, cultural spaces (e.g. museums) or public services, can improve their marketing strategies, increase users’ satisfaction through a personalized interaction experience and improve their Customers Relationship Management (CRM) to a new level.

The proposed model contributes to the creation of massified smart spaces. The model’s approach is based on the assumption that users increasingly have technology that somehow can be constituted as their digital identity, which facilitates the identification of users in a given physical space through positioning techniques. On the other hand, it explores the fact that there are more and more physical spaces loaded with devices with data output and input capabilities. The combination of these two realities makes it possible to develop technological solutions to create true intelligent spaces. However, the proposed model emphasizes the privacy profile of each user. This emphasis is fundamental for us and should not be seen as a break on the creation of intelligent spaces and approaches, but on the contrary, as an enabler of solutions that are effectively usable and desirable for the end user.

The implementation of a prototype system according to the proposed model has already started but has not been discussed in this paper and it will be explored in future publications.  

Acknowledgments

UNIAG, R&D unit funded by the FCT – Portuguese Foundation for the Development of Science and Technology, Ministry of Science, Technology and Higher Education. UID/GES/4752/2019.

References

• Arafa, A., Jin, X., & Klukas, R. (2012). Wireless indoor optical positioning with a differential photosensor. IEEE Photonics Technology Letters, 24(12), 1027-1029. DOI: 10.1109/LPT.2012.2194140
• Cano, J., Cano, J., Calafate, C. T., & Manzoni, P. (2010). Experiences in Developing Ubiquitous Applications. In F. M. Neto , & Neto, P. F. (Ed.), Designing Solutions-Based Ubiquitous and Pervasive Computing: New Issues and Trends. (pp. 97-112).
• Castelli, G., Mamei, M., & Zambonelli, F. (2011). The Changing Role of Pervasive Middleware: from Discovery and Orchestration to Recommendation and Planning. Paper presented at the 8th IEEE PERCOM Workshop on Middleware and System Support for Pervasive Computing, Seattle.
• Cook, D. J., Augusto, J. C., & Jakkula, V. R. (2009). Ambient intelligence: Technologies, applications, and opportunities. Pervasive and Mobile Computing, 5(4), 277-298.
• Cook, D. J., & Das, S. K. (2007). S.: How smart are our environments? An updated look at the state of the art. Pervasive and Mobile Computing. Vol, 3, 53-73.
• Costa, C., Yamin, A., & Geyer, C. (2008). Toward a General Software Infrastructure for Ubiquitous Computing. Pervasive Computing, IEEE, 7(1), 64-73.
• Costa, P. (2003). Towards a Services Platform for Context-Aware Applications. (Master Thesis), University of Twente, Enschede.
• Dmitry, G. K., Sergey, A. M., Andrey, S. V., & Oksana, B. P. (2016). A Semantic Approach to Designing Information Services for Smart Museums. International Journal of Embedded and Real-Time Communication Systems (IJERTCS), 7(2), 15-34. doi:10.4018/IJERTCS.2016070102
• Gang, H.-S., & Pyun, J.-Y. (2019). A Smartphone Indoor Positioning System Using Hybrid Localization Technology. Energies, 12(19), 3702. https://doi.org/10.3390/en12193702
• Gomes, J.P., Sousa, J.P.; Cunha, C.R.; Morais, E.P. (2018). An indoor navigation architecture using variable data sources for blind and visually impaired persons. In In 13th Iberian Conference on Information Systems and Technologies (CISTI). Cáceres. p. 1-5. ISBN 978-989984348-6
• Heydon, R., Hunn, N. (2012). Bluetooth low energy. CSR Presentation, Bluetooth SIG https://www. bluetooth.org/DocMan/handlers/DownloadDoc.ashx.
• Jose, A., Manuel, B. S., Marxjhony, J., & Maribel, M. (2019). An Extension of the MiSCi Middleware for Smart Cities Based on Fog Computing Smart Cities and Smart Spaces: Concepts, Methodologies, Tools, and Applications (pp. 778-798). Hershey, PA, USA: IGI Global.
• Kavanagh, A., & Krishnan, S. (2005). U.S. Patent Application No. 10/712,954.
• Kindberg, T., & Fox, A. (2002). System software for ubiquitous computing. Pervasive Computing, IEEE, 1(1), 70-81.
• Korzun, D. (2016). On the Smart Spaces Approach to Semantic-Driven Design of Service-Oriented Information Systems. Paper presented at the Databases and Information Systems, Cham.
• Le Dortz, N., Gain, F., & Zetterberg, P. (2012). WiFi fingerprint indoor positioning system using probability distribution comparison. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2301-2304. DOI: 10.1109/ICASSP.2012.6288374
• Li, X., Wang, J. & Liu, C. (2015). A Bluetooth/PDR Integration Algorithm for an Indoor Positioning System. Sensors (Basel)., vol. 15, no. 10, pp. 24862–85.
• Liu, H., Darabi, H., Banerjee, P., & Liu, J. (2007). Survey of wireless indoor positioning techniques and systems. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 37(6), 1067-1080. DOI: 10.1109/TSMCC.2007.905750
• Lyytinen, K., & Yoo, Y. (2002). Issues and challenges in ubiquitous computing (Vol. 45, pp. 62-65): ACM Press.
• Mansilha, R., Gomes, João Pedro, & Cunha, C. R. (2011). A prototype for indoor orientation of blind people combining multilateration, inertial and magnetic techniques, using 802.11 networks and mobile devices. 6th Iberian Conference on Information Systems and Technologies. https://doi.org/978-989-96247-5-7
• Murphy, A. L., Picco, G. P., & Roman, G.-C. (2001). LIME: A Middleware for Physical and Logical Mobility. Paper presented at the Proceedings of the The 21st International Conference on Distributed Computing Systems.
• Park, S. M., Ryu, S. H., Lee, J. H., & Lee, E. S. (2015). U.S. Patent Application No. 29/475,301.
• Roman, M., & Campbell, R. H. (2001). A Model for Ubiquitous Applications. Retrieved from
• Sousa, J. P., Keathley, D., Le, M., Pham, L., Ryan, D., Rohira, S., . . . Williamson, S. (2011). TeC: End-User Development of Software Systems for Smart Spaces. Intl Journal of Space-Based and Situated Computing, to appear.
• Thomas, F., & Ros, L. (2005). Revisiting trilateration for robot localization. IEEE Transactions on Robotics, 21(1), 93-101. DOI: 10.1109/TRO.2004.833793
• Turgut, Z., Aydin, G. Z. G. and Sertbas, A. (2016). Indoor Localization Techniques for Smart Building Environment, Procedia Comput. Sci., vol. 83, pp. 1176–1181, Jan. 2016.
• Weiser, M. (1991). The Computer for the 21st Century. Scientific American, 265, 11.
• Yen, C. T., & Ke, C. H. (2017). Improving tracking error by dead reckoning and RSSI technologies with a fuzzy fusion scheme in indoor location. Microsystem Technologies, 24(10), 4025–4033. DOI: 10.1007/s00542-017-3614-3
• Weiser, M. (1993). Some computer science issues ubiquitous computing. COMMUNICATION OF ACM, 36.
• Youngblood, G. M., Heierman, E. O., Holder, L. B., & Cook, D. J. (2005). D.J.: Automation intelligence for the smart environment In: Proceedings of IJCAI (pp. 1513-1514).