A Conceptual Architecture and a Framework for Dealing with Variability in Mulsemedia Systems

Name: Estêvão Bissoli Saleme
Type: PhD thesis
Publication date: 03/12/2019
Advisor:

Name Rolesort descending
Celso Alberto Saibel Santos Advisor *

Examining board:

Name Rolesort descending
Celso Alberto Saibel Santos Advisor *
Gheorghita Ghinea Co advisor *
Debora Christina Muchaluat Saade External Examiner *
Niall Murray External Examiner *
José Gonçalves Pereira Filho Internal Examiner *
Magnos Martinello Internal Examiner *

Summary: The increasing interest in digital immersive experiences has drawn the attention of researchers into understanding human perception whilst adding sensory effects to multimedia systems such as VR (Virtual Reality) and AR (Augmented Reality) applications, multimedia players, and games. These so-called mulsemedia—multiple sensorial media—systems are capable of delivering wind, smell, vibration, among others, along with audiovisual content with the aim of enhancing users’ Quality of Experience (QoE) in areas such as entertainment, healthcare, education, culture, and marketing. To support the researchers’ investigation, there have been developed many standalone software solutions and incipient architectural proposals to bind these applications to sensory effects devices, such as wind fans, scent emitters, vibration chairs, etc. These devices, in turn, are constantly evolving, making it difficult to update applications to be compatible with them. There is little or no interoperability between software and hardware in this realm, hindering reuse in other contexts. Every time a mulsemedia application is needed, new software is built mostly from scratch. This model has proven to be demanding, time-consuming, and costly mainly because it requires researchers and developers alike to gain knowledge about new devices, connectivity, communication protocols, and other particulars. The fact is that building such systems imposes a number of challenges and requirements (which are discussed in this thesis) due mainly to their ever-evolving and heterogeneous traits. As a result, few mulsemedia systems have remained reusable to be applied to different research purposes as opposed to the use of open mulsemedia datasets. Therefore, the main contribution of this thesis is a decoupled conceptual architecture to deal with variability of scenarios in mulsemedia delivery systems, which includes recommendations to cope with the variation of end-user applications and sensory effect devices through the support and reuse of even unforeseen communication and connectivity protocols, and sensory effects metadata (SEM). To evaluate it, an open-source and robust mulsemedia framework was developed. Then, a performance assessment was carried out on communication protocols for the integration between event-based applications, WHEREby temporal restrictions play a role, and the framework. Results indicated statistically significant differences in response time providing directions for optimized integrations. Finally, a user QoE subjective evaluation comparing a monolithic mulsemedia system with this framework was undertaken with results suggesting no evinced statistically significant differences in userperceived QoE between the systems under different aspects. Therefore, it is hoped that this work fosters the area of mulsemedia and HCI (Human-Computer Interaction) in the sense that researchers can leverage either the conceptual architecture to design mulsemedia delivery systems or the framework to carry out their experiments.
Keywords: Mulsemedia systems. Multimedia applications. Variability. Conceptual architecture. Software Integration. Frameworks.

Access to document

Acesso à informação
Transparência Pública

© 2013 Universidade Federal do Espírito Santo. Todos os direitos reservados.
Av. Fernando Ferrari, 514 - Goiabeiras, Vitória - ES | CEP 29075-910