Communicating multimodal wayfinding messages for visually impaired people via wearables

Joey van der Bie; Somaya Ben Allouch; Christina  Jaschinski

doi:10.1145/3338286.3344419

Communicating multimodal wayfinding messages for visually impaired people via wearables

Joey van der Bie, Somaya Ben Allouch, Christina Jaschinski

Digital life

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic

2 Citations (Scopus)

242 Downloads (Pure)

Abstract

People with a visual impairment (PVI) often experience difficulties with wayfinding. Current navigation applications have limited communication channels and do not provide detailed enough information to support PVI. By transmitting wayfinding information via multimodal channels and combining these with wearables, we can provide tailored information for wayfinding and reduce the cognitive load. This study presents a framework for multimodal wayfinding communication via smartwatch. The framework consists of four modalities: audio, voice, tactile and visual. Audio and voice messages are transmitted using a bone conduction headphone, keeping the ears free to focus on the environment. With a smartwatch vibrations are directed to a sensitive part of the body (i.e., the wrist), making it easier to sense the vibrations. Icons and short textual feedback are viewed on the display of the watch, allowing for hands-free navigation.

Original language	English
Title of host publication	MobileHCI '19
Subtitle of host publication	proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services
Place of Publication	New YorkNYUnited States
Publisher	Association for Computing Machinery
Pages	1-7
ISBN (Print)	9781450368254
DOIs	https://doi.org/10.1145/3338286.3344419
Publication status	Published - 1 Oct 2019
Event	MobileHCI '19 21st International Conference on Human-Computer Interaction with Mobile Devices and Services: Bridging mobile across the world - Taipei International Convention Center, Taipei, Taiwan, Province of China Duration: 1 Oct 2019 → 4 Oct 2019 Conference number: 21 https://dl.acm.org/citation.cfm?id=3338286

Publication series

Name	Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services
Publisher	ACM

Conference

Conference	MobileHCI '19 21st International Conference on Human-Computer Interaction with Mobile Devices and Services
Abbreviated title	MobileHCI'19
Country/Territory	Taiwan, Province of China
City	Taipei
Period	1/10/19 → 4/10/19
Internet address	https://dl.acm.org/citation.cfm?id=3338286

Access to Document

10.1145/3338286.3344419Licence: Other

MOBILEHCI2019_Communicating_Multimodal_Wayfinding_Messages_For_Visually_Impaired_People_Via_WearablesAuthor accepted manuscript (AAM), 4.62 MBLicence: Other

Handle.net

Persistent link

Cite this

van der Bie, J., Ben Allouch, S., & Jaschinski, C. (2019). Communicating multimodal wayfinding messages for visually impaired people via wearables. In MobileHCI '19: proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services (pp. 1-7). (Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services). Association for Computing Machinery. https://doi.org/10.1145/3338286.3344419

van der Bie, Joey ; Ben Allouch, Somaya ; Jaschinski, Christina . / Communicating multimodal wayfinding messages for visually impaired people via wearables. MobileHCI '19: proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services. New YorkNYUnited States : Association for Computing Machinery, 2019. pp. 1-7 (Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services).

@inproceedings{8a71346abfa747f5ae16163bf1d61e07,

title = "Communicating multimodal wayfinding messages for visually impaired people via wearables",

abstract = "People with a visual impairment (PVI) often experience difficulties with wayfinding. Current navigation applications have limited communication channels and do not provide detailed enough information to support PVI. By transmitting wayfinding information via multimodal channels and combining these with wearables, we can provide tailored information for wayfinding and reduce the cognitive load. This study presents a framework for multimodal wayfinding communication via smartwatch. The framework consists of four modalities: audio, voice, tactile and visual. Audio and voice messages are transmitted using a bone conduction headphone, keeping the ears free to focus on the environment. With a smartwatch vibrations are directed to a sensitive part of the body (i.e., the wrist), making it easier to sense the vibrations. Icons and short textual feedback are viewed on the display of the watch, allowing for hands-free navigation.",

keywords = "Wayfinding, Visually impaired, Assistive technology, Smartphone, Smartwatch, Multimodal feedback, Tactile, Bone conduction, Ubiquitous computing",

author = "{van der Bie}, Joey and {Ben Allouch}, Somaya and Christina Jaschinski",

year = "2019",

month = oct,

day = "1",

doi = "10.1145/3338286.3344419",

language = "English",

isbn = "9781450368254",

series = "Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services",

publisher = "Association for Computing Machinery",

pages = "1--7",

booktitle = "MobileHCI '19",

note = "MobileHCI '19 21st International Conference on Human-Computer Interaction with Mobile Devices and Services : Bridging mobile across the world, MobileHCI'19 ; Conference date: 01-10-2019 Through 04-10-2019",

url = "https://dl.acm.org/citation.cfm?id=3338286",

}

van der Bie, J , Ben Allouch, S & Jaschinski, C 2019, Communicating multimodal wayfinding messages for visually impaired people via wearables. in MobileHCI '19: proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services. Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, Association for Computing Machinery, New YorkNYUnited States, pp. 1-7, MobileHCI '19 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, Taipei, Taiwan, Province of China, 1/10/19. https://doi.org/10.1145/3338286.3344419

Communicating multimodal wayfinding messages for visually impaired people via wearables. / van der Bie, Joey ; Ben Allouch, Somaya; Jaschinski, Christina .
MobileHCI '19: proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services. New YorkNYUnited States: Association for Computing Machinery, 2019. p. 1-7 (Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic

TY - GEN

T1 - Communicating multimodal wayfinding messages for visually impaired people via wearables

AU - van der Bie, Joey

AU - Ben Allouch, Somaya

AU - Jaschinski, Christina

N1 - Conference code: 21

PY - 2019/10/1

Y1 - 2019/10/1

N2 - People with a visual impairment (PVI) often experience difficulties with wayfinding. Current navigation applications have limited communication channels and do not provide detailed enough information to support PVI. By transmitting wayfinding information via multimodal channels and combining these with wearables, we can provide tailored information for wayfinding and reduce the cognitive load. This study presents a framework for multimodal wayfinding communication via smartwatch. The framework consists of four modalities: audio, voice, tactile and visual. Audio and voice messages are transmitted using a bone conduction headphone, keeping the ears free to focus on the environment. With a smartwatch vibrations are directed to a sensitive part of the body (i.e., the wrist), making it easier to sense the vibrations. Icons and short textual feedback are viewed on the display of the watch, allowing for hands-free navigation.

AB - People with a visual impairment (PVI) often experience difficulties with wayfinding. Current navigation applications have limited communication channels and do not provide detailed enough information to support PVI. By transmitting wayfinding information via multimodal channels and combining these with wearables, we can provide tailored information for wayfinding and reduce the cognitive load. This study presents a framework for multimodal wayfinding communication via smartwatch. The framework consists of four modalities: audio, voice, tactile and visual. Audio and voice messages are transmitted using a bone conduction headphone, keeping the ears free to focus on the environment. With a smartwatch vibrations are directed to a sensitive part of the body (i.e., the wrist), making it easier to sense the vibrations. Icons and short textual feedback are viewed on the display of the watch, allowing for hands-free navigation.

KW - Wayfinding

KW - Visually impaired

KW - Assistive technology

KW - Smartphone

KW - Smartwatch

KW - Multimodal feedback

KW - Tactile

KW - Bone conduction

KW - Ubiquitous computing

UR - http://www.scopus.com/inward/record.url?scp=85073511403&partnerID=8YFLogxK

U2 - 10.1145/3338286.3344419

DO - 10.1145/3338286.3344419

M3 - Conference contribution

SN - 9781450368254

T3 - Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services

SP - 1

EP - 7

BT - MobileHCI '19

PB - Association for Computing Machinery

CY - New YorkNYUnited States

T2 - MobileHCI '19 21st International Conference on Human-Computer Interaction with Mobile Devices and Services

Y2 - 1 October 2019 through 4 October 2019

ER -

van der Bie J , Ben Allouch S, Jaschinski C. Communicating multimodal wayfinding messages for visually impaired people via wearables. In MobileHCI '19: proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services. New YorkNYUnited States: Association for Computing Machinery. 2019. p. 1-7. (Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services). doi: 10.1145/3338286.3344419

Communicating multimodal wayfinding messages for visually impaired people via wearables

Abstract

Publication series

Conference

Access to Document

Handle.net

Other files and links

Fingerprint

Cite this