Augmenting the maritime world

The HoloLens 2 has the potential of being useful in maritime settings. In my master project, I am exploring these opportunities. I'm developing a prototype for that facilitates coastal navigation using the HoloLens 2 as a means.
A field test with Bjørn Aage Krane on his vessel named “Arion”, a Bavaria 38 Cruiser. Bjørn is using the prototype that shows other vessels in the vicinity on the Microsoft HoloLens 2. He is interacting with the prototype using hand gestures. Photo: Sjoerd de Winter.
Publisert: 23. november 2021

Bergen is located on the coast of the North Sea, and this makes it a hotspot for maritime development. Combining the expertise of the city in both maritime and information technology, I am developing a prototype that improves the clarity and safety of information systems on board of vessels. Together with end users, I design and develop a system that places information, commonly found on displays fixed to the vessel itself, directly into the real world and onto the ocean surface. With this project, TekLab and the Department of Information Science and Media Studies at the University of Bergen are now moving into the maritime world with a new, exciting collaboration with The Oslo School of Architecture and Design. 

Safety on board

The ship bridge is often custom designed for every vessel that is built and consists of highly specialized equipment. The HoloLens 2 enables us to generalize this implementation by visualizing informational graphics directly onto the ocean surface. Safety on board may be improved drastically by reducing the head-down time (HDT) of the captain, whilst maximizing the time the captain looks outside. It is a challenge to design an interface that accommodates all the needs of the captain, but with the HoloLens, possibilities are endless.

Ocean Industries Concept Lab

Abel is testing designs of augmented reality (AR) components on a ship bridge that has been recreated in virtual reality at Ocean Industries Concept Lab (OICL) in Oslo. Photo: Kjetil Nordby.

I am collaborating with Ocean Industries Concept Lab (OICL), a research group led by Kjetil Nordby at The Oslo School of Architecture and Design focussing on creating a streamlined design language for ship bridges. Fortunately, OICL is not new to AR: they have tested designs of AR graphics for the ship bridge for a few years now. Combining their domain knowledge with my own background in computer science, a strong team is formed. My work will contribute to expanding our knowledge about designing for and interacting with augmented reality elements for safety-critical situations. 

Karl Robert Røttingen, captain of the research vessel Kronprins Haakon is trying out augmented reality for the first time during a participatory design workshop in a lab. We are designing AR elements together using his extensive experience at sea. Photo: Floris van den Oever.


Participatory Design

The goal is to create a functional prototype that augments navigational data of traditional equipment and extends the display area onto the oceanscape. I am aware that as an interaction designer and computer scientist, I do not have the domain knowledge to design for such a complex environment. Therefore, I collaborate with seamen directly through workshops and field trips to quickly ideate, develop and test new ideas. 

The seamen can then suggest new features and explain how those should be visualized. Later in the process, I test these new features with other experts. This ensures that I am building features that are relevant and implemented correctly. These sessions can be located both in and out of context, and this process is called participatory design or co-design.

Sailing with the HoloLens 2

To test the prototype in a real-world scenario, I reached out to the Immersive Media Experiments (IMMEKS) project in Stavanger. There I was put in contact with Bjørn Aage Krane, an experienced sailor who invited me on his sailing boat for a day to test my prototype. 

In anticipation of the sunny weather, I used sun protection film to cover the visor of the HoloLens to ensure visibility of the graphics. After an introduction to AR, we spent a beautiful day sailing. During the day we tried out the HoloLens in different lighting scenarios and ideated about how AR can augment the sailing experience. 

Bjørn called the prototype a really good start that was not difficult to use. The knowledge gained during this field trip will be manifested in the prototype and hopefully I will get the chance to test the prototype with Bjørn once more throughout this project.

Bjørn Aage Krane uses the augmented reality prototype on the HoloLens 2 on his own sailing vessel in the fjords of Stavanger. Dusk has fallen. The HoloLens is aiding Bjørn in the noticing of obstacles that have become harder to see. Photo: Abel van Beek.


A mixed reality capture from the Microsoft HoloLens 2 running the prototype that has been developed. Bjørn his left hand is being tracked in augmented reality. He is using his hand to interact with the augmented reality graphics. Photo: Bjørn Aage Krane.

Publisert: 23. november 2021
linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram