Category: IRIS Kanaleneiland

This post is part of a series we wrote after wrapping up the IRIS project.

Design notes part 1: Digital impacting Physical

When building a mixed-reality setup, the digital components impose some limits on the physical aspects. Ideally, you would like to move all components to the realm (digital/physical) in which they are the easiest and/or most effective to implement, but due to the restrictions of technologies this is not always possible. Here are some examples of how the technologies we used impacted our design choices.

Pepper’s Ghost screens

The technology has to work effectively in the box you design. As the Pepper’s Ghost effect works with a 45-degree-angled screen, this screen needs to fit the apartment and its furniture. Furthermore, the effect works best when viewed from the front. Although it has a reasonable viewing angle, it will not work at all when viewed from the sides as you will look onto the edge of the screen. This way, the box needs to be built ‘peep box style’, with only visibility from the direct front.

Most practical, the size of the box is determined by the size of the screen you use for projecting the Pepper’s Ghost. In our case we went for a 15,6“ screen, which meant the effective width of the play area inside the box could be no wider than 34,4 cm.

Lighting

During development we found that lighting can be done both through the digital layer and the physical layer. Atmospheric lighting can be generated by the laptop screen, but also by LEDs in the box. However, too many lights within the box will wash out the visibility of the digital layer.

The box also needs to prevent too much ambient light to flood in from outside the box, as this will reduce visibility as well. In order to do so, we designed the physical box so it is fully enclosed form all sides except the front from where the players look inside. All the lights inside the box are precisely dimmed so that even when they simulate daylight conditions, they are bright enough to light the scene, yet they are dim enough not the wash out the peppers ghost effect. This part took some trial and error to figure out.

Even with these precautions, the effect works best when you avoid any light from coming into the box from behind the player. Therefore the front of the box should not face windows or lamps as this could result in reflections on the screen, spoiling the effect.

Part 1: Mixed Realities during Lockdown Measures

Part 3: Design 2: physical impacting digital

Back to the IRIS page

This post is part of a series we wrote after wrapping up the IRIS project.

Design notes part 2: Physical impacting Digital

Even before the lockdown measures, the design of the apartment was done digitally to help with rapid prototyping. The designs were then translated to physical form either through model-building techniques or 3D printing. Digitally modelling the physical box meant we could quickly estimate the proportions of the physical box, without having to fully build it first.

The box was built in such a way that all the electronics could be fitted inside the box’ cavities without laying a finger on any material. Besides being a really rapid way of prototyping and conducting ‘form studies’ it also is a great way to save material otherwise spent on making different iterations of the box in real life. Sidenote: it’s a fun fact that prototyping in the physical realm often saves time when building a digital solution or game. In this case it were the digital tools that provided us with quick ways to test our assumptions and prototype ideas before spending time on the physical building process.

The Apartment

The setup was built ‘peep-box‘ style in order to effectively use the Pepper’s Ghost effect. In a Peppers Ghost setup, the character is best able to move along the surface of the screen: along a single 2-dimensional plane parallel to the peppers ghost’ horizontal axis. We can get some some faking of depth into the third dimension (the Z axis) because of everything except the digital character being in 3 dimensions. When scaling the digital content up or down, we are generating the illusion of movement on the Z axis and thus giving ourselves some play area within the given constraints of the physical box. While doing this, you will still have to take into account the occlusion by objects around the screen. As this depends on the viewing angle of the player it takes some trial and error to make it work effectively.

Because of the peep box style setup we needed to place all elements and rooms of the apartment next to each other, which you wouldn’t do when building fully digital. For the same reason we chose to go for a single-room apartment, with the bathroom and the hallway the only other rooms. Those rooms are only used by the character in certain circumstances, most of the action is located in the living room. This meant that the apartment layout is not realistic, but actually almost 1-dimensional.

Controls and UI

A project that would be fully digital would have a separate controller, either a mouse or gamepad. As our box was designed to be fully digital as well as mixed-reality, we chose to have the same button interface in the digital version as in the physical version.

In this way we tried to mimic the physical box experience as much as possible, so the frame/interface of the physical box & its buttons where literally translated to the digital version. Even if you as a player are using a mouse controller, you would still push virtual,  ‘physical‘ buttons. This would still (indirectly) give the feel of really interacting with a miniature world in a doll-like house setup. 

When testing the digital overlays like UI in the physical box, it quickly became apparent that for the Pepper’s Ghost effect to work properly, certain aspects needed to be exaggerated compared to a fully digital version. For visibility, the icons that denote the needs of the character were made much larger and with thicker lines than we did later in the digital version.

Character design

In the digital version everything had to become more subtle that way. While creating the digital version from the basis of the mixed-reality setup, we discovered that the character style we had developed for overlaying the physical environment did not match the fully digital environment style at all.

On one hand, this is because of the fact that the 3d model of the character we used for the Pepper’s Ghost effect was very basic, based on a crude block-out. This was done because this resulted in the best visual readability in the physical box.

On the other hand, as in the mixed reality setup the character is small and transparent, the expressions and movements had to be exaggerated to be clearly readable to the player. When transitioning to the fully digital environment, these exaggerations came across as too cartoony and overacted. We found that people could no longer connect with the character on an emotional level any more as well.

At this point we could have put a lot of time and effort into adjusting the environment and the assets to match the cartoony style of the character. Instead, decided to completely replace the character with a more emotionally immersive one that would also fit the modelled environment. That second route to us was the obvious choice, as it would take way less time and generate a relatable and  attractive character that would fit the needs of a digital version. In the end we used a concept model that we developed into the 3d model.

Part 1: Mixed Realities during Lockdown Measures

Part 2: Design 1: digital impacting physical

Back to the IRIS page

This post is part of a series we wrote after wrapping up the IRIS project.

When the IRIS-box project started, we aimed at developing a mixed-reality box and intended to design both the physical and digital components in parallel. Then we hit the Coronavirus lockdown-measures. This meant a couple of things for our design process, as we couldn’t…

• …collaborate on the physical box in person
• …playtest a physical box with the target audience
• …show a prototype to our project partners.

We had to make a decision on how to continue the project. As it was unclear how the situation would develop, we chose to assume that the duration of the lockdown would be longer than the project itself. We therefore switched to a fully digital representation of the physical box. Still with the intent to move to a mixed-reality setup when possible and before the end of the project. In this way we could worry about the development of the physical components later, but would still be able to playtest with the target audience by sharing the digital version online. It also meant that we could show a fully digital prototype to the project partners, that would still give a pretty good approximation of the final product.

Moving to fully digital

During further development the digital version developed into a complete branche of the project, a digital twin of the physical object. Due to the success of the digital version and the prolonged lockdown measures, the digital version became a deliverable in its own right. It existed in the project alongside the physical version, instead of being just a digital derivative.

As mentioned, one of the biggest advantages of the fully digital version was that we could very easily share it with our target audience.

While building the fully digital version, we were able to quickly implement some designs that might not have come up in the physical development. Like the TV being on, switchable lighting inside the room and ambient lighting from outside falling into the apartment and emphasizing the day/night cycle. Some of these elements, like the TV being switched on, is definitely a bridge too far for the physical box. The fidelity of a fully 3D-rendered digital version can not be matched with the Pepper’s Ghost technology. We believe we squeezed out every bit of technological possibility we could.

Even when we went fully digital for our intermediate iterations, we still kept the physical box in mind. Ideally, the digital version should still reflect what the clients can expect from the physical experience. So keeping a close check with the actual physical box in mind is crucial. In the way we designed it, it should in theory not be a problem for players to switch from an online digital version to the physical mixed-reality box since all elements match 1:1.

Part 2: Design 1: digital impacting physical

Back to the IRIS page