This post is part of a series we wrote after wrapping up the IRIS project.
Design notes part 1: Digital impacting Physical
When building a mixed-reality setup, the digital components impose some limits on the physical aspects. Ideally, you would like to move all components to the realm (digital/physical) in which they are the easiest and/or most effective to implement, but due to the restrictions of technologies this is not always possible. Here are some examples of how the technologies we used impacted our design choices.
Pepper’s Ghost screens
The technology has to work effectively in the box you design. As the Pepper’s Ghost effect works with a 45-degree-angled screen, this screen needs to fit the apartment and its furniture. Furthermore, the effect works best when viewed from the front. Although it has a reasonable viewing angle, it will not work at all when viewed from the sides as you will look onto the edge of the screen. This way, the box needs to be built ‘peep box style’, with only visibility from the direct front.
Most practical, the size of the box is determined by the size of the screen you use for projecting the Pepper’s Ghost. In our case we went for a 15,6“ screen, which meant the effective width of the play area inside the box could be no wider than 34,4 cm.
The box also needs to prevent too much ambient light to flood in from outside the box, as this will reduce visibility as well. In order to do so, we designed the physical box so it is fully enclosed form all sides except the front from where the players look inside. All the lights inside the box are precisely dimmed so that even when they simulate daylight conditions, they are bright enough to light the scene, yet they are dim enough not the wash out the peppers ghost effect. This part took some trial and error to figure out.
Even with these precautions, the effect works best when you avoid any light from coming into the box from behind the player. Therefore the front of the box should not face windows or lamps as this could result in reflections on the screen, spoiling the effect.
This post is part of a series we wrote after wrapping up the IRIS project.
Design notes part 2: Physical impacting Digital
Even before the lockdown measures, the design of the apartment was done digitally to help with rapid prototyping. The designs were then translated to physical form either through model-building techniques or 3D printing. Digitally modelling the physical box meant we could quickly estimate the proportions of the physical box, without having to fully build it first.
The box was built in such a way that all the electronics could be fitted inside the box’ cavities without laying a finger on any material. Besides being a really rapid way of prototyping and conducting ‘form studies’ it also is a great way to save material otherwise spent on making different iterations of the box in real life. Sidenote: it’s a fun fact that prototyping in the physical realm often saves time when building a digital solution or game. In this case it were the digital tools that provided us with quick ways to test our assumptions and prototype ideas before spending time on the physical building process.
The setup was built ‘peep-box‘ style in order to effectively use the Pepper’s Ghost effect. In a Peppers Ghost setup, the character is best able to move along the surface of the screen: along a single 2-dimensional plane parallel to the peppers ghost’ horizontal axis. We can get some some faking of depth into the third dimension (the Z axis) because of everything except the digital character being in 3 dimensions. When scaling the digital content up or down, we are generating the illusion of movement on the Z axis and thus giving ourselves some play area within the given constraints of the physical box. While doing this, you will still have to take into account the occlusion by objects around the screen. As this depends on the viewing angle of the player it takes some trial and error to make it work effectively.
Because of the peep box style setup we needed to place all elements and rooms of the apartment next to each other, which you wouldn’t do when building fully digital. For the same reason we chose to go for a single-room apartment, with the bathroom and the hallway the only other rooms. Those rooms are only used by the character in certain circumstances, most of the action is located in the living room. This meant that the apartment layout is not realistic, but actually almost 1-dimensional.
Controls and UI
A project that would be fully digital would have a separate controller, either a mouse or gamepad. As our box was designed to be fully digital as well as mixed-reality, we chose to have the same button interface in the digital version as in the physical version.
In this way we tried to mimic the physical box experience as much as possible, so the frame/interface of the physical box & its buttons where literally translated to the digital version. Even if you as a player are using a mouse controller, you would still push virtual, ‘physical‘ buttons. This would still (indirectly) give the feel of really interacting with a miniature world in a doll-like house setup.
When testing the digital overlays like UI in the physical box, it quickly became apparent that for the Pepper’s Ghost effect to work properly, certain aspects needed to be exaggerated compared to a fully digital version. For visibility, the icons that denote the needs of the character were made much larger and with thicker lines than we did later in the digital version.
In the digital version everything had to become more subtle that way. While creating the digital version from the basis of the mixed-reality setup, we discovered that the character style we had developed for overlaying the physical environment did not match the fully digital environment style at all.
On one hand, this is because of the fact that the 3d model of the character we used for the Pepper’s Ghost effect was very basic, based on a crude block-out. This was done because this resulted in the best visual readability in the physical box.
On the other hand, as in the mixed reality setup the character is small and transparent, the expressions and movements had to be exaggerated to be clearly readable to the player. When transitioning to the fully digital environment, these exaggerations came across as too cartoony and overacted. We found that people could no longer connect with the character on an emotional level any more as well.
At this point we could have put a lot of time and effort into adjusting the environment and the assets to match the cartoony style of the character. Instead, decided to completely replace the character with a more emotionally immersive one that would also fit the modelled environment. That second route to us was the obvious choice, as it would take way less time and generate a relatable and attractive character that would fit the needs of a digital version. In the end we used a concept model that we developed into the 3d model.
This post is part of a series we wrote after wrapping up the IRIS project.
When the IRIS-box project started, we aimed at developing a mixed-reality box and intended to design both the physical and digital components in parallel. Then we hit the Coronavirus lockdown-measures. This meant a couple of things for our design process, as we couldn’t…
…collaborate on the physical box in person
…playtest a physical box with the target audience
…show a prototype to our project partners.
We had to make a decision on how to continue the project. As it was unclear how the situation would develop, we chose to assume that the duration of the lockdown would be longer than the project itself. We therefore switched to a fully digital representation of the physical box. Still with the intent to move to a mixed-reality setup when possible and before the end of the project. In this way we could worry about the development of the physical components later, but would still be able to playtest with the target audience by sharing the digital version online. It also meant that we could show a fully digital prototype to the project partners, that would still give a pretty good approximation of the final product.
Moving to fully digital
During further development the digital version developed into a complete branche of the project, a digital twin of the physical object. Due to the success of the digital version and the prolonged lockdown measures, the digital version became a deliverable in its own right. It existed in the project alongside the physical version, instead of being just a digital derivative.
As mentioned, one of the biggest advantages of the fully digital version was that we could very easily share it with our target audience.
While building the fully digital version, we were able to quickly implement some designs that might not have come up in the physical development. Like the TV being on, switchable lighting inside the room and ambient lighting from outside falling into the apartment and emphasizing the day/night cycle. Some of these elements, like the TV being switched on, is definitely a bridge too far for the physical box. The fidelity of a fully 3D-rendered digital version can not be matched with the Pepper’s Ghost technology. We believe we squeezed out every bit of technological possibility we could.
Even when we went fully digital for our intermediate iterations, we still kept the physical box in mind. Ideally, the digital version should still reflect what the clients can expect from the physical experience. So keeping a close check with the actual physical box in mind is crucial. In the way we designed it, it should in theory not be a problem for players to switch from an online digital version to the physical mixed-reality box since all elements match 1:1.
Game design has broad application as design discipline because it must consider the design of underlying systems, interaction with those systems, and must approach everything with design thinking. The purpose of this post is to share insights into the game system design behind the Kanaleneiland box. Keep in mind that the perspectives on our approach have a basis in game design thinking.
System Design Goals
approach, we use design goals to focus on results and avoid technology-driven
features. Design goals are demonstratable player results driven by a
combination of the system, interaction, medium, et.
design goals for the Kanaleneiland box was to:
the benefits of Bo-Ex upgrades in cost.
children in group 7-8 can play and learn about green energy.
how Bo-Ex upgrades will affect the Kanaleneiland resident’s cost of living.
The System Breakdown
The formal elements of a game system include time, game space, agency, objects, attributes, rules, objectives, game state, and incentives.
The time of a single session is 210 seconds divided into three days and two nights and ends with a paused feedback moment. Starting a new session begins with the game state fro the previous session. The design assumes that players will play multiple sessions and estimates that a typical player will play three or more sessions, i.e., about 10 to 15 minutes, before arriving at a meaningful play experience.
Space includes the Kanaleneiland box and screen that displays the hologram effect, a.k.a. Pepper’s ghost. The system space consists of the physical control interface, the physical space in the Kanaleneiland box, and the digital space. The HUD (heads-up display) and the declarative layer consisting of the character separate the digital space. The digital area provides the most amount of real-estate to the character, which invites players to take action and undergo actions in response to the player.
Seven binary setting buttons and a turn-nob determine the player agency. The buttons are organized into two groups 1) core activities; 2) Bo-Ex upgrades. The turn-nob controls the temperature. The core activities allow the player to switch on/off lights, turn on/off the stove, plug-in/out a mobile device, and use hot water. Bo-Ex upgrades allow the players:
switch to solar power;
switch to USB plugs;
change the stove to electric;
\use a Toon (i.e., digital thermostat) to regulate temperature;
install double glass to minimal temperature regulation;
and switch all Bo-ex upgrades on/off altogether.
Objects and Attributes
The system considers the stove, shower, lights, thermostat, and charging. All these objects have a cost attribute, gas or electricity attribute, or temperature attribute. When the Bo-Ex upgrades are applied, they primarily affect the attributes by slowing down costs, changing gas to electric, or slowing the rate of the temperature change.
Game State Feedback
The player can interpret the game state by 1) monitoring the accumulative costs displayed by the HUD; 2) during the paused end-session feedback screen(s). The later provides the most in-depth insight into the game state by giving the player with varying degrees of feedback. The primary game state indicator is an aggregated grade between A+ to D, which indicates how well the player balanced the character’s needs and cost of utilities during the previous session. Additionally, individual elements that affect the character’s quality of life are also graded A+ to D to provide further details. The next level of feedback offers an overview of the costs of gas, heating, and electricity. The summary also includes the cost of rent, which never changes. Including the cost of rent demonstrates to the player that Bo-ex upgrades do not increase the rent. The last level of feedback provides an overview of previous game states in terms of final grade and costs.
The system has specific rules to handle the grades given to the player. These rules challenge the player to find the balance between costs and the quality of the character’s life. The system primarily relies on a simplified simulation based on thermal dynamics.
Because the Box’s interface has multiple interaction points, two people can play at the same time without conflict over the agency.
Incentives The system is designed as a sandbox experience. The sessions allow players to experiment without consequence. While the game state feedback encourages the player to find the optimal configurations.
A system for gaming or playing
“A game is a synthetic procedural system that stimulates regulated play.” – M. Hrehovcsik, 2010
What is the Kanaleneiland box? Is it a game? An argument for this system being a game is that it grades the player’s result from each session. The grading is a clear indication of how well the player played the game. However, because after each session, the game state remains persistent, i.e., never resets. Because the system it never resets, it misses a recognizable end-state. Many game definitions include the end-state as a requirement for a game. Others only indicate there needs to be a quantifiable outcome. The Kanaleneiland box system becomes a compelling case for thinking about what makes a game. In this post, the formal elements of a game system are used to analyze the Kanaleneiland box system. Based on the breakdown alone, a game system is recognizable. However, the results could still be labeled by some as a simulation or toy.
Today we created a list of assets that are needed to implement how the user can interact with the player avatar and the environment. Even though the scope of the project is ostensibly very limited, the amount of assets reveals that a lot of work still needs to be done.
Having a huge magnetic whiteboard is invaluable for this sort of activity as the whole team can stand around and provide input. The advantage compared to solutions such as Trello is that it’s easy to see how assets relate to the environment; however, it doesn’t show which assets are currently in development so we decided to also look into embedding Trello in Slack.