Magritte: An unknowable journey — Development of an immersive VR experience to discover the painter’s thought
Magritte: An unknowable journey, is a VR experience created in order to let people understand the philosophy behind the artworks of the famous painter René Magritte.
I wanted to write this article in order to show the way we approached to the design of an immersive VR Experience for educational purposes, what are the difficulties we encountered and eventually give some suggestion to people who want to develop their first VR experience without any prior knowledge.
Let’s start with the first issue: to develop the application we decided to use the famous (and most used) tool for developing VR apps and games: the game engine Unity 3d, a software that we have never used before and that has a very difficult learning curve (in our opinion).
And here comes the second (biggest) issue. As you may know, most of the features that can be created with this software are developed using code, in particular using the programming language C#. The team was composed completely by designers. We know a lot about UX, concept, aesthetic, UI, user research etc., but when we start to talk about code, it’s here that our anxiety begins.
YouTube, Unity Forum, Reddit and StackOverflow have become our best friends for weeks and if they weren’t there, we wouldn’t be here telling our story.
So, that being said, let’s start with our journey.
The development process followed three main stages:
- Research;
- Creation of the storyline;
- Technical development.
Differently from other kinds of projects, the consequentiality of these steps has been very important, since the research influenced the script and the storyline of the experience, which in turn influenced the final development on Unity.
Research
Everything started with a brainstorming and a deep research about the painter.
We conducted a research about his paintings and style, focusing also on his life and philosophy. We discovered that, in addition to being incredibly well made, it’s the thought behind the artworks and
the critique towards the human being and society, who compulsively seek a meaning behind the mystery
that makes his works unique.
There are many documentaries, books, articles and texts in the archives of the Magritte Foundation (https://www.magritte.be/en/), of museums and collections which explain expansively this theory and we were impressed by it.
It is a deep and interesting concept, but very difficult to be easily explained. Since we love hard challenges, we decided to start and try to share these concepts in an easy and engaging way.
Creation of the Storyline
Once we had a deep understanding in its theory and thoughts we had to decide how to narrate them, how to share the concepts to people and how to create an interactive and immersive experience, able to impress people and make them understand the contents we wanted to explain.
We conducted a deep research on other experiences with a similar aim created in AR or VR. Some interesting case studies are Dreams of Dalì, The Ochre Atelier (of Modigliani) and Threshold (this one is incredible).
Given the complexity of the concepts, we decided to go for an explorative and interactive experience, accompanied by a narrating voice. The narrator is the artist himself, that explains and guides the visitor into the experience and into his philosophy.
The whole storyline is divided in 6 environments (7 if we consider the initial menu):
- Main Menu;
- Magritte’s house;
- The unconscious;
- The Atelier;
- Essence of words;
- Symposium;
- The Empire of Lights.
In each of the environments, a key concept of Magritte’s philosophy is revealed. The user is invited to interact with objects, sounds and the environment around.
We tried to create iconic spaces, sometimes inspired by real environments (like the artist’s Atelier), through which the user could experience the concepts by himself and, eventually, be impressed by them.
The last step of this has been the creation of a script/storyboard, together with the description of all the macro elements that should have been developed like the environments, the main objects, the key interactions and the narration.
The brush
During the pre-development process, we decided to give the user a symbolic object that allows him to empathize with the artist. We came up with an artefact useful to navigate and discover the environment around him: the brush. We wanted to give the brush the power of a tool able to help the player to move inside Magritte’s head lighting up spaces, objects and unlocking different stages of the narration. The brush can be used to paint light inside the dark environments and must be used to discover hints and navigate inside the mystery, revealing an hidden meaning behind the objects, like the painter does through his paintings, expressing his concepts to the physical world.
The hints
To move from one environment to the other, or to the next piece of narration, the user has to interact with some iconic objects or move in certain locations following the affordances provided by the narration. To help him with that we developed a series of paper hints placed inside the environments. They have been written and left by the painter and they work as a persistent guide during the whole experience.
The audio
A very important characteristic of the experience is the audio. Since the story follows a narration, it had to be well designed and implemented. A great experience with a bad sound design would become hard to follow and meaningless.
We decided so to hire an American guy on Fiverr, making him read and act like an ancient man who explain his story and thought with a calm and sometimes melancholic tone of voice. It had to resemble the great Magritte that welcomes his visitors and makes them start a journey inside his head.
(The result was incredible in our opinion. You will hear it later in the article 😏)
The audio was then spatialized inside the environment (3d audio). It means that the user could understand the position of the audio source around him, enhancing the immersiveness of the whole story.
Technical development
Here started the real pain. Looking backwards we can consider it a pretty formative journey which taught us a lot about game development in Unity and, in particular, about mobile VR development. A lot of trials and errors have been done in order to reach the expected result.
VR Integration
The first thing to consider is the kind of interaction we would need to create:
- Teleportation (for long distance movements);
- Grab and collision (to interact with objects);
- Activation (to activate grabbed objects).
We decided to develop the experience for Oculus Quest 2. At that time the most used tool to develop VR experiences for Oculus was the Oculus Integration, but we decided to go with the newer XR Interaction Toolkit, that’s because we found it more complete and eventually compatible also with other non-Oculus devices.
Once we decided that, we started from the example file provided with the package to set up a VR Rig with the VR camera, the hands, the teleport ray and finally the XR Interaction Manager which allows the player to interact with the objects and the environment around. Starting from the example file allowed us (newbies) to test each single element and see what they were used for without starting from scratch.
Through a process trial and error we learned the role of each component inside the scene and how to modify them to match our needs.
At the end of this stage we were able to teleport around the environment using the left thumbstick, grab objects with the grab button and activate object’s behaviours through the trigger button of the Oculus’ controllers.
We maintained the same commands used in other VR applications in order to have a coherent interaction pattern. It resulted in a easiness of use for users already used to VR.
Environments’ recreation
The second step of the development phase has been the layout planning, and the actual design of all the assets we needed to recreate the environments of each level.
It has been the most important phase during the development since the layout, the assets and the number of them define the immersiveness of the player inside the narration and the quality of the overall experience.
We don’t hide it, most of the assets have been downloaded from the Unity Asset Store and eventually edited to match our requirements. Some of them, the most iconic and the ones we could’t find, have been 3d modelled and textured by ourselves. It includes the rooms’ architecture, the entire last scene (The Empire of Lights) and some furniture in Magritte’s Atelier.
It allowed us to save a lot of time, have high quality models and have an overall better aesthetic.
A strong limitation we faced during this phase was the fact that we had to develop the experience for an Oculus Quest 2. It is a very powerful headset for his kind, but it is still a mobile headset, which means that, compared to a PC VR, it has a lower computing power. All the assets inside our experience had to be as lightweight as possible in matter of polygons and texture resolution, in order to be run smoothly on the device and without any lag, which otherwise translates in a sense of discomfort and eventually can produce motion sickness.
At the end (before building the app) we also baked the lights of each scene in order to not have a real time lighting which can slow down a lot the headset.
Implementation of the interactions
The last step has been the implementation of the interactions, which allowed us to make every object interactable and activate animations.
We developed the triggers, animations and the overall interactiveness of the experience through multiple custom C# scripts.
UnityEvents allowed us to create flexible scripts able to be reused on multiple use cases and different assets.
The XR Interaction Toolkit provided easy to use interactors, like the XR Grab to grab objects; the XR Socket Interactor that has been used also to create an useful pocket in which the user could put the brush while not used; and finally the XR Direct Interactor.
Last but not least we had to synchronize the visual events with the audio narration and vice versa. The moving narrating voice works as a guide between the environments while explains the concepts behind the painter’s vision.
The End
After less than a month of crazy hours on our computers (around 18h a day, yes 🥲), stress, bad words, hope, frustration, surprise, happiness and A LOT of learned things, we came to an end.
Thanks to the whole team team, composed by me, Elisa Manzoni, Fedele Cavaliere and Andrea De Bernardi Magritte: An unknowable journey is finally a thing. It is an experience able, through his interactive nature and custom designed narration to explain the complex thought and concepts behind the great mind of René Magritte.
The VR format allowed us to immerse the player inside the artist’s head with more senses, moving from a classic passive learning through text, images and audio, to a real and remarkable experience of concepts, hopefully able to imprint the contents inside the user’s mind.
From reading a book, to living a unique experience.
We are currently looking for opportunities to share our experience with as many people as possible that don’t have access to a VR headset, so stay tuned for upcoming news and follow us on our social profiles.
If you want to test it by yourself and you have an Oculus Quest 2 you can do it simply by downloading the experience form the SideQuest Store, or it will be available soon on the Oculus Store.
If you do so, please share your comments and thoughts! Also, if you liked the article, you found some inspirations or you would like to share some suggestion or critiques, please write a comment!
If you are really interested in this project and you want to try it, you can contact us and we will organize a test session with you.
Thank you for reading the article! You can have a look at this epic style trailer of the experience we created to promote it on the stores.
p.s. some technical suggestions that will save you a lot of time: MacOS is great, i truly love it, but if you have a Windows machine or partition, you can directly test the experience without building it first (saving you A TON of time every time you want to try something you have just created). If you still want to develop on MacOS (like we did), save the project folder wherever you want, but NOT in a folder with iCloud Drive synchronization activated. It will temporarily delete some of your files to save disk’s space and you DON’T want that, unless you want to go crazy 🥲 . I took me a while to discover it and I wanted to kill myself during that time.
