Main menu Close main menu Main menu Menu ×

Give Me A Sign: Developing The Narrative

A peek behind the scenes of Give Me a Sign, an installation combining aspects of Indian philosophy and practices with narratives around Artificial Intelligence and climate change

A blog entry by BeFantastic Within Fellows and creators of Give me a Sign, one of the winning commissions due to premier at FutureFantastic Festival in Bangalore March 2023, as part of the British Council’s programme, India/UK Together Season of Culture.


Artists: Diane Edwards & Upasana Nattoji Roy


We wish to highlight the significance and power of the physical gesture in expressing meaning with relation to our physical, living environment by repurposing mudras to reconnect with nature and convey the climate concerns we face. Highlighting the granular, tactile, and organic disconnect that purely virtual experiences create, we aim to not only build a bridge between human and machine, but also to build a bridge between cultures.  Gestures or mudras are powerful tools in telling stories,  they go beyond verbal communication,  opening up to a more emotive, intuitive, inner and even spiritual space;  a form of expression that an AI system may never fully decipher.  Although technological advancements and AI may help us deal with aspects of climate change, it is essential that we do not disregard other forms of knowledge and practices, which may aid in rethinking our way through the impending climate disaster. We envision this work as a form of acceptance, a tool to think through new imaginaries. 

Climate & Human Impact

The narrative is focused around human accelerated climate change. Shunya manifests as a fantastical digital entity, whose insights provide an alternative lens in challenging our understanding of climate change. Keeping in mind the tradition of science fiction it can be useful to think about such difficult and overwhelming situations through illusionary means, probing into and confronting our insecurities of what change is coming and how might we live in this new and potentially hostile environment.

We are continuing to develop our script for the character of Shunya. When an audience member selects and presents a mudra, one out of three outputs will appear on a screen in the form of a short animated video, accompanied by sound and text. Each of these clips will address a certain theme which branches from the original meaning of the Indian dance mudra. Shunya will respond with a statement, philosophy or a leading question which we are scripting in parts collaborating with ChatGPT3.

We envision this work as a form of acceptance, a tool to think through new imaginaries.

Mudras & the Theme of Climate Action

We have been researching and interpreting the meaning of the mudras further by using them as tools to explore narratives and urgent conversations around possible futures and the climate crisis.

Similar to our first prototype we are using 6 mudras to converse with Shunya:

  • Mookoola: This sign is interpreted as the bud of a flower. It can also mean to demonstrate the action of eating, or used to show the navel of a human, animal or God. Here the navel is meant to represent the connection of life — the umbilical cord. 
  • Prithvi or Mayura: Mayura means ‘Peacock’ in indian dance. And in yoga the same mudra is called Prithvi, which denotes the Earth.
  • Shikara: Shikhara is the human element.  It translates into the word ‘peak’. The peak of a mountain, the peak of civilization, the top of the food chain, man or the masculine, male Gods, Shiva, leader of the tribe/ pack / society, a phallic symbol.
  • Soochi: Soochi literally means ‘needle’, it can be used to indicate or point to something or symbolise a tool or technology. 
  • Mushthi: This sign takes the form of a fist, which can represent power, grasping at something stolen or holding a weapon. 

Pataka: The Pataka mudra means’ flag’, it can be used as a sign to stop,  or as a blessing. In dance depending on the way it is performed it can denote the sea, forest, clouds, river, wind, shield, among many others.


This project is a successor from an earlier prototype where we used Google’s Teachable Machine. The key problem with the first version that we wanted to address was the issue of running the project successfully no matter who is sitting in front of the camera, what they are wearing, and their surroundings. 

For this new iteration of Give Me a Sign we have been learning and experimenting with more robust gesture recognition Machine Learning techniques. We are exploring the possibility of hand pose recognition, where instead of the computer classifying images based on matching pixels, it can detect the position of a hand. 

The hand is broken up into ‘landmark’ points which the Machine Learning model can be trained on, mapping the arrangement of these points to our desired triggered outputs such as sound or animation. We began using libraries such as the Tensorflow/MediaPipe Handpose Model (which we found to be a better version of the ML5 Handpose model), p5.js, and MI5 Neural Network. The project needed to function fully offline, while running video and real-time skeletal hand recognition. 

Instead of the computer classifying images based on matching pixels, it can detect the position of a hand.

We want the user to have an experience beyond the basic human-computer interaction, generating the perception of communication and collaboration with the machine. By feeding it a gestural meaning and receiving a unique response, that builds a conversation between the user and the machine; a story that can be narrated depending on which gesture is presented and in which order.

User Interface: Holding Page

The holding page is a key element of the project, acting as a User Interface (UI) for interaction. Our speculative/fantastical character Shunya, is one that is learning from us and sharing their knowledge back with the user.

Currently we are aiming for this holding page to have an animated hand that cyclically runs through our six selected gestures, until an audience member is prompted to choose a gesture to hold up to the webcam, thus activating Shunya.

Visual Language

Visual language is the key to communicating the key themes of each of the selected Mudras meanings in relation to the climate change narrative.

The elements we are designing:

  • 3D Character 
  • Mask 
  • Environment – stable diffusion, void, fog, after effects
  • Video transitions

The aim is to experience beyond the basic human-computer interaction, generating the perception of communication and collaboration with the machine


We are designing 3D mask models for each of gesture, reflecting on the meaning and message we wish to convey with the gesture.

For instance, Prithvi (Earth) can correspond to ecological awareness, which corresponds to entanglement, vines, roots, and mycelium.


Early sketch of a mask which could be modelled and used for the gesture, Prithvi.

Instead of a human head – deconstructing our theme into elements that evoke them and using those to construct Shunya. 

Character Design

Early experiments creating our bespoke character within Unreal Engine’s Metahuman. We have decided not to go in this direction.

Layout Mockup

Instead of a full humanoid figure we will have an invisible body and head, with 6 alternate masks and corresponding floating hands. Text will also appear on the screen, which matches the statements, philosophy and questions which will be audible. 


For the holding page we are planning on using a wireframe single hand. Shunya however will have hands which are textured to correspond to the aesthetic of their mask. 


We are using Deepmotion to animate our character. Shunya will perform the mudra that the audience member has presented, and the way the mudra is performed takes inspiration from Indian dance and other more contemporary uses.


We are working with a sound designer, bringing together a mixture of traditional indian instruments with a more ambient synth spacey soundscape, and layered spoken word elements.


Having such a small team has been a challenge, it has been difficult to balance the technical aspects of the project alongside the more artistic and design elements. We now have Hasan on board to help us complete the programming, which has allowed us to really get sucked into the visuals, narrative and storytelling.

Give Me A Sign is the product of the BeFantastic Fellowship, an online programme fostering international collaborations between creatives in the UK and India, exploring AI technologies and creating provocative performance pieces amplified with creative AI.

As one of the BeFantastic winning commissions, Give Me A Sign will be presented at HOME in Manchester, February 25th – 26th and in March 2023 at the FutureFantastic Festival, an exciting and ambitious new AI+Art festival as part of the British Council’s India Together UK Season of Culture, celebrating the remarkable bond between two countries and exploring our cultures, our shared planet and our relationship with digital technologies that will shape our future together. Conceptualized by Jaaga’s BeFantastic (India), in association with FutureEverything.