Main menu Close main menu Main menu Menu ×
News

Where do I come from? Where do I go?: Learning to Capture the Concept

An insight into the making of an AI-generated installation waterfall using garbage and waste datasets

A blog entry by BeFantastic Within Fellows and creators of Where do I come from? Where do I go?, one of the winning commissions due to premier at FutureFantastic Festival in Bangalore March 2023, as part of the British Council’s India/UK Together Season of Culture.

 

Artists: Asli Dinc, Malavika PC, Papia Chakraborty and Pritha Kundu

Our initial idea was to have three parts to our installation: 

Part 1: A great garbage waterfall 

Part 2: Performative gestures of discarding garbage, and 

Part 3: Textual gameplay with the AI we trained to talk to the audience regarding garbage and its disposal. 

All of this eventually combined into a motion-censored interaction with an audience that could turn a potentially overwhelming massive garbage waterfall into a clear and clean flowing water body. During our mentor sessions, it was brought to our attention that our project had too many ideas and that it could board well for us to find the core of our idea, stick with it, and grow it out.

Through the references given to us during our mentor session by Jake Elwes and Madhu Nataraj we discovered that the process of documentation itself could come from the ethics and sentiment of a performer’s core of paying attention to the material, time and body they deal with – in our case, garbage and AI. Here are some of the key artists and references suggested by our mentors that left a strong impact on us:

Trevor Paglen

  • ImageNet Roulette: This project gave us an insight into wondering about the kinds of “schooling” that an AI model needs, and what kinds of datasets it can comprehend. The machine is a scientific invention, thus its classifications and categorizations are ipso facto scientific — can we stop there, or must one acknowledge that we project our ingrained human training onto the machine, which has little control over what it is fed?
  • From ‘Apple’ to ‘Anomaly’: About 30,000 individually printed photographs made up the work, which was meant to be a form of extended homage to Magritte’s ‘The Treachery of Images’ for the era of machine learning.  Taking a close look at a widely used dataset for training AI — ImageNet — it displayed the hazardous links between images and labels, provoking us to rethink how we create meaning through our datasets.

Kate Crawford and Vladan Joler

  • Anatomy of an AI:  Artificial intelligence (AI) may seem far away and abstract, but it already permeates every aspect of our daily life. ‘Anatomy of an AI’ carefully compiles and condenses this enormous volume of information into a detailed high-resolution graphic by analysing the massive networks that support the “birth, life, and death” of a single Amazon Echo smart speaker. This data visualisation helped us understand the enormous amount of resources that go into the creation, distribution, and disposal of the speaker, breaking down the otherwise strange concept of AI into something we are more familiar with.

Tehching Hsieh

  • One Year Performance: The works of this American-Vietnamese performer were brought to our attention in order to introduce us to the depth of performance analysis and the importance of performance-related documentation. His works are not particularly AI-related. On the other hand, it has a lot to do with the decision to dedicate oneself to timed documentation of the experiments that the artist decided to undergo just as an artistic exercise. Our mentor, Jake Elwes, introduced us to the world of Hsieh by using the specific example of ‘One Year Performance’ (1980-1981), in which he set up shop in one location, punched a clock every hour, and took a photo of himself on each punch.

Gaining insights from these artworks, we discussed and decided that it was the question of the garbage that concerned us most, so we unanimously decided to set aside the performative gestures and gameplay aspects of our proposal aside. Thus began our deep dive into documenting our garbage. 

Our Learnings
It has been an interesting journey for us so far on quite a few levels. First and foremost our biggest challenge has been to keep in touch and coordinate our meetings between the four of us across the globe. Initially, given each coordinator’s life and the great fear of dealing with a project that we playfully pitched for that we needed to execute now, we spent a better part of our initial learnings showing up on calls when we said we would. Even though it seems simplistic to just show up, it was harder than we expected it to be. We are relieved to say we got better at organising ourselves come the second month of the project. At this time we have hired a photographer, Ankit Banerjee, to train the three of us to photograph the kind of images we need for the AI to train so that all our images look cohesive under one dataset. 

This too might seem simple to an AI expert but to Malavika and Papia it came as quite a learning curve. Sometimes we imagine Asli Dinc nodding her head wondering what could do with our novice skills; however, with patience and some humour, we pull through.

Learning to Document
At the start, we shot the images in our own style. It took us some time to narrow down the type of image we wanted, and more importantly, to understand and acknowledge why an image needs to be a certain way and what that means for training the AI. 

For example, initially, when Malavika set up a sample DIY photo booth and went through meticulously recording about 3 weeks’ worth of her garbage, Asli Dinc sweetly but surely shot it all down. Malavika being a visual artist got excited with the shadows that the garbage was casting on the floor and the backdrop.

Most of the pictures focused on beautifully composing the garbage and taking graphically interesting pictures of it. Given her artistic interest, she was making the documentation more aesthetic than it was meant to be. Asli then pointed out that shadows confuse the AI!

Papia is new to the camera. With good advice from Ankit, Papia purchased a second-hand Lumix Panasonic digital camera for her process documentation. He then gave her exercises and she set out to start her documentation from the outdoors. She learned how to control the settings in her camera, how to manage light, how to compose the garbage and how to avoid shadows of herself falling on the garbage she was shooting. Upon seeing this set as well, Asli gave us more pointers. She told us to avoid shiny surfaces as the shine confuses the AI too. She also advised us that this was the chance for us to shoot mass garbage. We were also each shooting our images in different ratios of rectangles. One can only imagine the many colours that Asli was turning upon looking at our multiple amateur attempts in the beginning.

By this point we had learnt that shooting at 2040 pixels x 2040 pixels, at 300 dpi, RGB, and in .jpeg format was ideal for our dataset. We were on the path to creating two types of datasets: 

  1. Single Images
    This is the set that is shot indoors in an infinity booth with a mutually agreed upon preset light and camera setting (as advised by Ankit and Asli Dinc). Every few days we took pictures of our personal garbage — which we saved item by item. The preset setting of the camera and the booth allows for all the images from Papia, Asli and Malavika to be similar and makes it appear as if it comes from one voice
  2. Combination Images
    This is purely an outdoor exercise, where we take images of the garbage we find around us. This allowed us to study our surroundings and understand the states of garbage from 3 different places in our world.

We decided to each deliver a minimum of 1000 images each towards our final dataset using specific settings on our camera to garner the best image to train our AI model. Our camera settings were 25 mm lens, F-5.6 aperture, 1/80th shutter speed, 250-400 ISO depending on the camera. Depending on the kind of training needed for the AI, environment and required outcome, we learnt that the settings are subject to change.  Following is how we expected our final images to look. Yes, we finally nailed it!

DIY Photostations 

Under the patient guidance of photographer Ankit Banerjee and our project colleague, Asli Dinc, we have finally discovered exactly how to photograph for our dataset. Therefore, now all our cameras are fixed and coordinated towards the same settings and we have arrived at the final set-up for the infinity booth that we have installed in our respective studio/living spaces. As all three of us live in fairly small single rooms, which double up as our sleeping, living and studio space – given that this photo booth will have to be a permanent fixture for the coming months, we had to think on our feet to create a unit that we could live alongside with. Our respective studios/bedrooms are also part of cohabiting spaces where our housemates will have to cross our photo booth setups to access our common bathrooms at various points in the day and the night. This means our photobooth setup also has to be in a manner where we don’t bump into the rig, hurt ourselves and others or destroy our carefully positioned photo space and equipment.

The biggest space saver in this matter has been with regards to the photo backdrop stand which would typically occupy about 6 feet of width and about 3 feet of depth, which means including our camera on a tripod and space for us to stand and the lights installed, we end up using an area of 8 ft by 8 ft. We literally have to sit or stand on the edge of our beds to photograph the garbage to have enough depth of field. The next issue was that of the background screen, which is both expensive and prone to getting very dirty as we would repeatedly place dirty garbage on it, no matter how much we tried to wipe it. And then there is garbage which we cannot dust off well, like for example the cigarette butts with their ash etc.


Both these issues were tackled when we hit upon the idea of using regular canvas and stitching it up like a curtain and fixing two dowels on them, installing them on a wall and letting it drop over our respective work desks. The canvas can be wiped with a wet cloth and painted over with fairly inexpensive white paint whenever we need to clean off the dirt from our previous shoots. 

Our Reflections
Just the act of looking at our garbage on a daily basis is truly disgusting. As we collect all our personal waste and shoot it once in 3 to 4 days, we now also have to keep it safe to be documented as per our requirements. So much like the photo booth, now we also have a dedicated space where we have our dedicated waste segregation and saving space inside our studio/rooms.

We bring as much as we can to the table from used pads, and tampons to our drain hair - all of them truly disgusting to be saved and relooked at a few days later under a camera and then labeled and tagged for the AI to train.

Papia for instance now looks very closely at her waste and finds herself deeply affected by the nature in which they are thrown. Wherever she goes, even on her travels to residencies and workshops she conducts she attempts and successfully finds places where she can continue this dialogue and effort of segregation and recycling with a community. She lives in an apartment in Kolkata. She has managed to convince many residents in her apartment to participate in an exploration of dealing with their waste as a community. From digging a pit for biodegradable waste within the apartment complex grounds to establishing a timetable to collect paper and plastic from all the houses, and organizing trash pick-up companies to come and pick up the waste of everyone at fixed schedules; she has truly tried to expand her understanding.

Asli coming from Turkey and establishing her immigrant status in Berlin deals with moving from the methods of negotiating garbage from a third-world country where much like in India garbage is part of our ethos. Whereas she is now in a first-world country where every toothpick and piece of dust is segregated with utmost care. She sees this systemic change is affecting her process.

Malavika, on the other hand, has had to face her addictions which surface all too evidently as her primary waste ends up being a whopping 60 packets of cigarettes and 600 butts at the end of a month.

As she also works as an artist in residence at a studio/gallery she comes face to face with the kind of garbage an organization creates. Especially an institution where different artists live for a few months and create work. Art making creates its own kinds of waste from glass, wire, plaster of paris, cardboard, dust, dry paint, wet paint, paint buckets, covers, seeds, rusted iron, and e-waste; each artist eats according to their own diet preferences, the community kitchen is always cooking food for a minimum of 10 people per meal, the list goes on. The amount of waste generated is truly astounding. Malavika is methodically trying to establish an order within the institution to create spaces to segregate and discard waste. Despite the system not working sometimes, she is determined to get it back on track. She hopes that she can also add this garbage to our dataset.

The Path Ahead
By mid-November, we will be ready to have our first encounters with training our AI with our datasets. We are also in the process of procuring a heavy-duty computer for Asli to run the programs to create our waterfall sketch drafts. We are also consulting with new media artist Debanshu Bhaumik to create a robust and effective motion sensor. Papia has come to Bangalore so that we can practice, work and meet our resource people together for a few weeks. Papia and Malavika ride on the confidence they have in Asli’s ability to gather the reins on the training. We are waiting in great anticipation to see how this will shape up.

Between the three of us, we understand that we are at differing levels, positions and points of view regarding dealing with and looking at our garbage. When we initially decided to apply for this project, we did it to find a new way of working and looking at the world around us.

Although we realized that we would be learning how to accommodate a new medium that was well beyond our grasp and that we would have to work quite hard, what we did not anticipate was that through this project we were offering the closest representation of our lives up for judgement for the world to respond to. It feels like we are writing our personal diaries and then sharing them with everyone to see. We underestimated how personal a process this is. We cannot escape it, we are shaken, excited and grateful for this opportunity.

 

Where do I come from? Where do I go? is the product of an artistic collaboration formed during the BeFantastic Within Fellowship, an online programme fostering international connections between creatives in the UK and India, exploring AI technologies and creating provocative performance pieces amplified with creative AI.

As one of the BeFantastic winning commissions, Where do I come from? Where do I go? will premier in March 2023 at the FutureFantastic Festival, an exciting and ambitious new AI+Art festival as part of the British Council’s India Together UK Season of Culture, celebrating the remarkable bond between two countries and exploring our cultures, our shared planet and our relationship with digital technologies that will shape our future together. Conceptualized by Jaaga’s BeFantastic (India), in association with FutureEverything.

Related Projects and Articles