Material Interaction:
Sand Notebook

Interaction Design, Design Research, Material Interaction, Prototyping, Somaesthetic Design

A co-creative sand sketch is an interface to co-create a resultant sketch by two people at the time who work on their own notes or prompts. The resultant note is a representation of their engagement in the sketching/drawing process.

Roles

• Design Research
• Tangible Protoype

Methods

• Material Analysis
• Practitional Observation
• Literature Survey
• Prototyping

Tools

• Sand
• Wood
• Stretch Sensor
• Arduino
• Servo Motors

Time

• 3 Months

1. Scope

What can we learn from our interactions with material things to improve HCI and digital media?

How can we work with traditional materials such as wood, clay, fabric and talk to crafters to learn about their practices with these materials. The seen counters will serve as foundations for designs of novel interaction pieces that "follow the material".

2.1 Material Research

a) Material of Focus - Sand

Fuelled by childhood nostalgia and the feeling of interacting with sand along with the fun & fear of dirtying the place around.

Our somas not only shape our designs but also serve as a design material, to be shaped and influenced by the designs we create.” - Somaesthetic Design

The contradiction of sand being so form susceptible to sink down & yet form structures or help form structures felt interesting to me to explore.

b) What about Sand as a Material?

Figure 1. Bolten, Juergen. (2014). The Dune Model – or: How to Describe Cultures. AFS icNews Link, NY

c) Sand is forming, building, creating

2.2 Material Analysis

In my material analysis of sand, I look to refer to historical approaches to use of sand as a material and sand usage in culture. I also look to test desert sand with various moisture levels, imprint, dissolution and form.

Figure 2. From left to write, testing the physicality and from of the material through mixing with water, moulding, dissolving, imprinting

Insights

3. Practitioner Observation

a) Additive Scupturing

Figure 3.1. From left to right i) Foundation ii) Leveling iii) Mapping the height to a rough reference sketch (Gravity is sometihing they are always have be aware of, parts in air, they create supports if there isn’t one in the sketch. This is material dictating the sketch form)

b) Compacting

Figure 3.2. From left to right i) & ii) Compacting happens layer by layer, toughen ups the sand. iii) Tools used for compacting

c) Reductive Sculpturing

Figure 3.3. From left to right i) A mini sculpture acts as a prototype to check gravity  ii) Reducing the bigger layers to carve out the shape from the layers of sand iii) Replying on smaller tools for finer details

Figure 3.4. The couple worked with sand as if they were dancing around the sculpture coordinating and complementing each other. They met in 2003 in a huge sand sculpture park in Belgium and have been together ever since. They travel the world creating their sand sculptures and love meeting people from different cultures.

4. Related Works

Some related applications of using the sand material agency is the project Sand Playground where the researchers examined the possibility of utilizing a tangible interface to facilitate creative activities as an initial step toward achieving human-AI co-creation. To explore this, they conducted a preliminary study that focused on using sand as a medium for co-creation and identifying possible opportunities for AI to mediate the process.

Figure 4. Co- Creation happening between the human and AI via sand as a medium

Dina EL-Zanfaly, Yiwei Huang, and Yanwen Dong. 2022. Sand Playground: Designing Human-AI physical Interface for Co-creation in Motion. In Proceedings of the 14th Conference on Creativity and Cognition (C&C '22). Association for Computing Machinery, New York, NY, USA, 49–55. https://doi.org/10.1145/3527927.3532791

a) Questions Asked

How is sand being used as both a medium & tool for a creation? What interactions emerge when users interact with a malleable and intuitive canvas such as sand?

b) Findings

c) Key Takeaways

5. Results

Based on the works of Richard Schechner’s Performance Theory, and Matt Ratto’s Critical Making, this artifact aims to explore the possibilities of combining theory and practice, thought and action, to create something truly unique.

Schechner's performance theory [2] guided the exploration of the performative aspects of art-making, and the ways in which our creations can communicate meaning beyond just what is physically present on the page.

Ratto's critical making approach [1] explored experimenting and playing with the materials, exploring new techniques and meanings. I also explored material as medium and a manifestation through Nitsche’s discussion on:

“If media are networks of actions that include human expression, then objects can be read as manifestations of such actions, and human traces can be found in the making of such objects”

I explored the performative aspect of:

• Artmaking to communicate
• Critical making approach of playing with material to form new meanings
• The findings from the material analysis about imprints,
• Practitioner observation about a relationship forming in sand and
• Literature on that explored co-creativity on sand.

a) Initial Prototype

Figure 5.1. Exploring how co-creation can take place with an initial iteration of a notebook, in this case, only one at a time creation was possible and the sand would create exact replica of the note.

b) 2nd Iteration

Figure 5.2 a) A second iteration made out of carboard and wires to rapid prototype co-creation and verify stretch sensor output.
b) The final design of the “pen” that draws on sand with wooden box layout and inspired by 3D - printheads

c) Final Iteration

Figure 5.4. A snap of co-creation taking place on the final resultant interface.

01

When the two individuals drew on paper placed on each side of the sand box, the servo motor moved in response to their actions, resulting in a uniform stroke on the sand.

The tension is read from the Adafruit Rubber Cord conductive wire and the wire is attached to the pen tops. It is also directly done so that the tension could also be felt.  The difference in tension is mapped. The digital component is being used to cause the disruption.

02

However, when the rubber cord was pulled with unequal tensions, the pen on the servo motor began to vibrate, creating a disruption in the uniformity of the resulting figure.

The mapping are passed to the servo motor that is attached on to the ruler to act as a print head. The rubber cord and string would act as guides for the print head to make abstract shapes on the sand as a result of co-creation in both sides of the interface.

This mechanism was designed to simulate the dynamics of human relationships and interactions, where equal contributions from both parties lead to harmonious results. In contrast, unequal contributions create tension and disrupt the outcome. The mechanism is also a reference to Ingold’s view on material interaction. [6]

Figure 5.3. Show an overview of how the co-creation setup works with two papers on each side connected via stretch sensor to an Arduino. The middle box is the canvas where is co-creation results in a resultant creation.

A series of on the ground observations of the governments schools done to understand better prevelant patterns and behaviour among rural students. Below is some of the glimpses from out study done remotely and the currrent prevelant softwares.
‍

I was exploring the performative aspect of artmaking to communicate, critical making approach of playing with material to form new meanings, the findings from the material analysis about imprints, practitioner observation about a relationship forming in sand and finally a paper that explored co-creativity on sand.

6. Reflections

Figure 6.1. a)  Ma’ii Ats’áá’ Yílwoí (Coyotes Running Opposite Ways). Photograph by Donna Haraway

Figure 6.2 b) The core of the sand sketch interface

One of many ways the project made sense was how it was an interpretation of Donna Haraway’s string figures. Haraway's string figures are a metaphor for the ways in which we are all connected and entangled with each other and the world around us.

In the project, the rubber conductive thread that connects the two pens over the sandbox can be seen as a physical manifestation of this string figure metaphor.

The thread represents the connections and relationships between the two human individuals and their collaborative effort in creating a shared outcome on a non-human material.

Furthermore, the tensions created by the unequal contributions of the two individuals in the project also reflect Haraway's emphasis on the importance of recognizing and addressing power differentials in relationships.
‍

Overall, the project described can be seen as an embodiment of Haraway's ideas around interconnectedness and the importance of recognizing and addressing power dynamics in relationships and also how digital component here acts as a disruption to relationships.

7. References