Written by Anna Chapman
10 July
Testing functionality of the system; Martin built a very quick mock-up patch to test the basic principles of connecting drawing and sound. I was able to draw a cup of coffee to the accompaniment of white noise and we went out for a coffee.
11 July
Pressure sensitivity
Martin developed a way of drawing better (i.e. more sensitive) lines. Pressure sensitivity was mapped to point size, transparency of the line and Z dimension (or depth) in the openGL environment.
Tilt of the pen
Martin also enabled us to rotate the drawing within a 3D environment using the tilt of the tablet pen. This could be really interesting when used subtly: a static image of a place could suddenly shift slightly. The viewer’s position is then altered.
Cursor
I asked Martin for a cursor to allow for drawings where you need to locate where you are on the screen. He created a discreet spherical cursor, which is great; the only problem now is that it doesn’t always behave itself. There are spots in the drawing where the cursor doesn’t want to go, and the line has a kind of tantrum and flicks back at you again and again. It’s a bit like trying to paint with water-based medium over an oily surface – nothing adheres.
Line To command
Martin got rid of the lines that connect your movements! This is very satisfying, as now the drawings can start to be drawings that relate to your movements, rather than simply resembling a computer mapping data…
12 July
I drew the horseman monument from a postcard picked up in a junk shop downtown. We found that if we start with the viewing angle rotated through 90”, then as I draw it’s not apparent what I’m drawing. Then I can tilt the image and suddenly the mess of line turns into a horseman. We had fun making him gallop about using the rotate features of Martin’s software [see mini-clips].
We decided that starting by drawing postcard views was a good way to explore and develop our tools: it’s a connection to place at an instant level, it removes a lot of choice and it involves recognisable elements in perspective so that we can play with recognition and it’s disruption.
Martin looked into smoothing the line through some kind of anti-aliasing; this is a tough one.
We found out how to hide the menu etc. in Illustrator, making this a possible interface for performance now. The tool icons are still visible but changing between them can be done with shortcut keys, eliminating need for any tool palettes on screen.
We’ll continue to develop and use the Max interface, but knowing we can also employ Illustrator is a great second option, particularly as it may be impossible to recreate its smoothness/quality of line in the Max interface. I’m also happy because I can get going with techniques and concepts in Illustrator (working with layers) rather than having to wait for tools to be developed.
Plan for tomorrow: record some sounds. Create a bank of drawing sounds – finger on a pane of glass, stone across a rock, stylus on wood. Take drawing, sonically at least, beyond the computer and into our surroundings.
July 13
Martin worked on dividing the screen into zones so that working in different parts of the screen can result in different sound worlds.
We started playing with sound connections more – using pressure sensitivity of the stylus to increase grain (granular synthesis) or volume and looking at different relationships between speed of movement in drawing and how fast the sound changes. Martin refined the sound entry at initial contact with the tablet so that it comes in less bluntly.
I created a bank of tourism images, scanning and cropping details from Tourism board brochures. I think we could use these in different ways: together with real experienced sound recordings (gap between expectation and experience of places) and more abstractly, extracting shapes and forms from them to use in conjunction with lines in constructing drawings.
July 16-17
Over the weekend Martin developed interface ideas in three ways –
Sampling drawn gestures
Dividing the screen into boxes/zones
Spectral filtering
He has created controls for the filtering which mean that the relationship with drawing can be much less obvious, i.e. not simply moving through the sound as I draw from left to right/bottom to top.
July 18
We talked through the ideas that have come up so far and how they could be used conceptually. We came up with half a dozen potential ‘pieces’:
1
Free improv
Martin records and samples the gestures I make in drawing. As these are played back, I can interact with them, drawing over them, extending them visually, etc. We end up with the screen dense with marks.
2
Stones
A stone piece using a recorded bank of stone sounds (we will collect some stones from the shoreline tonight and record them here in the studio, out of the wind). Very simple idea: one of the sounds is randomly (or according to certain criteria) selected as I make a mark. The sounds and marks remain present, building up into quite a dense sonic and visual presence. Stopping point could be determined by the computer crashing. In drawing I could use stones as shape sources.
3
Other drawing sounds
Extend the bank of scraping sounds by recording drawing motions with different media, as discussed earlier, e.g. scissors across the desk, coat across a wall.
4
Birdsong
Martin has produced a beautiful sound sample of birdsong with spectral filtering.
As you draw, particular frequencies of it are revealed in a quite intricate and beautiful way.
I just need to come up with a visual approach for this and the piece is there.
5
Multi-knob synth
This is a complex idea for an interface M is working on using percentages of each zone (e.g. the screen divided by 64) being filled with lines. I need to wait until he has developed this further before I can imagine what implications it could have for visuals.
6
Scanner
Scanning bar piece: a static image (photograph) with point scanning it L to R and bottom to top. This produces a rhythmic ‘beep’ which is at some point interrupted by drawing which acts as an intervention in the image. Image-wise we could use an empty field, very plain, with the interruption being some unexpected mark/object in the near distance.
7
Sound Pipes
There are lots of pipes, cables and tubes lying around in the vicinity of our building. It would be nice to refer to them. I’d like to use an image of some pipes, make it a singe colour (e.g. red) then draw in that colour with semi-opacity so that the drawing is part of the image rather than sitting on top of it. So there is this static image and then a line appears as if some substance is moving through the pipe. In terms of sound we could use sounds derived from the voice (windpipe), gurgling, etc.
8
Films
Taking drawing beyond the graphics tablet into real world situations.
Drawing as moving through space, marking out territory etc.
Film a drawing event:
A woman pulling her coat across a wall over 100 yards.
A man scuffing his foot across the ground in circles from where he is sitting
(narrative implications could be developed or not)
Martin records the sound it makes; the sound may then be amplified/ enriched/ processed/ played with or added to etc;
The film and sound are combined.
9
Martin has just created a system where the line drawn reveals a film beneath.
I am very excited about this as it could be a way to get timbre into lines – using different textured films slowly moving etc.
Later
The film texture is amazing! For the first time, we are getting aesthetically pleasing results, with a feeling of depth and subtlety to the drawing.
July 25
We began by making sound recordings of drawing with stones (lava stones from the shore nearby). Martin recorded me drawing on variety of surfaces with a variety of gestures.
We can use these plain but filtering means that they can be controlled by the graphics tablet; the issue then becomes getting something that retains the organic ‘grain’ of the sounds without the throbbing of the processing.
I am imagining that we make something like 8 pieces using 8 different stones; each uses a different sound file, chosen and filtered according to the shape of the stone; the drawings respond to these qualities also so that one piece might be very slow and smooth, another loud with jagged edges, another larger and blacker, another finely detailed etc. Kind of John Cage-ish I suppose. We’ve tried black/deep greys on white, which for me feels better, and we can change the thickness of line and pressure sensitivity as appropriate.
Martin is getting some great sounds for these; I think having say 8 pieces also gives more room for him to work more interestingly (i.e. in a more composed way?) with the sounds, creating variety or progression between the pieces and within each piece having control over the filtering of particular sounds.
I’d also like to try having the sounds of each line remain, being added to rather than replaced as a new line (e.g. second outline of the shape) is drawn. As the drawn lines remain on the screen it makes sense to reflect this sonically (another option being to make the drawing temporal like the sound). The build up of visual information in parallel with sonic grain could be really interesting. Martin is now working on a looping buffer to achieve this.
…
It’s working! I think we’re both quite excited by this as now we’re starting to address the issue of time or temporality, i.e. one of the main differences between sound and drawing being that sound occurs through time whereas drawing is created in time but then remains as a complete thing.
Martin is now working on making the loop the length of the first line gesture, so that the loop or repeat is ‘scaled’ to the drawing.
Now Martin suggested what if the line draws itself again and again, as if recreated by the (looped) sound?
We’ve discovered if I do some rapid, short lines the loop is broken down and effectively we bring the piece to a close. Really useful for performance.
As well as looping, M is saving the last sonic ‘moment’ of each line so that there is a build up into a sort of chord. The stone sound files are quite limited in range of frequencies so the available notes/frequencies get exhausted relatively soon; therefore we might need to play around with changing the captured frequencies in some way to get a richer range.