Doc Now 2011

Doc Now Logo

Spring is here. You can tell because the Doc Now 2011 website is up. As part of our degree, our graduating class is showing our work in a several-week-long festival. Most of it takes place in June, although Serena’s show is part of Contact, and opens on Thursday.

There’s tons of awesome work in the festival. I’m lucky to have spent the past year and a half studying with such smart people.

Also lucky for me, my project, Every Face In The Americans, is going up at Toronto Image Works at the beginning of June. On Saturday, David asked me what it’s like to be making flat art that doesn’t move. I like it.

Progress report

Oh my. Summer’s over, school’s begun and scholarship is in. It’s time to start paying myself to work on my thesis.

In Thesis Development class today, I was supposed to present a progress report; we ran out of time before my turn, so it goes here instead. I was a bit apprehensive about this presentation, as there’s not a lot of actual progress to show. The summer’s been spent trying to put away money (and failing) and trying to clear away some mental and physical space in anticipation of getting to work. It’s been a good exercise to just sit down and write out what’s on my mind, thesis-wise, and try and put it in some kind of logical order somebody else might understand.

So the question I need to answer (after I answer, “what have you done this summer?”) is: “Now that you’ve been doing this for a year, what is your thesis about, and what are you doing with it?”

My answer, as of 3pm today, looks like this:

My thesis project explores the work of reading and what happens when medium and technology take on some of the responsibilities traditionally assumed by the reader. This is my “cocktail description” – what I’d tell somebody at a cocktail party.

The un-cocktail elaboration on the above:

To varying degrees, the medium we use always affects the information we’re trying to convey. This is no great discovery. Sometimes the effect is subtle, and sometimes, like in Thomas Ruff’s JPEGs, it is so much more present that the work becomes as much about the medium as it is about the content. There’s a narrative, there’s a story being told, by the foregrounding of the mechanics.

What I’m interested in is the effect this has on the process of reading. The responsibility for the work of reading becomes displaced, or shared by the various agents of the communication. These days, technology plays a really big part in this displacement, and I find what happens as a result really interesting. Especially in art, the idea that these super-fast, very complex systems that are still at their core unable to really do anything with “meaning” are helping us with some of the heavy lifting of understanding is fascinating, and a great deal of fun.

My goal is to reflect this process and make something beautiful out of it.

Art that talks about this stuff

(I’m veering towards using the term “Literature review” but I want to be careful with that term)

Roee Rosen – The Confessions of Roee Rosen: Hito Steyerl showed us this in class last year. It’s an “advertisement” for The Confessions of Roee Rosen. In this video, an Israeli boy reads the entire English script phonetically, having no idea what he’s saying. (This is what we’re told – I have no idea whether or not it’s actually true.) What I love about it is how it highlights one of the steps in the mechanics of reading. At some point in the process, our brains have to do exactly the same thing – spell out the words so we can then figure out what they’re supposed to mean.

Jessica Field: Jessica Field is one of the first artists that got me thinking about the mechanics of understanding. The robots I’ve seen taht she’s built are concerned with trying to turn perception into understanding. What’s remarkable is how successful they are based on a relatively small set of instructions.

Kristan Horton – OracleThis is a piece, again, explicitly about reading. Kristan Horton built a system that turns audiobooks back into real books using speech recognition software. He uses this to retranscribe the Oddysey.

Where it’s at right now

If you have a pulse and you’ve stood still near me for more than about 5 minutes anytime during the past 6 months, you’ve heard about my photo project from last year, Every Face in the Americans. Currently, it’s the center of my thesis project.

iPhoto’s face recognition feature is designed to help us deal with the staggering amount of visual documentation we have in our lives. When you add an image to your iPhoto library, the program automatically looks for faces, and stores anything it finds in hidden files on your computer. It then offers to help you attach names to faces so you can see all of your pictures of your mom, or your friend Dave, or your cat. My book is the result of adding scans of Robert Frank’s The Americans to my iPhoto library.

Like Roee Rosen’s “victims”, iPhoto is acting out the mechanics of reading, without an awareness of the significance of what it’s saying. It’s using math to find faces, just as the boy is using phonetics to read English words, but has no understanding of the context from which it’s pulling these pictures.


The next step, for me, is to try and use these images as sources for some kind of sound piece. I’m slowly writing some software that analyzes the images and produces data in structures that I can use as the basis for sound work (music!). This is in its early stages, so I’m short on details. Essentially, the idea is to create my own “reader” to do a bit more dumb interpretation on the photos, then use that reading as a starting point in composition. I’m also slowly warming to the idea of using geographical information. All of the photos in Frank’s book are labeled by place – often, that’s the only information he gives. It’ll be interesting to see what it looks like on a map and then in music.


I’m currently considering three possibilities for presentation:

  1. a photo/sound installation: I’m beginning to print my thumbnails, and they look awesome enlarged. Not sure I can afford the whole book at that size, and wall space might be a problem. I may try them on the wall at 8×8, as they are in my book.
  2. a video piece with a focus on sound
  3. an expansion of the book to include a sound component: Right now, this is my favourite. The book would be my original book, plus a supplementary book including code, writing (as in, my thesis paper) and a CD of the recordings. How do I exhibit this publicly though?

I’m going to start updating this pretty regularly. This is the first time I’ve written this out since the spring, and it’s helping to get ideas bouncing around again.

Face Detection .02

Better face detection: colour images, saves the original frame. Also (with the help of the php script) writes some info to a database: time, location (hardcoded for now) face position and dimensions, folder name, original image name. Everything can be linked back to the original and to everything else.


The next step is to do some after-the-fact analysis of the faces. Some kind of shape recognition so I can start swapping similar faces in the originals.

I would love to make this some kind of collaborative effort, where I try to make a narrative out of the material I’m gathering, while the computer’s trying to do the same thing, using its own criteria. Hopefully we’d work well together some of the time, but also completely disagree at other times. But first I’m going to try and do something similar with sound. That’ll be what I present for class next week.

First face recognition in Processing


This Processing sketch looks for faces 4 times a second and writes a thumbnail image of every face it finds. Thanks to OpenCV and the Processing folks I’m well on the way to iPhoto’s face recognition!


My computer has a crush on Nick Stedman. I pointed it at him and two students and it took like 300 of these, and none of the students: