PhD

an early saturday start to attend the ‘audience through time’ conference organised by the drama department at queen mary. it was a good effort, and my chairing of the ‘technology and liveness’ panel seemed to go down well — phew. i especially enjoyed martin barker’s talk, which was spot-on topic for me and presented with gusto: motivated by the issue of ‘liveness’ it started by asking how do audiences make sense of and respond to the near-live quality of streamed performances in cinemas, but soon progressed to an empirically backed provocation of a ‘scandal to theory’ that really showed the value of crossing disciplines.

its interesting seeing the different conventions of the disciplines at play, and i still cannot reconcile my love of the debate in drama seminars i have attended with the seeming pointlessness of reading out densely worded performance theory papers verbatim to a darkened hall (ref. my aside about auslander). something to ponder more, for i am one of the organisers of another conference on audience coming this may…

to newcastle to the all-hands meeting of the digital economy programme, aka where my funding comes from. the summer had seen an internship project use the data i had generated in my bbc internship the year before, and from this me and saul got talking about how that could go further. importantly for me, in the demonstration proposal for it, the first bit of writing i’ve been happy with.

it also meant i could get back to the visualisation i’d made for the bbc project, and tie the display of the story-world information to the playing video. and my, how browsers have come on: processing.js, fonts and the canvas element are now happy bedfellows, and will happily alpha-blend over smooth playback of a movie. check it on github

Media consumption is increasingly networked, with richer experiences requiring ever-richer metadata to provide context and so link-ability. However creating meaningful metadata for rich media such as TV programming is fraught with practical and philosophical issues, starting with just who has the time to make it anyway. Through two Media & Arts Technology DTC internship projects – with the BBC (2010) and BT (2011) – two very different sets of metadata have been created that, representing the same TV programme, provide an interesting opportunity to investigate these issues. In one set we have a semantic, authorial representation modelling the content and narrative, in the other a free-text aggregation of mediated conversation about the programme by viewers. As the programme plays, we can compare viewers’ utterances with the TV production’s own modelling of the content.

Our demonstration will be an installation that plays the TV programme – an episode of Doctor Who – with corresponding animation juxtaposing the two sets of metadata. Our research agenda centres around the practical benefit of a mixing the two approaches in creating metadata and exploring the dissonance between the two representations. In short: how much top down do you need to make the bottom up work (or should that be the other way around?); where do attempts to map one to the other fall back to attempts to find some tractability fall back to conclusions that one or the other representation is invalid (and if so, which one – a librarian’s fantasy exposed or interactions ill-suited to being co-opted).

We would gladly host a ‘Heckle at Who’ session, where delegates will watch the TV programme and use their mobile device to contribute to the conversation around the programme. We could even turn this into semantic bingo: can we produce meaning from their utterance derived from the semantic modelling work. This would be well-suited to an evening, social activity.

after nottingham, this time its to highwire at lancaster for the yearly gathering of the digital economy doctoral training centres. it was almost demoralising: we find a green field site and a wonderful brand new dedicated building designed and constructed seemingly in the blink of an eye. not the kind of trick that is possible at queen mary, campus crammed in london’s east end.

thankfully the viva was like a good supervision session rather than a critical demolition. if only i had actually pressed the record button on the dictaphone app like i thought i had. possibly the best insight came right at the end, almost as an afterthought from my drama supervisor: its really all about attention.

in the written feedback:
The committee were impressed by the amount of work done and the quality of the literature review. This draws together some very interesting material and combines it well and shows good critical powers.

yay! ah - but these things always seem to have some kind of “subject to the usual corrections” clause. and, lo, mine does:
The committee requested that a revised submission should be made for a second review. No new reading is required, it is much more about refining the way the research issues are presented and giving a clear, coherent and tractable focus. There’s a lot of good work already done here but it would benefit from being sharpened. Specifically:
1. Produce a new section that provides a clearer definition of the research
questions and, in particular, a significant narrowing of the background concept
of ‘liveness’ to a more conceptually and empirically tractable, and thus more
focussed, issue (see below).
2. Provide a new section that explains the methodological approach and, in
particular how the initial system requirements / design will be motivated.
3. Provide some discussion of how the work will link coherently - in terms of
both key concepts and methodological strategies - between potentially diverse
field environments.

three things you don’t want together: wedding organisation, alt-wedding organisation, and writing the first-year dry-run of your PhD thesis. all so important in life; all epic on the deadline front, all with just a week between them.

first to pass: the PhD nine-month review. 10+k words, finally a title i’m happy with, and most importantly, in it a coherent research programme that articulates both the bigger picture of why i got into this in the first place and the concrete in what i am going to study. liveness is a nebulous topic, and it has been quite the journey to get to this point.

the abstract is possibly the worst thing to put here, as it was the last thing to be re-written and i was beat by that point, but it gives the flavour. and in archiving this here, when the PhD is further along i can look back an wince…

Liveness: Exploiting the here and now of us together

The concept of liveness is fundamental to our understanding of what makes performance engaging but there is little consensus about what it is. This thesis will explore the issue by focussing on the role of interaction in liveness.
A review of technological interventions in these interactions has shown novel instrumentation, new modalities, and aspirations of immersion in dialog, yet overall the picture is one of clickers and twitter backchannels: little has been informed by any attempt to understand and design for the fine-grained interactional organisation of performer, audience and audience-member.
To address this a clear and appropriate problem has been identified, against which ideas of amplifying and augmenting interactional signals, behaviours and organisational features will be explored. In short: there is no point in a lecture continuing if the delivery is incomprehensible to the students, so how does the lecturer find out, how do the students let the lecturer know? Moreover, how do they do this while maintaining the shared focus of attention that is their very reason for being there? Pervasive media will be the means, and a iterative cycle of development, deployment and formative evaluation the process.
Leveraging human-computer interaction, this research shifts the analysis from crowd computing and active spectating to the performer-audience interaction required for informed performance.

there is a full gallery of prints and photos from the open studios up on flickr. i had camera in hand pretty much the whole time (or rather with this lovely strap) and there’s a number of shots i think that get towards the spirit of what happened. but my favourite is this quiet shot, which for me tells the whole story: kinect, custom code yet in the context of an ornate frame from an age where mirrors themselves were the cutting edge of technology. for this mirror can play with your image, reconstructing and (re)animating a shadow from skeleton positional data. its also serious research: if we can systematically manipulate how your mirror image reacts, we can learn much about human-human interaction and notions like empathy. what illusion is shattered when your technologically mediated shadow decides to pick its nose?

the programme i’m part of at university — media & arts technology — is quite a leap for the department to have made. perhaps it could be characterised as they realised they had lots of technology research, but nobody around who did things with technology, so they got in some artists, hackers and whatnot in to see what would happen. but rather than create an island of ‘cool’, the plan was to embed MAT within the culture of a science and technology department. very laudable, but has been so frustrating at times: institutional intertia and so on.

qmedia open studios felt like the turning of the tide. an exhibition of work going on in under the umbrella of qmedia, it was a new way of doing things for the department, from minor victories like getting computer science to buy white art plinths and equipping a workshop as a proper hack lab, to helming conceptual shifts in how research can be linked up across the department and communicated to the public. i’m proud to have been a force behind it.

having just overhauled the ‘about the live in live cinema’ presentation for the IMAP seminar, i thought it should be quite straightforward to translate this into a 4,000 word essay — my penance for sitting in on a module from QMUL’s excellent drama department last semester. how wrong could i be, the structure to my argument turned out quite differently. all the better for it, however.

a day in a coffee shop making a major revision to the about the live in live cinema talk, and then back to USC to give it as a IMAP seminar. the host was holly willis — former editor of the amazing res magazine amongst other things — and somebody who not only is published on digital cinema and live cinema, but whose definition of live cinema is often embraced: “real-time mixing of images and sound for an audience, where the sounds and images no longer exist in a fixed and finished form but evolve as they occur, and the artist’s role becomes performative…” Holly Willis (Afterimage July/Aug 2009, Vol 37, No 1, p. 11). my talk is not about definitions however, its about exploiting the liveness, and what that could mean.

also found the adage that everything can change in two blocks is indeed true, routing from the recommended coffee shop that turned out to be closed to one that was open put me through some streets i’d prefer not to walk down, let alone with a bag packed with my digital life. the no-healthcare-mashed-up-bodies of the homeless seemingly kettled / corralled to then be routed around: it isn’t a phenomenon unique to america, but its the one that alienates me the most.

i also had my media+arts technology comrades in town for the draw of transmediale itself. being in berlin, one thing on our agenda was art+com, who have pulled the trick of a) having produced some of the most inspiring work, period (a major reference of mine: computer vision tracked video mapped scenography at the opera… in 2002) and b) doing so with a productive cohesion of commercial work and academic practice. they truly are the model for ‘industrial outreach’; it’s the direction so many universities want to go in, but it seems to me still somehow industry and academia get viewed as oppositional here.