Multimedia Authoring Class Materials Michael Wojcik

This is a collection of materials I've created for WRA 417, Multimedia Authoring, taught by Bump Halbritter at Michigan State University in Fall 2009.

Presentations

Boring Information

The first significant work I created for WRA 417 is a self-introduction titled “Boring Information” (2.6 MB Powerpoint presentation). It's quite short, only a couple of minutes long, heavy on images and animation (mostly overlays).

The slides are not entirely self-explanatory; this is after all a presentation, and it partakes of the traditional rhetorical mode of oratory. But I think it's watchable on its own. If you do watch it, I recommend letting the timed effects on each slide complete before moving on to the next one. (At some point I may revise it into an entirely self-animating presentation, with slide-transition timings.)

Speech Made Visible proposal

For the second half of the semester, Matt Penniman proposed a project to experiment with the visual representation of qualities of spoken language, such as intonation and tempo. Chris Huang and I joined him to form the SMV Group. We formalized our project proposal with this presentation (PowerPoint, 1 MB).

Projects

The Code Show

For my first project, I returned to the subject of my ATTW presentation (“Writing Code”) last spring: the problem of reading software source code, and of treating source code as a document to be read by human beings as well as by machines.

“The Code Show” is a showcase of various ideas for using multiple media to explore the source code for an application. As is so often the case with this sort of thing, I only had time to dabble in several of the areas I wanted to explore, so the result is rather uneven and far from straightforward. It includes a couple of short videos where I talk about the code or the problem of producing readable code, HTML source listings with "folding" functionality to let the reader drill down for more detail, and a couple of screencasts that describe individual pieces of functionality. There's also a fairly long "Making Of" piece that explains the theory behind it and its construction.

“The Code Show” is just a prototype — and not even a complete one, more a collection of prototypelets buzzing around a still-unfruited idea — but I think it points toward some of the ways software authors could make source code more accessible to readers.

DJ JD

DJ JD (Windows Media video, about 22MB) is the primary result of my 417 remix project — a mildly satirical look at Larry Lessig (who is, in my humble opinion, sometimes a tad overrated). To be honest, I'm not overly pleased with the end result, which turned out rather less funny than it seemed when I conceived it; and the dominant trope of responding to clips of Lessig speaking with bits of song lyrics turned out to be more awkward than clever.

But nonetheless I'm very glad to have completed the project. While I don't think remix is my genre, working in it was a great opportunity to experiment with new writing techniques. (I had a similar experience a couple of years ago in AL805 when I wrote a collage essay.) And I found the technical aspects, such as dealing with various encodings and editing audio and video, very engaging — perhaps not surprising considering my technical background. In any case, that contributed to my decision to work on the Speech Made Visible project for the second half of the semester.

Used Without Permission

“Used Without Permission: An Annotated Improprietography” is the annotated mediography of the sources I drew on in creating my second project for WRA 417, the remix movie DJ JD. Many of the citations include comments on any or all of three topics: the origin of the work, my right (or lack thereof) to use it, and how DJ JD would have been affected had I not been able to include that sample.

Like parts of The Code Show, the actual “Used Without Permission” document was generated from a more concise working format using an ad hoc toolchain I assembled for the project. There's more information about that in the introduction at the beginning of UWP.

Speech Made Visible

Speech Made Visible is a joint project I worked on with Matt Penniman and Chris Huang, for the second half of 417. It was conceived by Matt and refined in our group discussions and research. Initially, Matt was interested in ways that text had historically been annotated for singing and chanting, to indicate intended changes in pitch and so forth; he wanted to investigate how those print technologies could be incorporated into and improved on with digital text. That grew into the SMV project: to render text in ways that display some of the prosodic aspects of speech, and at least partially automate the process so that authors could use it without special expertise in either analyzing speech or in graphic design.

We proposed and delivered a prototype web application where:

  1. The user uploads a recording of someone speaking, and a transcript of the recording.
  2. The application analyzes the recording, trying to identify word breaks. For each word it records the starting and ending pitch, the average intensity, and the starting and ending time.
  3. The user has an opportunity to listen to the fragments that the analyzer identified as "words", and mark up the transcript to correct the relationship between audio fragments and actual words.
  4. Then the user is taken to the rendering page, where the transcript is converted into a series of HTML spans annotated with CSS classes for relatively low and high pitch, intensity, and speed. The user can see the text displayed using various stylesheets: some represent pitch with color, for example, while others vary the baseline position (so high-pitched text actually rises higher in the line).

Also, Chris produced a short video describing the origin and development of this project. After we finished the research phase, Matt and I worked on coding (I worked primarily on the analyzer, and Matt on the renderer), and Chris documented the process. This video was the first complete piece of documentary work he produced, but he did a number of interviews with Matt and me and collected lots of other material. (Matt and I both made some webcam recordings screencasts as we worked on the code, for example.) Chris hopes to develop more documentary videos using those materials. Obviously, I'm quite interested in this as another way of documenting the SMV sources, following along the lines of The Code Show.

Because SMV isn't quite ready for public consumption (in particular, we don't want to put links to the prototype site on a public site where they'll be indexed), I don't have a link to the working application here, though I'd be happy to provide private demonstrations. In the meantime, here are some related materials: