mac online apple blackjack

HSS8121: Dan Foster Smith

DanFS_Yossarian 48

HSS8121: Serena Korda


DMS8013: 6. Terrible Interactivity: From Diegetic Prototyping to the ‘Gestural Excess’

Screen Shot 2016-04-05 at 15.28.36


  • To thinking about what makes interaction successful or otherwise
  • To develop a critical position towards interaction as a cultural entity
  • To familiarise ourselves with some tools, techniques and considerations for building interactives


This session is not intended to teach you how to make good interaction (though it will certainly help). Instead I want to provide something of an overview of some different approaches to interaction from the field of Interaction Design itself, HCI, software studies and STS. We’ll look at the following:

  • Affordance (and affordance ecologies)
  • Fictionality and Prototyping
  • Embodiment
  • Public Interaction

Perhaps the most important point I want to make today is that interaction involves our bodies, and the way that this is framed is vastly important for the way that we encounter the world and the way that that is imagined in the future.


“specific combination of the properties of its substance and its surfaces taken with reference to an animal” Gibson, J. J. (1977). The Theory of Affordances. In R. Shaw & B. J (Eds.), Perceiving, Acting, and Knowing: Toward an Ecological Psychology (pp. 67–82). Book Section, Hillsdale, NJ, USA: Lawrence Erlbaum Associates. p67

‘Affordances are properties of the world that are compatible with and relevant for people’s interactions. When affordances are perceptible, they offer a direct link between perception and action; hidden and false affordances lead to mistakes. […] The notion of affordances is in many ways the epitome of the ecological approach, encapsulating ideas about ecological physics, perceptual information, and the links between perception and action. In this account, affordances are the fundamental objects of perception. People perceive the environment directly in terms of its potentials for action, without significant intermediate stages involving memory or inferences. For instance, we perceive stairways in terms of their “climbability,” a measurable property of the relationship between people and stairs.’  Gaver, W. W. (1991). Technology affordances. In Proceedings of the SIGCHI conference on Human factors in computing systems Reaching through technology – CHI ’91 (pp. 79–84). New York, New York, USA: ACM Press.

Gaver (and others) make a distinction between affordance proper and ‘perceived affordance’. An affordance is a relational property of the world. A perceived affordance is just what it sounds like – the sensory + cognitive combination in an animal that allows it to act on an affordance. He also notes how artefacts may cluster so that the affordance of one item supports another.

‘nested affordances describe affordances that are grouped in space. For instance, a handle alone only appears to afford pulling. A door alone may suggest an affordance for manipulation due to its partial separation from the wall, but not what sort of manipulation will be effective. Only by seeing the affordance of pulling the handle as nested within an affordance of pulling the door can opening the door be a perceptible affordance.’ Gaver, W. W. (1991). Technology affordances. In Proceedings of the SIGCHI conference on Human factors in computing systems Reaching through technology – CHI ’91 (pp. 79–84). New York, New York, USA: ACM Press.

Fictionality and Prototyping

The discussion above is very much centred in HCI/Interaction Design literature and framed by references to areas like cognitive psychology. It’s very much involved with the interplay between the real world and technological simulations of it (think of onscreen buttons mimicking ‘real’ buttons). IT’s all the more interesting then that recently in Interaction Design questions around the value of fiction and speculation for proposing, developing or presenting prototypes have emerged. This discussion could become very broad indeed but lets examine a couple of example prototype interactions.

Propositional Prototypes

More from MIT Tangible Media Group

Kinetic Blocks is a prototype of a new kind of tangible interaction. One of the things that’s interesting (and problematic) about this kind of interaction is that it is so ‘pure’. It is a product of a bunch of researchers sitting around and saying ‘wouldn’t it be neat if…’ Applications come afterwards – effectively post interaction design. I tend to think of this kind of interaction design prototypes as ‘propositional’. They propose a kind of interaction with the expectation that something/someone will come along and partake in the conversation.

You can find a fascinating historical critique of the MIT Media Lab (which is distinct from the Tangible Media Group) here.

 …this particular brand of humanism has always been tied to an imaginary future, it was a particular kind of inhuman humanism that began in the Architecture Machine Group and went on to flourish in the Media Lab. This philosoophy constantly invokes an imagined future human who never really comes into existence, partly because the future is ever-receding, but also because this imagined future human is only ever a privileged, highly individualized, boundary-policing, disembodied, white, western, male human.’  Lori Emerson, 2016

Diegetic Prototypes

A stage further from this is the so-called ‘diegetic prototype’. The most famous example being the ‘Minority Report’ gestural interface. Diegetic prototypes are embedded in the ‘diegesis’, the fictional context of a film. Some commenters have noted that this is a convenient backdoor to getting your prototype / proposition accepted as normal, natural maybe even inevitable.

Witnessing a technology as a naturalistic part of a cin- ematic landscape is significant, but it is not sufficient to convince audi- ences of a technology’s essential worth. Diegetic prototypes entail an additional visual and narrative rhetoric specifically framed so as to encour- age audience support for the development of the technology on the screen.’ p44


‘If scholarship in the history and sociology of technology has taught us nothing else, it has taught us that technological development is not inevitable, pre-destined or linear. Any number of obstacles can impede or alter the development of a potential technology including a lack of funding, public apathy over the need for the technology, public concerns about potential applications, or a fundamental belief that the technology will not work (Williams & Edge, 1996). For scientists and engineers, the best way to jump-start technical development is to produce a working physical pro- totype. Working physical prototypes, however, are time consuming, expen- sive and require initial funds. […] technological advocates who construct diegetic prototypes have a vested interest in conveying to audiences that these fictional technologies can and should exist in the real world. In essence, they are creating ‘pre-product placements’ for technologies that do not yet exist.’ Kirby, D. (2009). The Future is Now: Diegetic Prototypes and the Role of Popular Films in Generating Real-world Technological Development. Social Studies of Science, 40(1), 41–70. p6

A concrete example of this is Underkoffler and the his ‘g-speak’ (seriously) interface. The film apparently allowed him to sell it on to the military so in that sense I suppose we should be glad it’s so ridiculous.

‘These approaches [from potential funders] led to the funds he needed to start the company Oblong Industries and to turn his diegetic prototype into a physical prototype. This real world prototype in turn led to a development contract with defence giant Raytheon to produce gestural interface technology for the US military.7 From Underkoffler’s perspective, his work as science consultant on Minority Report was not simply a minor component in this story; his well-worked out diegetic prototype was the crucial element in the development process.’ Kirby, D. (2009). The Future is Now: Diegetic Prototypes and the Role of Popular Films in Generating Real-world Technological Development. Social Studies of Science, 40(1), 41–70. p11

The reason that diegetic prototyping as well as Design Fictions are so interesting in the context of Interaction Design is that they get a “get out of jail free” card for issues of embodiment.


Terrible interaction (such as the minority report interface) treats the body as mechanical. Not as a body, but as an animated corpse. It ignores affordance and the situated character of embodied interaction. To

demonstrate this there are always a couple of great (by which I mean terrible) kinect examples. As you watch ask yourself

  1. Whose body language is this? What culture (political, social, ethnic, professional) does it come from
  2. Who is the audience? Who is NOT the audience?

The V Motion Project from Assembly on Vimeo.

Live Looping with Ableton and Xbox Kinect from Chris Vik on Vimeo.


The Gestural Excess

contrast this:

with this

And to really bring it home lets play ‘KINECT PONG‘.  As we do so, let’s think about the ‘gestural excess’.

‘the style produced through gestural excess is a part of the social and embodied experience of digital game play, which exists in relation to — but is not indexed, measured or evaluated by — gaming technologies.’ Apperley, T. H. (2013). The body of the gamer: game art and gestural excess. Digital Creativity, 24(2), 145–156. p152

See also:
Freeman, D., Hilliges, O., Sellen, A., O’Hara, K., Izadi, S., & Wood, K. (2012). The role of physical controllers in motion video gaming. In Proceedings of the Designing Interactive Systems Conference on – DIS ’12 (p. 701). New York, New York, USA: ACM Press.
Simon, Bart, Wii are Out of Control: Bodies, Game Screens and the Production of Gestural Excess (March 5, 2009). Available at SSRN: or

Public Interaction

Terrible interaction has no environmental awareness. Reeves et al note the many fine nuances involved in public interaction (particularly in explicitly public scenarios like interactive art).

‘Our taxonomy uncovers four broad design strategies: ‘secretive,’ where manipulations and effects are largely hidden; ‘expressive,’ where they tend to be revealed enabling the spectator to fully appreciate the performer’s interaction; ‘magical,’ where effects are revealed but the manipulations that caused them are hidden; and finally ‘suspenseful,’ where manipulations are apparent but effects are only revealed as the spectator takes their turn.’ Reeves, S., Benford, S., & Malley, C. O. (2005). Designing the Spectator Experience. In Proc. CHI 2005 (pp. 741–750). Portland, Oregon: ACM Press.

One of the things that the gestural excess teaches us is that gaming cultures have an aesthetic and social character which exists effectively around the game without being recognised by it. It’s kind of a paratextual element to the game(s).

Experimenting with Interactivity

Download the live face detection example here (you’ll need to install both processing video and OpenCV using Tools->add Tool and choosing the libraries tab).

We’re going to use this example to think about what makes a good or and interaction. One of the really tricky things about computer vision is getting the lighting setup right. So lets experiment.

factors to try are:

  • camera resolution
  • framerate
  • camera positioning
  • lighting -experiment with contrast, direction, intensity, diffusion

Then some straightforward ideas to try are:

  • make something happen when faces are detected (e.g. add a background)
  • work with the face pixels themselves
  • link some action or level of activity to the number of faces detected or their proximity

Terrible interactivity

As you’re doing the above – some of the things to think about (and be prepared to talk about later) are:

  • what does face detection afford?
  • what is the difference between doing this activity by yourself or in a group?
  • what are the people watching doing?
  • where are participants looking?
  • what do the participants think they’re doing?
  • how do the participants feel about what they’re doing
  • how are participants’ bodies used?
  • what is the fiction being proposed?



Read Kirby, D. ‘The Future is Now. Diegetic Prototypes and the Role of Popular Films in Generating Real-world Technological Development’

Download and install the OpenCV library for processing (go to tools-> add tool -> Libraries and search for OpenCV). Look through the examples and have a play seeing if you can adapt them.

DMS8013: 5. 3D and Making: ‘we cannot discuss “things” outside of their shape’


I was standing inline in a farm equipment shop in Montana once, buying parts for a project, when I noticed that of seven people in the line I was the only one who had two working pairs of hands. eyes. ears or legs. Until then, I had flattered myself that I worked with my hands. Chris Csikszentmihalyi, 16 Reflective Bits about the Maker Movement.


  • To look at some tools and technologies for drawing, animating and making in 3D
  • To learn about how computers ‘see’ 3D space
  • To think about the politics of making artefacts with computers

Tools and Technologies

We are taking on (or possibly conflating) a lot in one session here. We might break down some of the tools to include:

  • Generative 3D (graphics and modelling). Software includes; Processing, Grasshopper/Rhino, Openframeworks, Cinder
  • Building software for CNC (computer numerical control) such as; Solidworks, Sketchup,
  • 3D animation/modelling/gaming e.g. Maya, 3D Studio Max, Cinema 4D, Unity

Despite this (arguable) conflation, there are a lot of things we can think about in common between (some of) them such as:

  • use of coordinate space x,y,z
  • terminology and concepts including; textures, normals, uv mapping, vertices, edges, faces
  • OPENGL (and DirectX); matrix transformation, graphics buffers, renders, lighting, cameras

Seeing 3D space

In high performance applications 3D graphics are processed on the GPU (Graphics Processing Unit) rather than the CPU (Central Processing Unit). The GPU has a frame buffer(s) – a chunk of memory for storing pixel data and a processor setup that’s good for doing a lot of things in parallel (like transforming matrix data).

The matrix

Drawing things in 3D is complicated. A lot of the more difficult things are wrapped up for us in environments like Processing or Openframeworks but sometimes it helps to have an idea what’s going on underneath. To take an example of this let’s have a look at ‘vertex winding’ (see the openframeworks docs for more).


Another example is in so-called ‘matrix transformations’.

There’s an excellent introduction into matrix translations here. If you like that kind of thing, you can also find the maths here.


“a realist guarantee for the unreal”

The industry has expended huge amounts of time and money trying to mimic the way objects behave in real life or in early art forms (such as Renaissance painting). For instance see this article about cameras, this one about lights and this about physics (specifically it’s about Box2D an ‘engine’ for recreating collisions, gravity etc behind the hit game ‘Angry Birds’).

The ‘Maker Movement’

So far we’ve mostly discussed 3D on screens but of course the development of 3D modelling tools is connected to a hugely important phenomenon – that of the modern 3D prototyping ‘fab’ lab and more broadly the ‘Maker Movement’.

Makers’ Bill of Rights

Politics and Prosumerism

One perspective on the Maker Movement is that is a manifestation of political action in the form of prosumerism. It is a reaction to the blandness and homogeneity of industrial capitalism. If we can modify, hack and create for ourselves, this is ostensibly a form of political protest. Many commenters point out though that this aspect of maker culture has been effectively co-opted by industry – in particular O’Reilly, Maker Faire and Make magazine.

‘What is called ‘making’ in North America and Europe is. frankly, a luxurious pastime of wealthy people who rightly recognise that their lives are less full because they are alienated from material culture, almost all of which is products produced by corporate interests. Sadly, rather than address the problem. makers develop a hobby that solves the symptom for them, but if anything slightly strengthens the disease.’ Chris Csikszentmihalyi, 16 Reflective Bits about the Maker Movement.

‘Socially engaged making, of necessity. is engaged in a dialectic with its alternatives: commercial and corporate mass production on the one hand, and craft on the other Even when making is about self-expression. practitioners choose this form because they are attracted to the technological product as a genre. […] Making is always a political act. even if the denotative utility of the thing made is not political.’ Chris Csikszentmihalyi, 16 Reflective Bits about the Maker Movement.

Other Material Cultures

The point about material culture though is an interesting one, for artists specifically. For some the connection between non-linear computer technologies and the capacity to create things in the physical world is a way of re-evaluating craft practice. It’s also worth noting that the intersection of traditional crafts (such as knitting) and contemporary technologies (like arduino) has proved an in-point for people who don’t necessarily identify with common tropes of computing aesthetics – e.g. chip tunes, glitch, computer vision generated imagery, projection mapping etc.

Image Varvara and Mar

…and ‘Other’ Communities

As I hinted above, one kind of value, perhaps, for the Maker Movement (or better movements) is in fostering particular kinds of community, often but not always around a particular maker space or project. For instance Kaiton Williams on Jamaican DIY describes how his father’s propensity to tinker inflected the son’s future engagement with the material world. A perceived value for the made outputs of many communities is in expressing a vision for the material world which is not produced by a narrow band of society and is consequently reflective of other kinds of value and priority. None of this is necessarily contingent on access to CNC-type tools but there is a sense in which assuming contemporary forms of production proposes a different kind of ‘answer’ to the dominance of mass produced products. A nice example to finish on is here. In this project, Kuznetsov and her co-authors build arduino based soil quality sensors with members of a community garden. Cheap and accurate commercial sensors are available but the authors describe the value of the building process in learning about the specifics of the local soil chemistry, interacting with their environment and perceiving time differently.

Further Reading

A truly excellent resource for reading about Critical Making can be found here.

Pre Task

Read the article here on matrix translations and code the accompanying examples.

Read Geert Lovink and Michael Dieter on Making in the Digital Age here. Come prepared to explain one of the theses and explain why you agree or disagree.

And also look at processing examples in the following sections:


DMS8013: 4. Algorithms and Generativity: The Map is not the Territory


  • To learn about the history of algorithms and generative computer code
  • To think about the ways that computers ‘model life’ or otherwise connect to the physical world
  • To experience creating generative systems

Algorithm (خوارزمي‎‎): “a description of the method by which a task is to be accomplished,”

History of Computer Science (and previously in mathematics)

In art/music/education Shintaro Miyazaki & Michael Chinen, Algorythmic Sorting

Literature and Poetry

The Oulipo and Exercises in Style, Raymond Queneau

Algorithmic Poetry


Formalism vs Action

The algorithm “is the unifying concept for all the activities which computer scientists engage in.” Provisionally a “de- scription of the method by which a task is to be accomplished,” the algorithm is thus the fundamental entity with which computer scientists operate.[…] But the algorithm is not simply the theoretical entity studied by computer scientists. Algorithms have a real existence embodied in the class libraries of programming languages, in the software used to render web pages in a browser (indeed, in the code used to render a browser itself on a screen), in the sorting of entries in a spreadsheet and so on.

Fuller, M. (2008). Software Studies: A Lexicon. Leonardo Books, MIT Press. p17

So (what I’ll call) the mode of material expression is vital, powerful etc.

A conception of the algorithm as a statement as Michel Foucault used the term might allow us to understand this approach a little better. For Foucault, the statement is not analytically reducible to the syntactic or semantic features of a language; it refers instead to its historical existence and the way that this historical existence accomplishes particular actions. […] As Foucault puts it in The Archaeology of Knowledge, “to speak is to do some- thing—something other than to express what one thinks, to translate what one knows, and something other than to play with the structure of language.

Fuller, M. (2008). Software Studies: A Lexicon. Leonardo Books, MIT Press. p17

Generativity: Modelling life?

In a sense we can think of the field of cybernetics as an orientation

Cybernetics: “Our bodies are hardware, our behavior software”

‘In a sense, the original purpose of Cybernetics was to produce a unified theory of the control levels and types of messages used by men and machines and processes in normal operation. Thus the history of computer technology may be interpreted as progress in making communication between men and machines more natural and complete. This remains an ideal definition however, because quite often in industry human beings have been adapted to inhuman machine schedules, rather than the other way around. What is less realized is that most businesses of any size have had to adapt themselves,more or less traumatically,to radically different patterns of administration and organization as the result of information structures made possible by computer systems. So in part Software addresses itself to the personal and social sensibilities altered by this revolution.’


‘It is now empirically clear that Darwinian evolutionary theory contained a very great error in its identification of the unit of sur-vival under natural selection. The unit which was believed to be crucial and around which the theory was set up was either the breeding individual or the family line or the subspecies or some similar homogeneous set of conspecifics. Now I suggest that the last 100 years have demonstrated empirically that if an organism or aggregate of organisms sets to work with a focus on its own survival and thinks that that is the way to select its adaptive moves, its “progress” ends up with a destroyed environ- ment. […] The flexible environment must also be included along with the flexible organism because, as I have already said, the organism which destroys its environment destroys itself. The unit of survival is a flexible organism-in-its-environment.’ Bateson, Gregory. “Form, substance, and difference.” Essential Readings in Biosemiotics (1970): 501. p508


 Software Information Technology. Its Meaning for Art.

Generative Code

Meanwhile in computer science bleeding to art practice people became interested in algorithmic modelling, generative processes on both ontological and processual levels.

Such as Conway and the game of life.

‘These artistic systems are not wholly deterministic, running an image through pre-set parameters until it reaches perfection. Indeed, Latham realized early on that the most interesting outcomes of his program were quite unforeseen by him: his evolutionary program could arrive at unexpected conclusions. Even if an artist programs the computer from the start, there will always be an important element of mystery in the working of the software. Such quirks render the computer less mechanistic (and predictable) and more “artistic,” because the outcome of certain operations cannot always be foreseen. is unpredictability can be harnessed in the same way as the chemical reactions of pigments, or the densities of stone. In other words, an artist develops a feel for its working and gradually incorporates its idiosyncrasies into their work, which itself changes subtly or overtly to accommodate these properties.’ Lambert, Nicholas, William Latham, and Frederic Fol Leymarie. “The emergence and growth of evolutionary art: 1980–1993.” ACM SIGGRAPH 2013 Art Gallery. ACM, 2013.
‘For Lev Manovich, contemporary generative art is distinctively concerned with complexity, unlike the paradigm of reduction that characterised abstraction in the visual arts in the first half of the twentieth century.’

‘Software art systems are concrete collections of objects, relations, actions and processes. In part they are formal but constructed ontologies, describing entities and their interrelations. These ontologies are partly metaphorical or figurative—constructing for example «agents» in an «environment.» They are also partly technical / textual, in the sense that the implementation of these figures occurs within the structures of a formal language with particular representational and computational limits. How do we read such systems, critically? They are literally texts, in their source code, but also in a critical sense, in that they involve specific figurations, relations, decisions, values and ideologies.’ Whitelaw, Mitchell. “System stories and model worlds: A critical approach to generative art.” Readme 100 (2005): 135-154.


Sketches from today: session_4_algorithms_and_generativity



Take my generative boids sketch and make it sing the music of the spheres. You’ll need to:

  • include processing sound library
  • decide what sounds you’re going to make
  • look in to the particles class and decide how you’re going trigger or affect the sounds. This should probably be some function of the distance between nearby particles

DMS8013: 3. Communication: What hath God wrought?



  • To experience and discuss aspects of the history of communications technology
  • To think about what we mean by communication
  • To understand more about how networks work

Communicating Information vs How to do Things with Words (a contrast)

Lets consider the history of optical communication 

Beacons and Smoke Signals

Precursors to The Chappe Optical Telegraph



How wonderful it was that these various signs should be made to cleave the air with such precision as to convey to the distance of three hundred leagues the ideas and wishes of a man sitting at a table at one end of the line to another man similarly placed at the opposite extremity, and all this affected by a simple act of volition. Count of Monte Cristo, Alexandre Dumas

Early Experiments of the Chappe Brothers

Lets recreate one of these experiments using this online clock, and a small bell.

Standage (1998, p. 9) describes how the Chappe brothers, before arriving at the solution of semaphore arms as the most efficient way of encoding and relaying messages, experimented with a combination of synchronized timing devices and color-coded discs. The recipient of the message would watch for the change between black and white and take a note of the precise position of the stop-watch, translating this number via a pre-arranged code. What is historically significant about this process, is that by tying the change of a physical state to a regular time interval, the brothers introduced two completely new spatio-temporal arrangements of materiality, which are exercised, two hundred years later, by the iPhone which forms the basis for NBM. First the regulation of information by time interval, effectively prefigured the notion of bandwidth (without which, Shannon’s seminal (2001 [1948]) work, for instance, would not have been possible). Second the notion of a regulating clock signal to manage information processing was effectively born with this invention. In integrated circuits, different chips must be able to communicate with one another at the right time and for this purpose, a clock signal is referred to. It is no exaggeration to say that without the notion of a clock signal, there would be no microprocessors and hence, no digital computers. However, I do not claim that the Chappe brothers version of serial ‘genealogically’ lead to the development of the signal clock. Instead I suggest that this material arrangement of physicality and time can be traced back to this historical moment. Schofield, Tom, Materiality and Making in Experiential Ecologies (PhD Thesis).


Claude (Elwood!) Shannon

Information Theory. Signal vs Noise. The abstraction that Shannon brought to the notion of information was exactly that which made it computable. His work with boolean logic and encryption was made possible by a particular (and instrumental) vision of information. It is not Shannon’s fault that the specialised use of the word ‘communication’ described in his work has unproblematically expanded out of its home in (the foundations) of computer science.

So far we’ve talked about the history of optical ‘communication’. To what degree do you think we are actually talking about ‘communication’. To what degree are talking about transmission / reception? What is/could be the difference?


Performativity and doing things with words (and media)

If we wish to think seriously about the media of communication, particularly in the age of computational media we need to think seriously about what we mean when we say ‘communication’.

‘Performative utterances:
a. do not ‘describe’ or ‘report’ or constate anything at all, are not ‘true or false’
b. the uttering of the sentence is, or is a part of, the doing of an action, which again would not normally be described as ‘just’ saying something.’(adapted from) 1962 Austin, John L. “How to Do Things with Words.” Oxford: Clarendon Press, 1962. p6


‘…if gender is instituted through acts which are internally discontinuous, then the appearance of substance is precisely that, a constructed identity, a performative accomplishment which the mundane social audience, including the actors themselves, come to believe and to perform in the mode of belief. If the ground of gender identity is the stylised repetition of acts through time, and not a seemingly seamless identity, then the possibilities of gender transformation are to be found in the arbitrary relation between such acts in the possibility of a different sort of repeating, in the breaking of subversive repetition of that style.’ Butler, Judith. “Performative acts and gender constitution: An essay in phenomenology and feminist theory.” Theatre journal 40.4 (1988): 519-531. p520

Task: Re-read the quote above –
1. what are examples of the way that gender is constituted and re-consituted through speech acts?
2. what does Butler mean when she says that ‘gender is instituted through acts which are internally discontinuous’?
3. what kind of acts might produce the possibility of a different sort of repeating?

If language is performative may not other forms of communication be so?

We might also look to some recent DH work

‘In a model of materiality as fundamentally performative, we can show how forensic, evidentiary materiality and formal organization serve as a provocation for the creation of a reading as a constitutive interpretative act. The specific structures and forms, substrates and organizational features, are probability conditions for production of an interpretation. Knowledge creates the objects of its discourses, it does not “discover” them. Constructivist epistemology shifts our attention from knowledge to knowing, from objects that are observer-independent to the recognition of observer-dependent process, or events.’ Drucker, J. “Performative Materiality and Theoretical Approaches to Interface”, Digital Humanities Quarterly 7.1, 2013. (emphasis added)

Task: Lets consider some ways that communication technologies perform
IM (e.g. whatsapp)
Text Messaging
Screen sharing


Doing Communication

Optional Pre-task:

Download the Processing library ‘OscP5‘. Follow the installation instructions and have a play with the examples.

For extra brownie points you may want to have a look at this. An overview of how the internet is structure in terms of its backbone, ISP etc.

If you have an android mobile phone, download this free app. We are going to play with it in class.

All sketches for today’s class are here session_3_network_communication.

HSS8121: Media Archaeology

 What is Media Archaeology

According to Jussi Parikka and Erkki Huhtamo,

It investigates “alternate histories of suppressed, neglected, and forgotten media that do not point teleologically to the present media-cultural condition…” (Huhtamo & Parikka,Media Archaeology, 2011 ).

And Geert Lovink says,

‘Media archaeology is first and foremost a methodology, a hermeneutic reading of the “new” against the grain of the past, rather than a telling of the history of technologies from past to present. No comprehensive overview of the media archaeology approach is yet available, but we could mention a few scholars, such as Friedrich Kittler, Siegfried Zielinski, Werner Nekes, Jona- than Crary, Katherine Hayles, Werner Künzel, Avital Ronell, Christoph Asendorf, Erkki Huhtamo, Paul Virilio and others.’
(Lovink, My First Recession – Critical Internet Culture in Transition, 2003)

’In his Archaeology of the Cinema C. W. Ceram states: “What matters history is not whether certain chance discov- eries take place, but whether they take effect.”4 When Hertz experimented with electromagnetic waves he meant to prove Maxwell’s mathematical calcu- lations of the electromagnetic field; almost by accident he thereby practically invented radio transmission technology.5 How can we write media history when media systems create their Eigenzeit?’ (Ernst, Digital Memory and the Archive, 2013, p57)

We can say that Media Archaeology is not a single methodology, but an orientation – a direction which is common to a loose group of researchers (a lot of whom are German and related to a particular kind of german media theory and philosophy) and a particular flavour of research. If methodology is ‘a system of methods used in a particular area of study or activity’ (OED) then really Media Archaeology exists at a level of abstraction above this. It is an associated set of theories, methods, methodologies and skills which emphasise a close reading of technology itself, not just in its ability to be a cultural phenomenon. As Friedrich Kittler says:

‘History is not a list of, “directors, stars, studios and celebrities, which in the end remains organised around a series of titles” ‘ (Kittler, Optical Media, p. 26).

As Ernst summarises,

‘Equally close to disciplines that analyze material (hard- ware) culture and to the Foucauldian notion of the “archive” as the set of rules governing the range of what can be verbally, audiovisually, or alphanumeri- cally expressed at all, media archaeology is both a method and an aesthetics of practicing media criticism, a kind of epistemological reverse engineering, and an awareness of moments when media themselves, not exclusively humans anymore, become active “archaeologists” of knowledge.’ (Ernst, Digital Memory and the Archive, 2013, p55)

In practice

Although he is not usually referred to in the ‘canon’ of MA, we could look at Matthew Kirschenbaum’s archival practice with electronic literature as an example of MA in practice.

Meanwhile Wolfgang Ernst frequently uses the spatial and temporal specifics of technological kinds of writing to discuss and problematise the way we understand time and in particular historical narrative.

‘…the historical mode of describing temporal processes has been confronted with alternative modelings of time, When it comes to describing media in time, this aporia becomes crucial, since one can no longer simply subject media processes to a literary narrative without fundamentally misreading and misrepresenting their Eigenzeit. Historical media narratives take place in imaginary time. Storage technologies, on the other hand, take place in the symbolic temporal order…’ (Ernst, Huhtamo and Parikka, Media Archaeology, 2011, p. 242)

‘But is radio, when playing, ever in a historical state? Is it not in fact always in a present state? The medium only appears to conform to the logic of historical epochal concepts; in actuality, it undermines this logic and sets a different temporal economy. For example, an original record- ing resonating today from an old tube radio, provided it can still run on 220 volts, hardly makes history audible. A tube radio thus practices compressed time with respect to our sensory perception, as long as this is not overlaid with “historical meaning,” which corresponds not to the actual media work- ings of radio but rather to the logic of inscribed historiography.’ (Ernst, Digital Memory and the Archive, 2013, p159)

Ernst makes the point that our language and methods of discussing, and modelling time in terms of historical narratives just aren’t up to the task of considering what technological (particularly electronic and computational) media actually do.

He also has a lot of interesting things to say about counting,

‘The numerical order, the basis of digital technologies, has always already been performed as a cultural practice before becoming technically materialized.’ (Telling vs Counting in, Ernst, Digital Memory and the Archive, 2013, p147)

To tell, we learn, as a transitive verb, means not only “to give a live account in speech or writing of events or facts” (that is, to tell a story) but also “to count things” (to tell a rosary, for example). The very nature of digital operations and telling thus coincide.’ (Telling vs Counting in, Ernst, Digital Memory and the Archive, 2013, p147-8)

The conjunction between telling stories and counting time is more than just a word game: verbs like conter, contar, raccontare, erzählen, and to tell are testimonies to a way of perceiving realities that oscillates between narrative and statistics.’ (Telling vs Counting in, Ernst, Digital Memory and the Archive, 2013, p149)

I found this work (actually earlier published versions of this work) extremely compelling during the early stages of making this piece, Mark Inscriber.

Ernst also maintains the Medienarchäologischer ‘fundus‘.

In Art Practice

Often this work takes the past as a point for a future imaginary.

Jamie Allen’s ‘The Lie Machine

Pablo Garcia’s ‘Profilography

Imaginary Magnitude, By Stanislaw Lem

And (though he doesn’t use this term himself) we could think of our own Diego Trujillo-Pisanty’s most excellent ‘This Tape Will Self-destruct‘.

In Pedagogy

We’ve previously talked about Julian Oliver’s work. He and Danja Vasiliev have a series of workshops around understanding network fundamentals. Jussi Parka points out that,

‘We can speculate that such ideas and practices as Weise7- group’s are an indirect response to what Geert Lovink (2012: 22) has called the need for ‘materialist (read: hardware- and software- focused) and affect-related theory.’ In this case, theory is not executed only in the normal written format but as engineered situations: the other material infrastructures and modes of expression in which power operates, from code to networks.’ Parikka (critically engineered wireless politics, 2013)

He notes that,

‘In real time computing systems, however, the collection, organization and storage of information leads directly to action, to integrated surveillance and control over the object environment. This dynamic marriage of information and control in real time systems is a fusion of knowledge and action, and, through directed action in real time, information is expressed as power. (Sackman, 1968: 1492) (in Parikka,critically engineered wireless politics, 2013)

These workshops (and others like them) are oriented towards a kind of techno-politics based on hacking intervention and self-enablement (however genuine this ends up being). There’s a like-spirited endeavour in this paper which I saw present in Xcoax – a symposium you should all make yourselves aware of.


What does the Sack piece tell us about the way that memory has been conceived of in the history of computer science?

Warren Sack in: Fuller, Matthew. Software studies: A lexicon. Mit Press, 2008.

Electronic Memory in Practice

In this section we are going to have an archaeological look, a dig in fact, at electronic memory. To do this properly there are a certain number of things we need to understand first.


how does binary work?

1 bit – 2 possible states
2 bit – 4 possible states
3 bit – 8 possible states
4 bit – 16 possible states
5 bit – 32 possible states
6 bit – 64 possible states
7 bit – 128 possible states
8 bit (a byte) – 256 possible states

Here’s a way of working out the value:

(image cc wikivisual 2015)

(then add the results : 32+8+2=42)

And here’s another way:

how to read binary

(image cc wikivisual 2015)

(then, again, add the results : 32+8+2=42)

This is how we combine single bits to create more and more memory. But what we are interested in is how, on a the level of both logic and components, this works.

Latches and Flip flops

Latches and flip flops (we can talk about the difference – it depends who you ask) are an essential part of computer memory. Some version of this circuit is inside the most fundamental aspects of computer memory. They are therefore massively significant in thinking about what we mean when we say ‘computer memory’. We are going to build a state saving circuit ( a flip flop) and use it to explore what we do and don’t know about digital memory and how we can use that as part of a research and creative methodology.

History and archaeology of the flip flop.

Here’s the original patent, designed with vacuum tubes.

And what do vacuum tubes do?

How does it work?

Let’s hear a nice (rather slow) explanation.  To understand what flipflops are and why they are important we first need to know a few things.

Like what is boolean logic?

How can we combine two NOR gates into an OR gate? Simple (ish)! We invert it! See a bunch of examples here. Take one and explain it to your partner!

Step one: Building a NOR gate

This circuit uses transistors

Transistors are manufactured in different shapes but they have three leads (legs).
The BASE – which is the lead responsible for activating the transistor.
The COLLECTOR – which is the positive lead.
The EMITTER – which is the negative lead.



Here’s our circuit diagram. (and below obviously)

Step Two: Combine two NOR gates into a flip flop.

Look at the diagram below. How should we wire up our NORs to make a flip flop?

2 NORs making a flip flop. CC wikimedia


It works, what next?

A flip flop gives us a single bit, held in memory (as long as there is power). Here are somethings I want us to discuss:

  • what is the significance of holding one piece of memory – what can that memory mean? What can it do? Wittgenstein asks of a man given a piece of paper which asks for 5 apples, the following ‘But how does he know where and how he is to look up the word ‘red’ and what he is to do with the word ‘five’?” Well, I assume that he acts as I have described. Explanations come to an end somewhere.—But what is the meaning of the word ‘five’? No such thing was in question here, only how the word ‘five’ is used.’ What do we mean when we say a bit ‘means’?
  • Now imagine we have an encoding system for that bit. e.g. 0 = ‘apple’, 1=’pear’. How does that affect the above?
  • Stepping outside this question for a moment – how many real-world applications for the storage of one bit of information can you think of? How about for 2 or 3 bits?
  • If we all combined our individual bits into a large register – what could we store? How could we act?

But Why? Let’s talk about that

  • What elements of media archaeology (if any) do you identify in your work?
  • What would be the impact of this method?
  • Returning to our project, how could you take this exploration of digital memory further, how would you develop it?


Suggested readings

What is Media Archaeology? Parikka, 2012, Polity

Media Archaeology, Huhtamo & Parikka (eds), 2010

Digital Memory and the Archive, Ernst, 2012, University of Minnesota Press

Deep Time of the Media, Zielinkski, 2008, MIT Press


DMS8013: Serial Communication. “Down the wire”


Here’s a zip of all today’s sketches

HSS8123 – Public Data

Links from the session

Open Public Data

Visualizar 09

The Open Data Foundation

Home – Office for National Statistics

Crime In My Area – Interactive UK Crime Map | ADT

Open Cultural Data

Flickr: The Commons

Europeana Collections

Choose a License

ArtsAPI: Now Live » FutureEverything

Arts API: Artwork

And lets not forget music for programming!

musicForProgramming(“03: Datassette”);

All examples can be found in the zip here

Your task is to make a public data dashboard.

This will

1. use real public data in the form of text, images or other media
2. remix this data by combining it with other data sources or presenting them together
3. intervene in this data by using it as the basis of some process


HSS8121 Session one: Walking in Public


  • To talk about walking methods in different kinds of research and consider how they might be relevant for us
  • To focus within that on sound practices with walking as an in depth example
  • To explore public space differently through sound walking
  • To use recording to attend to public space in new ways


Research and walking

The interest in walking as part of research methods is, of course, varied among disciplines but among the many concerns are the ways that walking configures our relationship between the body, space and time in ways that make us think some of the suppositions of research methods. People may talk, act and think different when walking and this makes researchers excited and go away and write research papers about it.

Ethnographies of walking and what they tell us

In anthropology and ethnography there has been a recognition that walking practices are a worthy object of study in their own right, that in fact the way we walk is subject to and productive of cultural norms. This has led to an interest in using walking itself as an ethnographic technique for understanding space through people’s motion.

Ingold and Lee present an edited volume combining some fascinating studies of this including with some hunter-gather societies and with so called “confluencers”

Some other examples

Rhythm and the passeggiata.

Vergunst, J. (2010). Rhythms of Walking: History and Presence in a City Street. Space and Culture, 13(4), 376–388.

Jo Vergunst explores the walking of busy city streets as an affair of rhythm and connects this to Lefebvre’s “The Production of Space” in which he (Lefebvre) notes how our sense of time is subject to capitalist and industrial senses of control (Engels makes related points).

“A walker entering the street becomes immersed in the movement and the sounds of the time of the day, week, and year and in the changing patterns of activity as the street and the city develop. The street itself is a place of rhythms and interactions. From this perspective, it is the sensing of rhythms in the street (be they coherent or chaotic) that enables it to be understood as a place and indeed form it as place. This is the essence of Henri Lefebvre’s concept of rhythm as evident in The Production of Space (Lefebvre, 1991) and extended in Rhythmanalysis (Lefebvre, 2004). Rhythm contributes to Lefebvre’s analysis of everyday life particularly in the conceptualization of time. In The Production of Space, time is configured as the result of capitalist control over space: “As for time, dominated by repetition and circularity, overwhelmed by the establishment of an immobile space which is the locus and environment of realized Reason, it loses all mean- ing” (Lefebvre, 1991, p. 21).” p378

His main point is that walking by virtues of its continuity blends time in a way that counters industrial divisions of it.
“In most walking, past, present, and future seem more blended and indistinct—we do not have a “moment,” as such, as we walk. This relates to the English verb form of the continuous present: “I am walking,” English-speakers say or think to themselves as they actually do it, not the merely general “I walk.” Walking as an activity implies a continuity rather than a moment.” p381

Yi’En, C. (2014). Telling Stories of the City:Walking Ethnography, Affective Materialities, and Mobile Encounters. Space and Culture, 17(3), 211–223.

In this paper En suggests walking as a way of differently encountering the materials of everyday urban existence as a kind of continuing collage of experience.

“I mobilize two aspects of urban life that are only beginning to attract scholarly attention in extant literature—urban materialities as affective materials for organizing mundane experience and urban mobilities as heterogeneous and rhythmic to demonstrate how “walking” is a practice that orientates the “walker” through different dimensions of “ordinary” and “everyday” urban life.” p2

Interviewing and walking

Including efforts to locate this on GIS maps (emphasis added)

Evans, J., & Jones, P. (2011). The walking interview: Methodology, mobility and place. Applied Geography, 31(2), 849–858.

“…a major advantage of walking interviews is their capacity to access people’s attitudes and knowledge about the surrounding environment. Walking has long been considered a more intimate way to engage with landscape that can offer privileged insights into both place and self (Solnit, 2001). Ingold and Lee (2008) suggest that walking with inter- viewees encourages a sense of connection with the environment, which allows researchers to understand how, for example, places are created by the routes people take”

“Representing qualitative data in map form makes them instantly more appealing to decision- makers. They offer an opportunity to make people’s values and local histories count more within a range of development processes. But the power of maps is well documented within geography (Wood & Fels, 1993), and care is required when using them to represent qualitative data. Maps simultaneously increase the potential damage that can be caused by misinterpretation and over-simplification. Further work exploring the potential to apply this technique in real-world decision-making scenarios is needed to understand the most effective ways in which to analyse and represent data.” p857

Soundwalks and acoustic ecologies

…could be a degree programme all to themselves but for the moment I’ll constrain myself to a couple of points. Historically they have a relationship with the notion of acoustic ecologies as described by Murray Shaeffer. There’s a bibliography here for those interested.

Hildegard Westerkamp speaks of the “disruptive nature of listening.” in terms which have a lot in common with some of the temporal points made by Vergunst and others about walking. It’s unsurprising therefore that some creative research methods involve doing both. We’re going to try some of this together.

“When I speak of the disruptive nature of listening, I agree with Michael Stocker who writes: ‘Our experience with sound unfolds as a continuous now.’ If we open our ears to this experience of sound unfolding as a continuous now it inevitably includes an opening to surprises, to the unexpected, to the difficult and uncomfortable, to noise or potential discomforts with silence. It means staying with the sound for a time no matter what reactions it may elicit in us.”

Westerkamp offers a long history of soundwalks available online like this one.

Discussion point: how does her narration affect our understanding of space throughout the recording.



Instructions for sound recording

1. Listen to a sound from up-close, try to position yourself so that this sound the only one you can hear, record it.
2. Record multiple sounds intersecting, see if you can move through the intersection.
3. Find the quietest place you can, stay there and record it for a while.