DMS8013: Sessions Overview

Below is a list of session titles and the dates they’ll happen. Excited?

1 – Physical Computing, 2nd February

2 – Communication 1 local devices, 9th February

3 – Communication 2 networks, 23rd February

4 – Algorithms and Generatively, 1st March

5 – Working in 3D, 12th April

6 – Hacks and collaborating, 19th April

7 - Terrible interactivity, 3rd May

8 – Data wrangling, 10th May

Current Students

Students currently studying Master in Creative Arts Practice

 

Meena Daneshyar

Maria Clemente – Albacete

Sean Cotterill

Xiyuan Tan

Yue Wang

Riar Rizaldi

Daniel Parry

 

Previous Students

CAP

2014-2015

MeteorEdmund Nesveda

Clive WrightChilly Rain

Wenya ChenTrong Cuong Dao

Yousif AbdulghaniTan

BartiZhang Wei

 

2013-2014

Yinzhen BaoTatiana Fujimori
Jaejun HwangSaksit Knunkitti
Wenchang LinClare Robertson
Tunc Karkutoglu

Mres Digital Media
Adrian ParkBen HoldenIsobel Taylor
James DavollNina LimardoXue Yan
Aaron SmilesAlessandro AltavillaAndrew Nixon
Andrzwej WojtasBen FreethBen Thompson
Helen CollardJane DudmanJoseph Pochciol
Pengfei ZhangSanjay Mortimer
Ewelina Aleksandrowicz (Tikul)

 

HSS8123 – just a report of my progress

Long time no see. Yeah, because I have been reading and researching and writing code…

At least I am glad that I am doing what I proposed in the HSS8123 module last year. Yes, I mean last year. I recognise that I would normally do what I said (or promise). For example, in the first year of high school, I said I wanted to study in Wuhan University, and I really did!! OK, enough. So, last year in a HSS8123 session I gave a presentation saying that I am going to make something funny and ridiculous, like Fountain or South Park. And so my research topic for final project is humour and joking in digital art. And explore whether people would appreciate joke as art.

May is a deadline month. So I took about a 2~3 week break in June, and then started literature research. I went to dig for materials related to jokes, humour and appreciation of art. I also spent some time reading neuroscience books because I like medicine and biology…anyway the literature research was quite interesting and gave me inspiration. And as a by-product of the research I, again, want to emphasise the power of culture difference…it’s just so powerful, which is one reason why I chose to apply for a uni in Singapore, an eastern country, to do PhD….OK, OK, I’m sidetracking again, I’ll shut up *LOL*. And of course the conclusion is that jokes should be considered art and we are appreciating them just like appreciating other artworks.

Besides the books and articles I also found some nice ideas when I googled some digital April Fool jokes. Check out this one: http://www.digitaltrends.com/mobile/best-april-fools-day-jokes-2015/

I also asked Tom for advice because I had difficulties finding DIGITAL ART jokes and pranks. Tom’s suggestions are really helpful and I think THIS guy is doing great digital clowning performance videos.

But so far I haven’t found any digital art pranksters. How can we make pranks with digital art?

After watching those prank videos I conclude that the most important thing in pranks is the ridicule of the victim…look at those prank videos showing a close-up of the victim’s face expression after being played a prank on. And what causes this ridicule is mostly the erring of the victims or faulty logic. Such as asking someone to help with a gadget that has been hacked so that the person who uses it will cause trouble for sure (erring of victims). Or, for example, your dining table crashes when you press the “call service” button (faulty logic).

Alright, so I am also going to make the visitors to our exhibition “err” when they walk to my installation. Imagine this: you’re enjoying this nice and neat digital art exhibition. Then you see this corner with a warning sign, but the area looks nothing dangerous. It has only got projections of cartoony cute little birds on the wall, and even has a small table and several sofas for you to relax. You are tired and you sit on the sofa. Then you hear a savage cockcrow behind you, and the projection on the wall start to glitch and show weird flashy shapes. You think you might have destroyed the installation by accident, then suddenly the blue screen of death appears. You are scared of being in trouble and you sneak out of the ballroom…

OK, the above are all assumptions. Tom says never make assumptions about how audience would react, but I would love to see this happen…Anyway I have contacted Abi and hope to talk to her about how to deal with audience’s possible reactions to a digital art installation.

The first week of July has almost passed and currently I am working on Processing code writing. I am using Optical Flow to detect people’s movement in my installation area, and I’ll get a force sensitive resistor to detect people sitting on the sofa. Everything is going on smoothly at the moment.

And please feel free to laugh at me if my project turned out to be a flat joke in the end…I love pranks, but I didn’t think creating my own prank can be so hard!!

 

HSS8123: at the CLIMAX of my journey

Screen Shot 2016-06-09 at 21.35.36

As it happens in the movies, when the story is reaching the end, the hero takes a decision that changes his path 180º or even 360 ºand he ends up arriving to a different destination. That’s exactly where I’m right now, I am at the CLIMAX of my journey (the master) and I am going to take a decision that will change what was expected: change my final project.

When I was doing the assignment for Digital Media module, I was struggling with it a lot. I was very frustrated until the point of wanting to dropout, as I had to create a final project that it was very related to what I was doing in that moment. I was asking myself: ‘Do I want to be like this over the summer, working on project that is too much for me?’ I was trying to achieve a result that resembled in some way the projects that I mentioned in a previous post like ‘Pilow Talk’, ‘Family Rituals 2.0′ etc. The authors of these projects had a better foundation in coding when they created them than I do have now, and I won’t have it over the summer either, not even if I locked myself in Culture lab 24/7: Joana, Montgomery, the author of ‘Pillow Talk’, has a Bachelor’s degree in Digital Interaction Design; David Chatting, one of the authors of ‘Family Rituals. 02’ has a Bachelor’s degree in Computer Science. Obviously these guys had the right foundation for achieving their projects successfully. Let’s be realistic, I am not ready for this kind of project, I don’t see myself doing it and I’ve realised it when I was doing the assignment (this is what’s call in writing anagnorisis). In addition, I don’t know how the final result could benefit in my career, how it would fit, and how it would be useful in terms of employability. I think in my career the outcome of this project doesn’t worth the stress that it would imply achieving it.

On the other hand, I have been working on a personal project since the first semester. It’s about Long Distance Relationships and it’s made of real testimonials. People from different countries and backgrounds share their personal stories in ‘Wireless Love’. It is becoming bigger and complex and I need to dedicate hours this summer to get it done. It started being just a unique documentary film, but it seems now that it will end up being  a serialized audio visual project.

When I was doing my assignment for Screenwriting module, I was stuck with the story I was writing. After a while I started to remember what the professor Tina Gharavi (the module’ leader) was constantly saying in class: ‘The problem for many of the scripts is that they were just “too wide” and very removed from you or reality. Would you really want to watch the film you are writing? Go back to the beat sheet you wrote in the first day. Go back to your starting point, your initial idea, and remember why you wanted to write that story.’ I applied that advice for my assignment and it was very helpful. Then, I decided to do the same with the final project as well. I looked at the title of my blog, which is ‘digital storytelling’; I checked my first post about hypothetical ideas for the final project, which was mainly ‘storytelling for kids’. Storytelling is my thing; it was what I wanted to do and what I still want to do. Creating or designing a device for L.D.R. couples has nothing to do with storytelling. In fact, with this last idea for my final project I was getting far away from my starting point, for the reason I came here, to Newcastle. I’m aware that my tutor Tom won’t be very happy with this decision because he always wanted me to do something different from video. But the thing is, even if video is my background owed to my Bachelor’s degree, it is something that I have not done for the last 2 years, probably 3 years. I was rusty when I came to Newcastle and actually one of the main reasons why I wanted to come here and do this specific Master was because I wanted to go back to filming. I chose this Master for the freedom that the students could have about the module’s choices (from different disciplines) that made sense for them and for their careers. I didn’t choose it for learning coding or becoming an artist or becoming a Phd student after the master. I chose it for the freedom that students have in order to create their projects, not matter what discipline is involved as long as it is artistic or creative. In fact, I have recently checked the handbook of ‘Creative Practice Project’ and there is one part that says: ‘Although many projects incorporate elements of digital technology, this is not a requirement’.

In conclusion, my project is going to be a serialised audio visual project. On one hand I will edit small videos about the couples I have been interviewing a long these moths and also I’d like to create a sort of “interactive storytelling’ for the web with Processing. I’d like to display the information about project on the web (videos and information) in an interactive way as these examples:

http://www.sbs.com.au/cronullariots/documentary#chapters/introduction/map

One of the ideas I have is creating a map that illustrates the distance that exist between the couple who starred my projects. So people can click on the couples and get information about them, they can watch the video etc.

Other kind of interactive storytelling that could be source of inspiration is:

http://www.sbs.com.au/theboat/

Well, these are just some ideas; I really have to research about this, and decide what exactly I am going to do with part of the project.

I’d like to finish clarifying that this project is actually the reason I came here (something I have forgotten a long the months unfortunately). As you can check on the blog the video documentation I have created for Wireless Love,  this project was an idea that I had some years ago. I came to Newcastle to create finally this audio visual project on my own (that was trapped in my head) for my portfolio, as an example of what I’m capable and what kind of projects I’d like to be working on the future. Now, everything makes sense for me, I understood why I am here, why I came here. I am looking forward to start this third act of my journey.

* As a curious anecdote I’d like to point out that the video documentation I did for Wireless Love is going to be useful for me actually, in order to apply a funding competition that is going on this month. So I am very happy that I have alredy done it and got some useful feedback from my classmates/tutor to improve it. Thanks guys

[HSS8121] – Presentation document

幻灯片1

[HSS 8121] – Research on the interrelationships between facial expression, colours, and emotion Research Proposal

图1片2

This proposal is composed of seven sections, the first part is introduces briefly what I would propose to do in the research, what kind of other research has been conduced in this area, and the general overview for this proposal. The second section mainly focus on the define the research questions, issues or problems, it lists the what questions I want to find answer of this study. Such as different facial expression link with basic emotion (happiness, sad, surprise, disgust, angry and fear) can be describe for what color or if we use opposite colors add in facial expression dose it also meaning original human emotion? The third part of includes the aim and objectives of this research proposal, which explore the target that I want to be achieved and why I want to research it.

The fourth section discusses the what research method I should employ, for example in the class we have learned four research method: Ethnography, Practice based research, Video analysis and Media archaeology, and why these are the particular approach chosen. The fifth section describes the literature review and background studies, what other research is being conduced in this area. The sixth section analysis on what knowledge I learned from this research proposal and used into my “LATES” events design for “In Your Face” exhibition. Finally, the last section concern about the conclusion and future development of this research proposal. Summary of whole proposal and review which problem I can answer or which question still need thinking about.

Problem Formulation

The first important question in this research proposal should be addressed is what kind of interrelationships between facial expression, colours, and emotion? As Cole and Moore (2014) points out, particular at infant facial activity express specific emotions, at early social interaction, without talk, facial expression have important relationship with emotion. Form other point of view, Gil and Le Bigot (2015) has examined that effect of colour background on the processing of stimuli is important factor in human interactions. Moreover, Geovany, Olac and Stephen, et al (2014) argues that human display emotion through different channels such as head poses, bodily gestures, head poses, speech prosody, and facial expression, but also through physiological signals, such as colours changed stimulate. Different colour might be evoke some specific human emotions, like red evoked people feel happiness, black evoked people feel sadness or fear. Some colours predict human emotions higher than other colours, such as when people look at red, the emotion effect higher than people look at blue. Color & Paints Interim Meeting of the International Color Association ( 2004) has reported that color can relate our emotion and feeling, matching colours (such as red, blue) to a certain number of emotions (such as angry, sadness), the data about emotions reaction to colours will be assessment. However, those research all study about two elements relationships whatever between facial expression and colours, between colours and emotion, or between facial expression and emotion. If three elements together appears, what kind of interrelationships they have?

 

The second question is how the human face recognition system to identify people’s facial expressions and emotions? Does people’s facial expressions eventually express the content is person’s emotion? For example, Barrett, Mesquita and Gendron (2011) has shown that facial expression is one of the main external form of expression of emotion, in order to guide the individual behavior of social situation have suitable interaction, people need correctly extract the corresponding emotional information from other people’s face. In real life, the facial expression maybe not happen individual, it is always associated with the particular situation and with particular emotional appeal. If we want to accurately identify the emotional state and to gained real meaning by the face expression of other peoples, we must identify the current emotional situation and integrating it with facial expressions.

The third question is does opposite colours will create mood illusion for face recognition? This question possible hardly to resolve, because I have not find some research suggestion correctly talk about opposite colours and mood illusion for human face. Just found some experiment from Yong, Sudirman and Chew (2012) investigate that the interrelationship between colours and facial expressions in order to realize the result acquired in psychology, art and design.

Research aim and objective

The first aim of the research proposal are try to solve the three questions above-mentioned, understanding the knowledge relating to the questions, and seeking to enhance knowledge from other people’s research.

The second aim of the research proposal are use the knowledge I researched to enhance my “LATES” events design for “In Your Face” exhibition. If I known the relationship between facial expression, emotion and colours and each factor’s specific function, it is possible I can improve my events activity design’s quality. For example,  reasonable arrangement for each factor in my design make it more effectual, in order to enhance audience events activity experience, which part of design should use colours change to guide audience’s facial expression change and create different emotion.

The reason why I attempting to do this research is I really interesting in facial expression, emotion and colours. In last year, I have a photography work named ‘ Half water, Half fire’, this photography works shown the opposite colours covers people’s image, when the colours effected on image, the photo’s original emotion will have change, even the people’s facial expression in the image have not change, but you will have totally different feeling with two same photo. Also, I think opposite colours can enhance audience’s feeling and emotion when they see the different facial expression on photo. But when I done this work in last year, I do not know this kind of knowledge, this research remind me rethinking about this project and want to find the develop for this practice based on research. Moreover, if change this photography work to be a events activities, what interaction I want to make with audience, how did I to redesign this project? So I really want to research on this area and find the knowledge improve design.

Research method

From HSS8121: Enterprise and Research Methods module, we have learned four research method. The first research method is Ethnography, as Hammersley and Atkinson (1983) point out that ethnography is an significant approaches using with social research today. In the early time, ‘ethnography’ was often seen as complementary, which referred to historicial and comparative analysis of no-Western societies and cultures. After that time, ‘ethnography’ refer to an integration of two part of, one is the first- hand empirical investigation and the theoretical, another is the comparative interpretation of social organization and culture.

Moreover, what ethnographyers should do? In class John has mentioned that Ethnography have three factors, firstly, Features include naturally occurring settings, ordinary activities, social meanings, the perspective of the participants, member categories, descriptive, participant observation and reflexivity. Secondly, Data to be covered: 1. Multiple data sources. 2. Interviews, observation, note-taking, video and audio, documents and artefacts. 3. Coding. 4. Qualitative, interpretative analysis. Thirdly, Disciplines about anthropology, sociology, concern for the underdog, and ethnography in CSCW, HCI and design.

In this question – what ethnographyers do? – Hammersley and Atkinson (1983) has more details mentioned, for example, they think in term of data collection, ethnography regular involves the researcher participating, overtly and covertly. In the everyday life, listening to what is said, watching what happen, or asking question through informal and formal interviews, collecting documents and artefacts.

The second research method is video analysis. When we use this research method, we need think about uses, features, conversation and video analysis. Different from other traditional research method such as interviews and questionnaires, as Heath et al. (2010) concerns that the research method for video analysis need taking the participant’s points earnestly and the resource from their dayliy activities. For example, when audiences presenting these of data, need pay attention to the selection of relevant examples, multiple fragments, presenting single and the graphical representation devices. For research method video analysis, we should not only focus on uses evaluative, formative, rethinking ‘interaction’ and related concepts, but also concerns about data transcription, look for phenomena of interest, and qualitative, interpretative analysis. Moreover, we need think about how activities are socially organised through speech, body movement, gesture and engagement with objects. When we do the recordings, maybe use multi-camera, shot a broad static, not the same as promotional or documentary video.

The third research method is Practice Based Research, this one can be more easy understand than Ethnography and video analysis.

The fourth research method is Media Archaeology. As Lovink (2003) argues that media archaeology is first and foremost a methodology, a hermeneutic reading of the “new” against the grain of the past, rather than a telling of the history of technologies from past to present. This research method might be use in electronic practice, art practice or pedagogy.

Following those four research methods, I think the research method Ethnography, Video analysis  and Practice based research I should employ. The reason I chosen Ethnography because in this research proposal, the interrelationships between facial expression, colours, and emotion, particularly I need participation, overtly and covertly, watching what happen in my “LATES” events design for “In Your Face” exhibition, collect the data from audience’s reaction. Moreover, I should combine with the research method Practice based research, to form a positive circle. For example, do research, practice, deeply research main question from practice, found the answer and practice again, participant observation practice activities, then collect data and analysis, found the new problem and research again to solved. The reason I chosen the research method Video analysis because the main basic design for this research is a photography work, I need use this method to analysis multiple fragments and take the graohical representation. In addition, rethinking ‘interaction’ and concerns about the photo meaning transcription.

 

Research review

Yong, Sudirman and Chew (2012) have an experiment about colour perception on facial expression towards emotion, the experiment used emotion wheel and six universal emotion (happiness, surprise, anger, disgust, fear, and sad). The experiment ask participants select a single colour for the each of the seven faces. Face is expressive of the six basic emotions and the neutral, the six emotions are not named (Fig.1).

 

Source: Yong, C., Sudirman, R. and Chew, K. (2012). Colour Perception on Facial Expression towards Emotion. TELKOMNIKA (Telecommunication Computing Electronics and Control), 10(4), p.783.

The whole experiment data results shown the concordance among the observed expression of human emotions, colours and their relationships. If I can collect more picture have various emotions more subtle and emotional, it might be more clear about the answer. A valuable color accessories at facial expression picture maybe can find more data form audience emotional state, feeling, emotions, mood, cognition and connotations. Form this experiment, as Yong, Sudirman and Chew (2012) point out that design should select colours to be more objective, each of the colour represents the consensus of the perception are different, designer can used this point to guide their works.

 

 

[HSS 8121] – Creative Entrepreneurship Project Report

图片1

In this creative entrepreneurship project report, have three sections was reported. Firstly, this report will focus on introduce my “LATES” events project and activities design for “In Your Face” exhibition include description my work, detail for the events design, picture analysis and events themes choosing. Secondly, thinking about the course knowledge link with my research proposal discuss how I used research method on my design. Finally, is a technical rider for what I need in the exhibition maybe prepare from National Media Museum and a biography of myself positioning my practice.

In Your Face: “LATES” events activities design

As the exhibition messages mentioned that we should present engaging projects of LATES evening events connecting adult audiences and following four exhibition themes: reading faces, changing faces, beautiful faces, and choosing faces. Explore how human and machines interpret different faces and facial expressions, why people change their faces, how people analysis their perception of beauty and why people represented their face as portraits. The projects I want to presented is my work “Half water, half fire”, it is a photography project, link with this project have two audiences interact activities design.

Photography project “Half water, half fire” is a series photos that focus on explore human emotion, photography colours change, and contrast vision (Fig 1 and 2).

 

In this project, the mainly colours are red and blue in one photo’s two layer, it is seems to be one photo have two portraits display. When we look at it use red card, will see the blue layer for the photo. Instead, when we use blue card would be see the red layer for the photo. Each layer in photo is different facial expression with different emotion from model. This processing for the visitor, possible is a method to choosing face and changing face, because when user decide to use which card will decide what face image and model emotion will be shown. If they don’t use the colour card see it will have a confusing and contrast vision, but I think audience still can feeling the model’s mood and strong visual impact.

Because this is a past time project for me, in order to make it suitable for the LATES evening events I have three pieces of development for the project based on my research proposal. One part of development is in the exhibition, this series of photos might be focus on portrait’s part of face, not shown the half-length portrait. This might meaning model’s facial expression and face will be more clearly and directly. One development is I will shot some new photography in same style not only have six basic facial expression emotion (happiness, sadness, surprise, anger, disgust, fear) with portrait, but also have more kind of facial expression such as crazy, indifference, confused and nervous. Another part of development is set three pairs of contrast colours, original pair of colour is red and blue, added two pairs of contrast colour is yellow/purple and green/orange. Visitor will have more experience from different pair of contrast colours and facial expressions, moreover, when they join follow-up activities will have more way to choosing in their own photo.

In the exhibition events, my plan is have a independent space to show the photography and running the activities. For example, following figure 3, photography have two ways to display: one way is use projector show at wall (number 1 area in the figure 3), another way is use large-screen (number 2 area in the figure). Those two screen will connect with computer scrolling play different images, the large-screen will also use to following activities, when audiences take same style face photos they can choosing their face scrolling play at large-screen if they want.

Activity plan A is virtual face photo shot, in the exhibition events I will prepare two cameras for audiences, one is digital camera connect with computer, audiences would be join the activity and choosing different “Half water, half fire ” filter style (red/blue, yellow/purple, green/orange) to take face photo, in this part of section, audiences required to do different facial expressions, preferably do two contrast face expressions, for example, one is looks like really vitality, another one is tired. In our common sense, vitality always presented use green, but I will suggest continue use contradictory method to show, such as matching tired facial expressions to green. This approach maybe make face vision effect more strong, enhance audience’s experience. Because every element part of this design is contrast and people can changing it.  After photo shot, audiences can choosing the photos show at the events, scrolling play at large-screen.

Activity plan B is real face photo shot, another camera I prepare for audiences is Polaroid camera (number 4 area in figure 3), and support different colour card(red/blue, yellow/purple, green/orange). People who join the activity would take face photos and get it immediately, when they get paper photos, they can use colour card try to look at different kind of face and find themself completely different side. For example like figure 4 and 5. It will become strong feeling experience for audience. If they want to put the paper photos show at events will have a part of stage (number 3 area in figure 3).

 

 

Based on the project and activity design, I think the exhibition themes which I can be explore is Changing Faces and Choosing Faces. During the activities, audience will continue make decision on change their faces or choosing face, even the activities plan without make-up, but when people use the colour card or choosing “ Half water, half fire” style filters to show their faces, multiple emotion experience still in their mind.

 

Analysis the project with the Module knowledge

Before I developed change the photography project “Half water, half fire” suitable for this exhibition events, I have do some research of the interrelationship between facial expression, emotion and colours. Because of some questions I cannot found answers in my research proposal, I want to found it in this changed project, such as I need collect data from different audience’s reactivity, feeling and photos use the research method Video analysis. Record the processing at events, take data transcriptions, look for phenomena of interest, and qualitative, interpretative analysis audience’s perceptions, actions, opinions and attitudes.

Moreover, I would to be a participant at activity and observe the people’s action, emotion and behavior in the processing use the research method Ethnography. Such as

Watching their color choices, listening to their exclamation when their face completely different shown and asking their feeling about the different opposite factors repeated combination of this design.

The reason for I add new pair of colors to this project is I want to find more variable factor in the activity, as my research proposal mentioned, multiple facial expressions, emotion and colour combine in one section, opposite elements (contrast colours, contrast facial expressions, contrast emotion) continue crash throughout in the design and activities, what audience want to say? How about their reaction? Why they take different choice for each opposite element? Maybe some questions I cannot answer but I might certain that visitor can get enjoyable experience.

Project and activities design technical rider

1. A bout 20 square meters space use for running activities and show the photography project.

2. Technical: large-screen and projector use for display photography project and showing audience’s activity photo.

3. Two cameras. One digital camera use for take virtual photo transfer by computer and display at large-screen (digital camera maybe I can prepare by myself). One Polaroid camera and photographic paper use for audiences take photos.

4. Five basic light and three photographic light use for make a little photographic studio light for audiences take face photos have better effect.

5. Five meter with four gang power strip more than three.

6. Two mini table and some cushion

 

A biography of my position

My project “Half water, half fire” focus on face photography and describe within colours and facial express emotion, it is might be a good way to shown the exhibition subject “Face”, two related activities design interacted with visitor focus on two exhibition themes: Changing Faces and Choosing faces. Playing with take photos, thinking with face change, choosing with inside subliminal, and feeling with opposite self, hope the project can make audiences get enjoyable experience.

HSS8121 – Video Analysis as Research Method

I use video as part of my research method in every work I produced. Video is part of my process for self-analysis and as well work as a medium for designing an idea. After a lecture on Video Analysis by John Bowers I found video could also used as a qualitative and interpretative analysis. Video is used as a tool for formative research on my work. I used to record every process I make in video and analysing it afterwards. Using a static shot on tripod and static angle technique with multiple cameras, I could clearly analyse the video in raw and unedited format. Video is also used as a medium for me to look for phenomena of interest. It also opens a possibility for me to analyse how activities are socially organized through body movement, gesture, engagement with objects, and surrounding.

I realised that video analysis is always part of my process without even realise that this could be work as a research method.

Here is an example of video I took several years ago and use it to analyse my performance and my body movement:

Click here to view the embedded video.

HSS8121 – Post-Disciplinarity

The themes of post-disciplinarity that Gabriella Arrigoni explored in her other lecture became another fundamental part of my proposed piece.

In her lecture, Arrigoni discussed a number of agencies that draw on creative practitioners in work that extends beyond art. For example, the Wellcome Trust (http://www.wellcome.ac.uk/) commission artists to produce works and aid in research that is related to health. These agencies tend to have a holistic view on practice, rather than a target-based one that an arts organisation might; they value the overarching contribution to the field that creative practitioners provide, and are confident that the research outcomes of this contribution will become clear over time rather than being immediately obvious.

This post-disciplinary approach encouraged me to think outside the immediately relevant areas of my practice such as open-source digital technologies, which is a subject I return to time and time again. During Transmediale I attended a number of talks, workshops and conversations about the subject of privacy and surveillance in the digital space, which was exemplified by their Anxious to Secure stream. I have a long-standing interest in the subject of surveillance in the digital space, having followed groups such as the EFF (https://www.eff.org/) and the Open Rights Group (https://www.openrightsgroup.org/) for some time. While artistic responses to surveillance culture are absolutely nothing new, in the spirit of post-disciplinary practice I chose to explore themes of surveillance in my final work through looking at facial capture technology. Facial capture technology by government agencies have been investigated by artists looking to subvert this technology, for example URME (http://www.urmesurveillance.com/), a ‘generic face mask’ designed to thwart surveillance cameras linked to massive facial recognition databases by transforming your face into that of Leo Salvaggio, the founder of URME.

Arrigoni’s presentation stressed the importance of working outside of your discipline, and this is something I will carry forward to my final piece, which will touch on the subject of surveillance by employing some techniques not uncommon to ethical hackers and private investigators.

HSS8121 – Final Proposal and Failure

Based on a session we did for DMS8013 on computer vision and face detection, my proposal to respond to the open call for this module involved a system designed to capture faces of members of the public in a gallery space and use them to create an ‘average face’ of gallery-goers in real time. However, I noticed early on in using openCV-based facial recognition software that it was very prone to error, recognising a number of background features of an image as faces and registering them in the program as such.

While this was initially a cause for concern, as this meant the program would have a number of false positive results that would have to be filtered out by another part of the program in order to produce ‘good’ results, I remembered the kinds of ‘errors’ that had been shown to us by Alexia. Alexia’s presentation of error was one of humour, with a number of her video examples in particular being presented with an almost sardonic attitude towards her former self. It was for this reason that I chose to incorporate these computer vision errors into the fabric of my work. Rather than design a system specifically to exclude these errors, which would potentially introduce more failures, and attempt to brush over the fact that the system I would build would be ‘incomplete’, I chose in the proposal to show the misrecognised faces as part of the ‘average face’, incorporating incidental background materials.

Incorporating error in this manner also allowed me to use humour as an artistic device. My intention was that audiences seeing their faces mashed up with random mis-recognised faces, as well as some very poorly lit faces and people who did not realise they were having their faces captured, would lead to quite a few laughs, but also to draw attention to the work itself. As the work I planned is fundamentally about surveillance, the levity the topics raised should not be downplayed by the work with humour, but the integration of the kinds of failures Alexia discussed in her presentation have become a major part of the proposal I submitted for this module.

HSS8121 – Practice Based Research

Gabriella Arrigoni’s session on Practice Based Research was particularly interesting for me as a creative practitioner who currently focuses most of their time on the creation of works, with relatively little time invested in theoretical writing around those works. To me, this approach to research as practice based is very much in the same vein as the themes of Research through Design that John introduced during his lectures. Culture Lab has a strong emphasis on a practice-based approach to conducting research more broadly too, and Gabriella’s session served to underline the key concepts of this type of research.

The production of prototypes and artefacts as research is something I was involved in as part of Unbound, a work that evolved over the course of a week and was passed between a number of practitioners at Culture Lab. Each practitioner was given a working day and a set of resources with which they could interact to form the work. When I took part in the work (about halfway through the process), there were already a number of disciplines involved in the work, including databending, painting, coding in processing, video work, physical sculpture and audio work. During the day I worked on Unbound, I filtered the work that previously existed through my own sets of practices, and then at the end of the day offered up the resulting artwork (and documentation) for the next person. This resulted in a shifting prototype which exhibited the individual hallmarks of each practitioner that had come into contact with it.

Importantly, Gabriella documented the process at a number of stages and conducted an interview with me during my work day about the kinds of processes that the work had undergone and the reasons for my selection of those processes. This was then formulated for showing at the Loops Layers Lines show at Culture Lab. This practice based research used actual practice as a basis upon which to explore themes of cross-disciplinary collaboration, and it was a very enjoyable piece to take part in. The prospect of conducting research not by reading but by doing is something that has always excited me; as a creative practitioner I thrive on the act of creation, and Gabriella’s explanation of using this AS research is something I will be taking forward into my final piece.

HSS8121 – Video Analysis

John’s session on Video Analysis was particularly useful to me, especially as I am beginning to have far too much documentation of old gigs etc. to do anything meaningful with.

For his session, John detailed a number of useful features of video analysis, as well as the kinds of established codes that operate around the field. Video analysis involves the observation of naturally occurring phenomena on a broad scale (ideally a camera that can apprehend the whole environment), much like Ethnography. This observation can be transcribed play-by-play, or simply observed to note any phenomena that become apparent only when the situation is viewed in this way. This can be used as a way to analyse in minute detail the situation that unfolded, picking out emergent characteristics that can inform your understanding of these events.

This is not something I had ever considered doing to my existing documentation. As I often perform for events that are video recorded for funding and research purposes (such as Algoraves for AHRC-funded TOPLAP) I often see video documentation of my performances without having any explicit use for it. From this session, John explained how these features could enhance understanding of the documentation I have available, and formulate it as a valid research concern.

The exercise portion of John’s session involved examining some raw documentation we had of our own work. I examined a static video taken from the back of a live coding set I performed at Power Lunches, as well as the documentation of my undergraduate final recital at culture lab (https://www.youtube.com/watch?v=OcwS6x4KRIU). The class then dissected the video of the session, noting when the audience performed certain actions (dancing, chatting, paying/not paying attention) and when I performed certain actions (executing lines of code, nodding my head, using body language). While I was tacitly aware of these features of my performances, I had never considered taking an analytic approach to the minutiae of performing for the public, and this has the potential to become a broader research concern.

I am aware that these techniques have been performed by Alex McLean, particularly on his video performing with duo Canute at Jubez Karlsruhe (https://www.youtube.com/watch?v=uAq4BAbvRS4), so video analysis of live coding concerts is evidently an established research concern among academics in the field.

I’d like to do some further research into the kinds of software I can use to do video analysis on existing documentation, one example I found was atlas.ti (http://atlasti.com/); however i’d like to keep my commitment to using free and open source technologies, so I am currently searching for an open alternative.