mac online apple blackjack http://www.euro-online.org

Scraping

HSS8120 : Scraping

Aims

This session is intended to introduce scraping as a cultural paradigm and as an artistic (and research) activity. We will look at some examples of scraping in art, music, visualisation and performance and learn about some tools and approaches used. From there we will think about what it means to use scraped data in our own creative work. The key point is that scraping (for me) signifies an approach to gathering data that emphasises activity, effort, non linearity and contingency. It is, in some senses the opposite of ‘fake news’ whose main feature is that it is provided to us.

 

SECTION 1:A completely selective and loosely structured overview of scraping: 

Unstructured Data

Neurotic Armageddon Indicator a wall clock for the end of the world.

  • web data and conventions of use
  • figurative statistics
  • qualitative and quantitative research

Daily Paywall Paolo Cirio

  • Hacking newspaper paywalls and scraping content

Structured Culture

Deutsche Digitale Bibliothek Using semi-structured data to frame and use culture.

  • aggregating culture
  • vagaries of scale

My own work on the Bloodaxe archive.

  • audiences for new hybrid objects

Signals

Including Detektors by Martyn Howse and Shintaro Miyazaki, their maps include JR Shinjuku station, Tokyo

  • scrying
  • divining

Listening

Audio scraping as mapping a layer. The Quiet Walk (Alessandro Altavilla, Tom Schofield)

Images

Bloodaxe Archive scrapings

  • As a contemporary form of scraping to make a mark. See William Blake.

As An Industry

Data ‘sifting’ is now a substantial industry.

Conceptually/Methodologically?

What does scraping get you? What does the term do?

  • It suggests an active mode of data gathering
  • It carries with it associated activities : filtering, ordering, saving – all of which can structure your work in culturally-situated ways. Thoughts about these activities as a site of work can inform your practice.
  • It provides a series of productive metaphors which can, in turn, become practices – digging, uncovering, sifting.
  • It can provide kinds of gesture – think burins, trowels, fingernails.

When can you scrape?

For instance on Wired.com you can’t:

copy, harvest, crawl, index, scrape, spider, mine, gather, extract, compile, obtain, aggregate, capture, or store any Content, including without limitation photos, images, text, music, audio, videos, podcasts, data, software, source or object code, algorithms, statistics, analysis, formulas, indexes, registries, repositories, or any other information available on or through the Service, including by an automated or manual process or otherwise, if we have taken steps to forbid, prohibit, or prevent you from doing so;

Don’t forget to read the robots.txt such as this one.

SECTION 2: To Work!

First we’ll need python 2.7

And also pip

And it helps to have Sublime Text 2

Task 1.

Writing a real python scraper with Python looking at seismic activity. This data is in a semi-structured state. How can we make it useful for instance to make this.

Task 2.

Scrying for wifi with an android phone or an iPhone. Is this Hertzian Space?

Explore and:

  • look for features of interest
  • find secrets
  • look for things you can use
  • think about the way you move through a building

For instance SSID 1-line ascii art.

Task 3.

Using an API. Many of them need you to register and receive a KEY.

Using the mediawiki api what can we find on our subject. How could this be used computationally to tell us something that we couldn’t simply read? What does this mean for the humanities – for humanism?

For instance we can programmatically generate a list of images for a given subject. Like Earthquakes.

Look at the tutorial here. What else can you find of use?

SECTION 3: To Play!

Write a scraper (in Python) that turns unusable public data into something useful.

Writing a scraper.

  • identify a changing data source on web (there’s a cool one here )
  • check for a robots.txt file to see if what you want to do is allowed. The site above has one here.
  • check any licensing information that will tell you what you can and can’t do with the data.
  • look at the HTML and see what we can identify that uniquely identifies the thing that we want
  • adapt the earthquake scraper to get it

 

 

Current Students

Students currently studying Master in Creative Arts Practice

Ares Rabe 

Shawn Ma

Chloe Manyue Yu 

Chrissy Shou Yu Chen

Ben Woolsey

Megan Wilson

Jade Mallabone

Garry Lydon

Michael Hirst

Sarah Davy

Alexei Crawley

Lewis Brown

Daniel Bradwell

Ashley Bowes

 

 

Previous Students

CAP

2015-2016

Meena DaneshyarMaria Clemente – AlbaceteSean Cotterill

Xiyuan TanYue WangRiar Rizaldi

Daniel Parry

 

Previous Students

CAP

2014-2015

MeteorEdmund Nesveda

Clive WrightChilly Rain

Wenya ChenTrong Cuong Dao

Yousif AbdulghaniTan

BartiZhang Wei

 

2013-2014

Yinzhen BaoTatiana Fujimori
Jaejun HwangSaksit Knunkitti
Wenchang LinClare Robertson
Tunc Karkutoglu

Mres Digital Media
Adrian ParkBen HoldenIsobel Taylor
James DavollNina LimardoXue Yan
Aaron SmilesAlessandro AltavillaAndrew Nixon
Andrzwej WojtasBen FreethBen Thompson
Helen CollardJane DudmanJoseph Pochciol
Pengfei ZhangSanjay Mortimer
Ewelina Aleksandrowicz (Tikul)

 

Menu

Theory/ Practice Research/ Enterpirse Live Electronic Performance Digital Media Curriculum