Housekeeping

• Next module meeting – NOVEMBER 3rd – 2PM
  
• Go over the module handbook
• Talk about expectations/goals (work load and help – Me!  Bren!  Hi Joey and Tom!)
• Delivery style and pace
• This blog, your blog
• Extra sessions after most classes – not today! (usually 4:30-8:00PM, Bren also scheduled to be available for some of this time)
• Our extensive use of Wikipedia on these notes is emphatically intentional – you are the best people to be engaging with a new shared conception of knowledge and fact (correct mistakes! help generate our a posteriori reality! join the conversation!)

Rationale

So.  This class is, as the handbook stated, is about creative reflections on technology, using technology.  We’re not necessarily always interested in the ‘right way’ to do things, in terms of commercial production, or ‘manual’ use of software, etc.  We are however interested in looking critically at tools, their use, and the assumptions and values they embody.

Most generally – the approach we’ll take in this class is about establishing a playful relationship with technology.  Something akin to playing with physical materials – an outlook that speaks of having some understanding of the FACT that there is no real difference between the the ‘digital’ and ‘analog’ world… It’s all the same damn world.  With that in mind:

• Last year’s module theme song – Iggy and the Stoges: Search and Destroy (studio version) – esp. right at 0:50 sec

• This year’s module theme song – 50 Cent: AYO Technology featuring Justing Timberlake – esp. the chorus
  

I hope this highlights that all approaches and intentions are welcome here:

• “I use technology in order to hate it more properly” – Nam June Paik
• “I think it’s fair to say that personal computers have become the most empowering tool we’ve ever created. They’re tools of communication, they’re tools of creativity, and they can be shaped by their user.” Bill Gates
• “Computers make it easier to do a lot of things, but most of the things they make it easier to do don’t need to be done.” Andy Rooney

This first meeting is intended to give us a grounding in the implications of digital systems to the known universe and the ways in which digital representations are used to represent (create?) linear and non-linear image, moving image and sound.

Abstraction & Representation

• What is a computer good for?

• What new facilities do they grant us in creativity and communication?
  • Temporal – dynamism and state behavior
  • Spatial – translocation of hermeneutic information
  • Ontological – objecthood and property rentention
  • Sensory – warping the humunculus
  • Bandwidth – extensions of our sensory system
  • Retention – tracing, tracking and transfer

• What are the properties of digital systems that allow for the above?
  • Configurability? Extendability? Programmability? Accessibility? Scalability?
  • ABSTRACTION!
    • the act of considering something as a general quality or characteristic, apart from concrete realities, specific objects, or actual instances.
  • Computers do not do anything, they only provide models of other things (just like the brain?)
  • “it is in model making and model use that relations of power and authority, both political and intellectual, make themselves manifest” – Ludmilla Jordanova
• McLuhan – Technology as a history of human extension
  • Mechanical abstraction – extending our physical abilities
    
  • Electrical abstraction – creating analogies to these new abilities
    
  • Digital abstraction – creating mathematically manipulable symbols from these analogs
    
  • Programmatic abstraction – once everything is a symbols, we work devise ways to do work on them
  • What is the next form of extension we are experiencing?

• Digital systems, are the most power frameworks for abstraction ever devised by earth creatures!!

• BBC Connections – Faith In Numbers (start 36:10)
• Chalres/Ray Eames – Logic by Machine (I & II)

• All things lead to philosophy – REPERCUSSIONS:
  • Analog vs. Digital is a false dichotomy?
  • All technologies are a continuation of our want/need to extend and abstract from the physical work?
  • Our models for understanding reality are increasingly distant from “classical” physical experience (i.e.: We have come to understand and manipulate the world by reference)
  • “Intuition is linear; our imaginations are weak. Even the brightest of us only extrapolate from what we know now; for the most part, we’re afraid to really stretch.” – Ray Kurtzweil

Signals & Sampling

• Signals
  • What is an analogy?
  • Transduction:
    • Converting variations in a physical quantity, such as pressure or brightness, into an electrical signal (for example)
      
  • Such ‘signals’ are characterized by their
    • period – the interval of time between successive occurrences of the same state in an oscillatory or cyclic phenomenon, in seconds.
      
    • frequency – the rate at which something occurs or is repeated over a particular period of time, in cycles/second = hertz
      
    • amplitude – the maximum extent of an oscillation, measured from the position of equilibrium.
  • Any signal can be broken down into a composite set of simple oscillations
    • Sine and Cosine oscillations
    • Adding and subtracting waves to create a complex signal
    • Bandwidth – a range of frequencies (often refers to the highest frequency in the signal, with zero as the assumed lower end)
    • The Fourier transform allows us to extract the ‘harmonic’ or oscillatory ‘roots’ of a signal
      • Interesting: The ear/brain do the same job – extracting frequency information. Much visual stimuli is assumed (incorrectly, apparently) to behave in this fashion.
      • Interesting: This is what a vocoder does
  • Simplest time-based signal representations
    • time graph
    • spectrum
  • Note – all this math stuff applies to anything changing in time or space (defined by classical physics) – WHICH IS EVERYTHING IN THE UNIVERSE!! whoa.  For example…

• Signals in Audio, Images & Video
  • Audio:
    • Frequency = Tone or note
    • Amplitude = Volume or loudness
    • Phase = “starting point” of the wave
  • Image
    • Spatial Frequency (Dimensional frequency)
    • Amplitude: “Dynamic range” in photography (how red is red)
  • Moving Image:
    • Spatial Frequency: As above, for each frame
    • Temporal Frequency: A measure of change between frames for a given vector
    • Spatio-Temporal… Multidimensionality…

• Sampling
  • “Anything physical can be represented by an analogous electrical voltage”
  • The conceptual difference between a signal and its digital representation lies the concept of “continuous” vs. discrete
  • Any signal presented in continuous time can be converted into a number set of discrete size such that no information is lost!
    • Digression:
      • What is discrete time?
      • We can measure time with greater accuracy than anything else on earth
      • 10^-43
  • This process is called “sampling” theoretically or “analog to digital conversion” in electronics
  • Nyquist’s theorum describes this processes’ potentials and limitations
  • Resolution & Sample rate
    • Audio quality examples
    • E.g.:
      • cmack_441_16bit.mp3:
      • cmack_441_8bit.mp3:
      • cmack_441_2bit.mp3:
      • cmack_220_16bit.mp3:
      • cmack_110_16bit.mp3:
      • cmack_055_16bit.mp3:
    • Image quality examples (spatial aliasing)
      • Imaging Sensor (e.g.: photography)
    • Moving image examples (temporal aliasing)
      • Wagon Wheel Effect
      • Wagon Wheel Effect demo
• ADC – Analog to Digital Conversion
• There is, of course – OUTPUT as well… DAC, Digital to Analog Conversion , Nate the mathematics guy explains aliasing

Decimal and Binary Representation

• Once we have a signal from the real world as a digital symbol, it can be presented in a number (ha!) of ways
• Exponent tables for
  • Decimal-base 10 numbers, e.g.: 24.4506954
  • Hexadecimal-base 16 numbers, e.g.: #FFCC00
  • Binary-base 2 numbers, e.g.: 10001101
    • resolution
    • number size & wrap-around
  • e.g.: adding binary numbers : 10110111 + 10101111  (What repercussions does this have for an 8-bit system?  a 16 bit system?)
  • e.g.: comparing binary numbers: 10101010 + 10001101 (The search for truth!!)

• Computer number types:
  • bits, bytes, words
  • characters (note this is listed here as a ‘number type’ because everything is actually a number! see character encoding)
  • integers (0 to 65,535 = 2^16) and other Integral Data Types
  • floating point numbers (“floats” or IEEE 754-2008)
  • boolean
  • arrays (“strings”), vectors, matrices

Typically, after we do all this defining of number types, we’re primarily concerned with how big of a normal number we can represent using binary representation.

Moving Numbers Around

• Microchips and processing power:
  • Basic computer architecture – ALU
  • Which is more ‘costly’ in terms of processing power?
    • subtraction vs. multiplication?
    • floating point subtraction vs. integer multiplication?
    • copying data vs. moving data?
    • searching for a specific value (=) vs. finding something that is greater than some value (<)
    • using the CPU versus using an offline processing chip (GPU, APU)?
  • File Storage vs. Cached or ‘in program‘ data

• As a quick taster, we’re going to do the whole module – RIGHT NOW
  
• Two tools:  Max/MSP & Arduino
  

The Max/MSP/Jitter Environment – quickly!

• The PATCH
• The INSPECTOR
• Important menu items:
  • View… Edit
  • Options… DSP Status
  • Options… segmented patch chords, etc.
• Objects (400+ installed with Max)
  • Bordered boxes which encapsulate a specific function or tool
  • Categorized as (there is overlap in here):
    • Basic – objects, messages, numbers and bangs
    • Sliders – graphical interfaces for the manipulation and display of data
    • Buttons – interface elements for on-screen manipulation
    • Interface – interface elements for presentation and GUI development
    • Data – Simple data graphs
    • Jitter – Anything from the jitter package
    • Audio – Most anything with a squiggle after it
    • Image – Non-Jitter image display

• The Magical Message Machine: Max The first program onto which MSP and Jitter were tacked on – MAX – is not really a multimedia program at all. It’s a data handler, mostly for timing based (i.e.: musically interesting) data. This musical data (events, note numbers, etc.) are called “messages” in Max, and it serves us well to really understand the way these work.

• Three types of data:
  • Message
    • Bangs (events to be scheduled – without at least one of these, nothing would ever really happen)
    • Numbers (integers and floating point numbers – CERTAIN OBJECTS ARE “TYPED”)
    • Symbols (characters, symbols and words)
    • Lists (arrays of numbers or arrays of symbols – MIDI messages, lists of commands)
  • MSP
    • Signals (audio)
    • Other stuff that varies with audio rate (e.g.: ramps, multiplications, etc.)
  • Matrix
    • Video
    • Other matrix data

Arduino – even more quickly!

• are-dween-eee-ooo
• What is it?
  • A micro-controller interface board
  • A software system that allows you to program it
  • 5V internal operation (12V power supply can be used)
  • POWER, ANALOG and DIGITAL sections of the board
    • POWER – access to onboard power and section for connection of batteries, etc.
    • ANALOG – 6 analog inputs @ 10 bit and 10KHz (1024 levels)
      • connect continuous sensors
      • read sensors
    • DIGITAL – 14 digital input/output pins (of which 6 can be used as PWM outputs)
      • on/off inputs: connect switches,
      • on/off outputs: lights, motors, solenoids, etc.
      • 6 of these pins (listed here) can be PWM outputs (like an analog output, sort of)

• There are a number of different versions:
  • Dueminalove (how to say it)
  • Diecimilia (last version of the board – not so different and still in the pipeline)
  • Nano (super small)
  • Mini (pretty small)
  • LilyPad (pretty!)

• Follow this presuming you’re on OSX (Please don’t skip the part where you install the serial drivers, PLEASE PLEASE PLEASE!)
  • Computer communicates w/ Arduino using RS-232 over USB cable
  • Programming:
    • You use numerical references to tell ‘pins’ on the chip to read input or send output
    • Always has the setup() and loop() functions
      • Setup is where you tell the Arduino chip what you’re going to do with it
      • Loop is where you perform a repetitive action that will continue whenever the board is on. Forever. (I.e.: You don’t have to be connected to the computer!!)

• Electronics in 30 seconds:
  • Voltage is a potential for flow of electrons that is prevented to varying degrees from doing so by something called resistance
  • Every physical thing has a resistance. From infinitely resistive (an ‘insulator’) to very very low resistance (a conductor or ‘superconductor’)
  • These varying resistances can be used to generate varying (analog) voltages

• Reading sensors/switches:
  • Rotation – rotary potentiometer – varying resistance – varying voltage
  • Use Arduino to measure that changing voltage and convert it into a binary, numerical representation
  • Transfer that number to Max (or whatever…)

To Do

• Go over the material from today’s class.
• Get your blog up and running. Seriously. Send Jamie, Bren, Joey and Tom the link before next meeting.
• Make a blog post about your hopes and fears for this module. Or why you don’t have hopes and fears.  Or why you don’t want to write about not having hopes and fears…
• Get/install/test all the necessary softwares… Max/MSP next week if you haven’t already.
• Buy an Arduino board if you haven’t already… they may take a while to get here.
• The Max/MSP/Jitter installation contains a good number of tutorials covering the initial structures and connections we talked about today:
  • Open Max/MSP
  • Click Help… Max Tutorials
  • Look at these this week (You don’t necessarily have to read them all, just hunt around in the example patches a bit)
    • Hello — Creating objects and connections
    • Bang! — The bang message
    • Numbers and Lists — Types of data in Max
    • Metro and Toggle — Creating automatic actions
    • Message Ordering — Debugging program flow
    • Simple Math — Performing calculations
    • Numerical User Interfaces — Sliders and dials
    • Encapsulation — Patchers inside of patchers
    • Abstractions — Creating libraries of re-usable code
    • Remote Messaging – Sending message without patchchords
    • Controlling Data Flow – Routing messages
    • Basics – Getting MIDI
    • Basic JavaScripting
  • Also have a look at these MSP Tutorials:
    • MSP Tutorial 1 – Test Tone
    • MSP Tutorial 13 – Recording and Playback
    • MSP Tutorial 16 – Record and Play Audio Files
    • MSP Tutorial 18 – Mapping MIDI to MSP
    • MSP Tutorial 23 – Viewing Signal Data

• HERE ARE A FEW MAX MSP JITTER FILES EXAMPLES and EXERCISES – MaxMaterials.zip
  • Work through these over the next 3 sessions
    
  • Use these as the basis for patches you’ll be asked to show at the beginning of each class
  • If you understand how all these work, you’re probably doing ‘ok’
    
• Look at these this week (You don’t necessarily have to read them all, just hunt around in the example patches a bit)
• These are the potentiometers you might want to buy/find - the ‘value’ of the potentiometer doesn’t matter
• MAKE A SIMPLE SOUND-ONLY MAX PATCH, CONTROLLED BY THE ARDUINO – TO SHOW THE GROUP NEXT MEETING!