(It’s good at the beginning of projects to research what’s come before, and Joe is pretty spectacular at finding references and explaining what’s interesting about each one. He’s done this for a few projects now, but we’ve never made his research public. Last year Joe put together a set of appearances of videophones in film. It’s a lovely collection, and it was a stimulating way to think around the subject! So I asked him to share it here. -Matt W)
Features wall-mounted analogue videophone. Joh Fredersen appears to use four separate dials to arrive at the correct frequency for the call. Two assign the correct call location and two smaller ones provide fine video tuning. He then picks up a phone receiver with one hand and uses the other to tap a rhythm on a panel that is relayed to the other phone and displayed as flashes of light to attract attention.
A public booth containing a large phone unit. The system communicates that it is in a ‘ready’ state through the screen. A call is made by entering a number into the type-pad and a connection established on pickup
An outdoor, public phone service. Network information is displayed on screen implying that it is subject to change. When Deckard begins to dial a ‘transmitting’ notification appears. The cost of the call is shown when the receiver line is closed.
The screen is used as a canvas, covered in scrawled messages. A cross indicates the optimal position for viewer’s head.
A “Picturephone” uses a rotary dial to make calls. A camera housed in the device is distinctly visible in a trapezium above the screen. Set in the future, the device seems to be a new invention that Marge isn’t quite used to yet, as she visibly crosses her fingers guaranteeing that Homer will behave at the forthcoming wedding.
“The things we are about to share our environment with are born themselves out of a domestication of inexpensive computation, the ‘Fractional AI’ and ‘Big Maths for trivial things’ that Matt Webb has spoken about.
‘Making Things See’ could be the the beginning of a ‘light-switch’ moment for everyday things with behaviour hacked-into them. For things with fractional AI, fractional agency – to be given a fractional sense of their environment.
This film uses found-footage from computer vision research to explore how machines are making sense of the world. And from a very high-level and non-expert viewing, it seems very true that machines have a tiny, fractional view of our environment, that sometimes echoes our own human vision, and sometimes doesn’t.
For a long time I have been struck by just how beautiful the visual expressions of machine vision can be. In many research papers and Siggraph experiments that float through our inboxes, there are moments with extraordinary visual qualities, probably quite separate from and unintended by the original research. Something about the crackly, jittery but yet often organic, insect-like or human quality of a robot’s interpetation of the world. It often looks unstable and unsure, and occasionally mechanically certain and accurate.
“Imagine it as, perhaps, the infant days of a young machine intelligence.”
The Robot-Readable World is pre-Cambrian at the moment, but machine vision is becoming a design material alongside metals, plastics and immaterials. It’s something we need to develop understandings and approaches to, as we begin to design, build and shape the senses of our new artificial companions.
As a sidenote, this has reminded me that I was long ago inspired by Paul Bush’s ‘Rumour of true things’ which is ‘constructed entirely from transient images – including computer games, weapons testing, production line monitoring and marriage agency tapes’ and a ”A remarkable anthropological portrait of a society obsessed with imaging itself.’. This found-footage tactic is fascinating: the process of gathering and selecting footage is an interesting R&D exercise, and cutting it all together reveals new meanings and concepts. Something to investigate, as a method of research and communication.
This is a blog post about a type of work we find successful – namely, video prototyping – and why we think it’s valuable.
We’ve made quite a few films in the last couple of years, that have had some success – in how they describe products, technologies and contexts of their use in public.
We’re lucky enough to work with Timo Arnall, as creative director, who guides all of our film output and is central to the way that we’ve been able to use the moving image as part of our design process – more of which later.
Film is a great way to show things that have behaviour in them – and the software, services and systems that literally animate them.
A skilled film-maker can get across the nature of that behaviour in a split-second with film – which would take thousands of words or ultra-clear infographics.
They can do this along with the bonuses of embedding humour, emotional-resonance, context and a hundred other tacit things about the product.
Film is also an easy way to show things that don’t exist yet, or can’t exist yet – and make claims about them.
We’ve all seen videos by corporations and large design companies that are glossy and exciting illustrations of the new future products they’ll almost certainly never make.
Some are dire, some are intriguing-but-flawed, some are awesome-but-unbelievable.
This is fine!
More than fine!
Ultimately they are communications – of brand and ambition – rather than legal promises.
Some of these communications though – have enormous purchase on our dreams and ambitions for years afterwards – for better, or for worse.
I’m thinking particularly of the Apple ‘Knowledge Navigator’ film of 1987, important in some of the invention it foreshadowed, even while some of the notions in it are now a little laughable.It was John Sculley‘s vision – not Jobs – and was quite controversial at the time.
Nevertheless, designers, technologists and businesses have pursued those ideas with greater and lesser success due to the hold that film had over the collective psyche of the technology industry for, say, 20 years.
“We began with as much research as we could do in a few days. We talked with Aaron Marcus and Paul Saffo. Stewart Brand’s book on the “Media Lab” was also a source—as well as earlier visits to the Architecture Machine Group. We also read William Gibson’s “Neuromancer” and Verber Vinge’s “True Names”.
Of course the company that authored it, Apple, I’d argue built it eventually to some extent with the iPhone.
The gravity well of the knowledge navigator was enormous, and fittingly, Apple punched out of it first with real product.
They are also commercial acts – perhaps with altruistic or collegiate motives woven in – but commercial all the same.
They illustrate a desirable microfuture wherein Brand-X’s product or services are central.
Dubberly, in his piece about Knowledge Navigator points out the importance of this – the influence the film had on the corporate imagination of the company, and of competitors:
“What is surprising is that the piece took on a life of its own. It spawned half a dozen or more sequels within Apple, and several other companies made similar pieces. These pieces were marketing materials. They supported the sale of computers by suggesting that a company making them has a plan for the future.
One effect of the video was engendering a discussion (both inside Apple and outside) about what computers should be like. On another level, the videos became a sort of management tool.
They suggested that Apple had a vision of the future, and they prompted a popular internal myth that the company was “inventing the future.”
Very recently, we’ve seen the rise of two other sub-genres of concept video.
It’s very early days for both, but both are remarkable for the ‘communications gravity’ they generate for very different commercial endeavours.
First of all – what Bruce Sterling has called the ‘vernacular video’ – often of products in use – created for startups and small companies.
Adam Lisagor has been hailed as the leader in this genre by Fast Company – and his short films for the like of Flipboard, Square and Jawbone have in many ways been defining of the vernacular in that space. They are short, and understated – and very clear about the central benefit of the product or service. Perfect for the sharing and re-sharing. Timo’s written about Adam’s work previously on his personal blog, and I’d agree with him when he says “He’s good at surfacing the joy and pleasure in some of the smallest interactions”. They serve as extraordinarily elegant pitches for products and services that are ‘real’ i.e. has usually already been made.
They are often very personal and emotive, but mix in somewhat of a documentary approach to making and construction around prototypes. They serve as invitations to support a journey.
So far, so what?
Video is a well-known way of communicating new or future products & services that reaches the mainstream – and we are seeing a boom in the amount of great short communication about design, invention and making with ever-higher production value as the tools of creation fall in cost, and the techniques of using them become available to small, nimble groups of creators.
Well, we think that’s just half of the potential of using video.
There is a great deal of potential in using video as a medium for design itself – not just communicating what’s been designed, or imagined.
Jack and Timo drew this for me a couple of months ago when we were discussing an upcoming project.
We were talking about the overlap between invention and storytelling that occurs when we make films, and how and why that seems to happen.
On the right is the ‘communications gravity’ that I’ve already talked about above – but the left-hand circle of the Venn is ‘product invention’.
During a project like Mag+ we used video prototyping throughout – in order to find what was believable, what seemed valuable, and how it might normalise into a mainstream product of worth.
In the initial workshopping stages we made very quick sketches with cut-up magazines, pasted together and filmed with an iPhone – but then played back on an iPhone to understand the quality of the layout and interaction on a small screen.
From these early animatics to discuss with our client at Bonnier R&D, we moved to the video prototype of the chosen route.
There were many iterations of the ‘material quality’ of the interface – we call it the ‘rulespace’ – the physics of the interactions, the responsiveness of the media – tuned in the animation and video until we had something that felt right – and that could communicate it’s ‘rightness’ in film.
You find what is literally self-evident.
You are faking everything except this ‘rulespace’ – it’s a block of wood, with green paper on it. But as we’ve written before, that gets you to intuitions about use and gesture – what will make you tired, what will feel awkward in public places, how it sits on the breakfast table.
Finding the rulespace is the thing that is the real work – and that is product invention through making a simulation.
We are making a model of how a product is, to the degree that we can in video. We subject it to as much rigour as we can in terms of the material and technological capabilities we think can be built.
It must not be magic, or else it won’t feel real.
I guess I’m saying sufficiently-advanced technology should be distinguishable from magic.
Some of that is about context – we try and illustrate a “universe-next-door” where the new product is the only novelty. Where there is still tea, and the traffic is still miserable.
This increases belief in our possible microfuture to be sure – but it also serves a purpose in our process of design and invention.
The context itself is a rulespace – that the surface and behaviour of the product must believably fit into for it to be successful. It becomes part of the material you explore. There are phenomena you discover that present obstacles and opportunities.
That leads me to the final, overlapping area of the Venn diagram above – “New Grammar”
In it, Arthur frames the realtionship between ‘natural phenomena’ as discovered and understood by science, and how technology is that which ‘programs phenomena to our use’.
“That a technology relies on some effect is general. A technology is always based on some phenomenon or truism of nature that can be exploited and used to a purpose. I say “always” for the simple reason that a technology that exploited nothing could achieve nothing.”
“Phenomena are the indispensable source from which all technologies arise. All technologies, no matter how simple or sophisticated, are dressed-up versions of the use of some effect—or more usually, of several effects.”
“Phenomena rarely can be used in raw form. They may have to be coaxed and tuned to operate satisfactorily, and they may work only in a narrow range of conditions. So the right combination of supporting means to set them up for the purpose intended must be found.”
“A technology is a phenomenon captured and put to use. Or more accurately I should say it is a collection of phenomena captured and put to use. I use the word “captured” here, but many other words would do as well. I could say the phenomenon is harnessed, seized, secured, used, employed, taken advantage of, or exploited for some purpose. To my mind though, “captured and put to use” states what I mean the best.”
“…technology is more than a mere means. It is a programming of phenomena for a purpose. A technology is an orchestration of phenomena to our use.”
This leads me to another use of film we find valuable – as documentary evidence and experimental probe. What Schulze calls ‘science on science’.
The work that he and Timo did on RFID exploring it’s ‘material’ qualities through film is a good example of this I think.
It’s almost a nature documentary in a way, pointing and poking at a phenomena in order to capture new (often visual) language to understand it.
Back to W.Brian Arthur:
“…phenomena used in technology now work at a scale and a range that casual observation and common sense have no access to.”
I think this is what Jack and Timo are trying to address with work such as ‘Immaterials’, and reffering to in the centre of their Venn – creating new grammar is an important part of both design investigation, and communication. It is an act of synthesis that can happen within and be expressed through the film-making process.
Arthur’s book goes on to underline the importance of such activities in invention:
“A new device or method is put together from the available components—the available vocabulary—of a domain. In this sense a domain forms a language; and a new technological artifact constructed from components of the domain is an utterance in the domain’s language. This makes technology as a whole a collection of several languages, because each new artifact may draw from several domains. And it means that the key activity in technology—engineering design—is a form of composition. It is expression within a language (or several).”
He goes on to quote Paul Klee on the the importance of increasing the grammar we have access to:
“…even adepts can never fully keep up with all the principles of combination in their domain. One result of this heavy investment in a domain is that a designer rarely puts a technology together from considerations of all domains available. The artist adapts himself, Paul Klee said, to the contents of his paintbox. “The painter… does not fit the paints to the world. He fits himself to the paint.” As in art, so in technology. Designers construct from the domains they know.”
I think one of the biggest rewards of this sort of work is finding new grammar from other domains. Or what Arthur calls the importance of ‘redomaining’ in invention.
“The reason… redomainings are powerful is not just that they provide a wholly new and more efficient way to carry out a purpose. They allow new possibilities.”
“A change in domain is the main way in which technology progresses.”
“…a single practitioner’s new projects typically contain little that is novel. But many different designers acting in parallel produce novel solutions: in the concepts used to achieve particular purposes; in the choice of domains; in component combinations; in materials, architectures, and manufacturing techniques. All these cumulate to push an existing technology and its domain forward.”
“At the creative heart of invention lies appropriation, some sort of mental borrowing that comes in the form of a half-conscious suggestion.”
“…associates a problem with a solution by reaching into his store of functionalities and imagining what will happen when certain ones are combined.”
“Invention at its core is mental association.”
It’s not necessarily an end product we are after – that comes through more thinking through making. And it also comes from a collegiate conversation using new grammars that work unearths.
But to get a new language, a map, even if it’s just a pirate map, just a confident sketch in an emerging territory – is invaluable in order to provoke the mental association Arthur refers to.
We’re going to continue to experiment with video as a medium for research, design and communication.
Recent efforts like ‘Clocks for Robots‘ are us trying to find something like a sketch, where we start a conversation about new grammar through video…
About a decade ago – I saw Oliver Sacks speak at the Rockerfeller Institute in NYC, talk about his work.
A phrase from his address has always stuck with me since. He said of what he did – his studies and then the writing of books aimed at popular understanding of his studies that ‘…sometimes the stories are the science’.
Sometimes our film work is the design work.
Again this is a commercial act, and we are a commercial design studio.
But it’s also something that we hope unpacks the near-future – or at least the near-microfutures – into a public where we can all talk about them.
It’s Friday. Here are links to some of what’s been blowing around the studio this week.
There’s an interview Geoffrey Hoyle about his 70’s book 2011: Living in the Future looking back at looking forward with some lovely, yet not altogether pleasing to the author, illustrations. via @futuryst
Jones pointed us to filmonpaper.com, Eddie Shannon’s extraordinary archive of film posters.
Dentsu London are developing an original product called Suwappu. Suwappu are woodland creatures that swap pants, toys that come to life in augmented reality. BERG have been brought in as consultant inventors, and we’ve made this film. Have a look!
Suwappu is a range of toys, animal characters that live in little digital worlds. The physical toys are canvasses upon which we can paint worlds, through a phone (or tablet) lens we can see into the narratives, games and media in which they live.
We think Suwappu represents a new kind of media platform, and all sorts of social, content and commercial possibilities.
Each character lives in different environments: Badger lives in a harsh and troubled world, Deer lives in a forest utopia, Fox in an urban garden, Tuna in a paddling pool of nicely rendered water. The worlds also contain other things, such as animated facial expression, dialogue pulled from traditional media and Twitter, and animated sidekick characters.
The first part of this film imagines and explores the Suwappu world. Here we are using film to explore how animation and behaviours can draw out character and narrative in physical toy settings. The second part is an explanation of how Suwappu products might work, from using animal patterns as markers for augmented reality, to testing out actual Augmented Reality (AR) worlds on a mobile phone.
We wanted to picture a toy world that was part-physical, part-digital and that acts as a platform for media. We imagine toys developing as connected products, pulling from and leaking into familiar media like Twitter and Youtube. Toys already have a long and tenuous relationship with media, as film or television tie-ins and merchandise. It hasn’t been an easy relationship. AR seems like a very apt way of giving cheap, small, non-interactive plastic objects an identity and set of behaviours in new and existing media worlds.
We see the media and animation content around the toys as almost episodic, like comic books. Their changing characters, behaviours and motivations played out across different media.
Toys are often related as merchandise to their screen based counterparts. Although as products toys have fantastic charm and an awesome legacy. They feel muted in comparison to their animated mirror selves on the big screens. As we worked with Dentsu on the product and brand space around the toys we speculated on animated narratives to accompany the thinking and characters developed.
In the film, one of the characters makes a reference to dreams. I love the idea that the toys in their physical form, dream their animated televised adventures in video. When they awake, into their plastic prisons, they half remember the super rendered full motion freedoms and adventures from the world of TV.
Each Suwappu character can be split into two parts, each half can be swapped with any other resulting in a new hybrid character. Each character has its own personality (governed by its top half) and ‘environment’ (dictated by its bottom half). This allows the creatures to visit each other’s worlds, and opens up for experimentation with the permutations of characters personality and the worlds that they inhabit. It’s possible to set up games and narratives based on the ways that the characters and their pants are manipulated.
This is not primarily a technology demo, it’s a video exploration of how toys and media might converge through computer vision and augmented video. We’ve used video both as a communication tool and as a material exploration of toys, animation, augmented reality and 3D worlds. We had to invent ways of turning inanimate models into believable living worlds through facial animation, environmental effects, sound design and written dialogue. There are other interesting findings in the exploration, such as the way in which the physical toys ‘cut out’ or ‘occlude’ their digital environments. This is done by masking out an invisible virtual version of the toy in 3D, which makes for a much more believable and satisfying experience, and something we haven’t seen much of in previous AR implementations.
We all remember making up stories with our toys when we were young, or our favourite childhood TV cartoon series where our toys seemed to have impossible, brilliant lives of their own. Now that we have the technology to have toys soak in media, what tales will they tell?
It’s a Friday afternoon, and I’m about to catch a train to Sheffield. But before I do that, it would be a shame not to round-up some links from the studio mailing list – especially given the box-fresh surroundings of the new BERG website.
Via infosthetics comes this set of photographs from Brian Steen. They depict data over time – in this case, the population of Germany from 1994 and 2009 (in red and white). The images are long exposures of data extruded through space, in a similar technique to that of Making Future Magic. It’s striking to see this technique used for visualisation of data, though – and it’ll be interesting to see where Brian goes with it.
Damjan Stanković’s Cipher Glass shows you what’s in it when it’s full of fluid – the name picked out in negative-space on the side when a particular colour fluid is in the glass.
Nick pointed out E4’s “Mess With The Misfits” promotional campaign. It’s really clever – dynamically compositing images and text from Facebook into Flash video; it’s also very tasteful, and leaves no trace on your Facebook profile unless you ask it to. A slick, inventive piece of imaging trickery; very cool.
The Official Ralph Lauren 4D Experience is a bombastic name for an impressive piece of motion graphics. Projected onto the side of their 1 New Bond Street store, it uses carefully mapped graphics and video to transform the outside of the store. My favourite segments are the ones where they physically reconfigure the form of the building, such as the transformation at 1:40. It’s always interesting to see the way large brands are using such techniques for promotional purposes.
Russell spotted Mayo Nissen’s City Tickets. City Tickets was a final thesis project at CIID. It’s a service that proposes using existing parking machines as a platform for citizens to feedback on issues with infrastructure – giving them small, receipt-like pieces of paper to feed back about local issues on. Mayo explains:
City Tickets makes the bureaucratic and opaque workings of governance more transparent and accountable, while redefining the balance of power supporting participatory urban planning and management processes. Updating current machines to also issue city tickets in addition to existing parking tickets allows this existing infrastructure, without the inclusion of any costly additional technology, to be reconsidered as a way to make neighbourhoods more liveable and cities more responsive to the needs and desires of their inhabitants.
It’s simple, acute, and charming: it already feels like a service I’d like to use. And, as we explored in the Media Surfaces films, it’s another exploration of print being quick.
Last week Timo and I finished filming and editing Nearness. Earlier in the year BERG was commissioned by AHO/Touch to produce a series of explorations into designerly applications for RFID (more to come on what that means). Over the coming weeks BERG will be sharing the results of the work here and on the Touch blog.
The film Nearness explores interacting without touching. With RFID it’s proximity that matters, and actual contact isn’t necessary. Much of Timo’s work in the Touch project addresses the fictions and speculations in the technology. Here we play with the problems of invisibility and the magic of being close.
So I’ve been thinking about hands and arms. I started by thinking of extremely small hands, on my hands. So here are some drawings from that thinking.
This drawing is of a toy that shrinks your hands down so you can play in a small world, with small figures. Your fingers are all connected up to a group of flex sensors, which converts the analogue movement to a cluster of servos. The servos collectively control fingers on the small hands by tightening or loosening. So the movements of your fingers become roughly and awkwardly analogous to those of the small hands in the toy. There is also a screen inside some goggles hooked up to a small camera in a glass ball between the two small arms. So when you look in to the goggles, you see what is in front of your arms. There are two wheels which you can twist to point the camera in different directions, like an eye. Kind of like an analogue version of virtual reality, only right in front of you and not virtual.
I would also like to have a very small hand at the end of my finger. To pick up pens and things. You control the small hand on one finger using your other fingers, with flex sensors (same as above). You lose one of your big hands to gain a little hand on the end of one of your fingers.
I came across Chad Thornton‘s work. He is at Google now, but he made a mechanical finger as part of his work at Carnegie Mellon Interaction Design programme (nice video here).
Maybe I’m carrying some latent affection for the Radio Shack Armatron here, I don’t know. These themes are common in films. This must be informed by Ripley’s Power Loader from Aliens:
The belt buckle, and rubberised keyboard make her rig seem really convincing, her trainers too, and how she locks into the unit. The cyborg fingers for typing in Ghost in the Shell are nice too.
No doubt there are more. It makes me think of Robocop‘s gun hip too although slightly off topic.
I like them, robot arms. I see them as a celebration of industrial process. I predict they will become a more widespread part of our lives. They are cheaper now (it appears that non-load bearing ones don’t require three phase power either) and since they are multi modal they can perform many tasks, in strange contexts. No doubt FDM or other fittings are/will be available, implications of that could be very large. Imagine a robot arm in your drive thr(o)u(gh), changing a tyre, and then printing out your happy meal. Our lives could become peppered with arrays of multi-buildy-arms.
Robotlab (via Roger Ibars) are a German partnership who have used industrial robot arms to perform a DJ set. Witnessing the arms is as important as their role. I find them disconcertingly accurate, mechanised confidence in something typically so analogue and expert and careful. There is also something about their inflexibility, their inability to reach inside certain arcs, too close to themselves. I like the way they occasionally find a sync with each other, and at other times drift out. I think these guys have a business model set up around this, so I’m very interested to see how that develops.