Blog posts tagged as 'augmented reality'

Dentsu London are developing an original product called Suwappu. Suwappu are woodland creatures that swap pants, toys that come to life in augmented reality. BERG have been brought in as consultant inventors, and we’ve made this film. Have a look!

Suwappu is a range of toys, animal characters that live in little digital worlds. The physical toys are canvasses upon which we can paint worlds, through a phone (or tablet) lens we can see into the narratives, games and media in which they live.

Dentsu London says:

We think Suwappu represents a new kind of media platform, and all sorts of social, content and commercial possibilities.

Each character lives in different environments: Badger lives in a harsh and troubled world, Deer lives in a forest utopia, Fox in an urban garden, Tuna in a paddling pool of nicely rendered water. The worlds also contain other things, such as animated facial expression, dialogue pulled from traditional media and Twitter, and animated sidekick characters.

The first part of this film imagines and explores the Suwappu world. Here we are using film to explore how animation and behaviours can draw out character and narrative in physical toy settings. The second part is an explanation of how Suwappu products might work, from using animal patterns as markers for augmented reality, to testing out actual Augmented Reality (AR) worlds on a mobile phone.

We wanted to picture a toy world that was part-physical, part-digital and that acts as a platform for media. We imagine toys developing as connected products, pulling from and leaking into familiar media like Twitter and Youtube. Toys already have a long and tenuous relationship with media, as film or television tie-ins and merchandise. It hasn’t been an easy relationship. AR seems like a very apt way of giving cheap, small, non-interactive plastic objects an identity and set of behaviours in new and existing media worlds.

Schulze says:

We see the media and animation content around the toys as almost episodic, like comic books. Their changing characters, behaviours and motivations played out across different media.

Toys are often related as merchandise to their screen based counterparts. Although as products toys have fantastic charm and an awesome legacy. They feel muted in comparison to their animated mirror selves on the big screens. As we worked with Dentsu on the product and brand space around the toys we speculated on animated narratives to accompany the thinking and characters developed.

In the film, one of the characters makes a reference to dreams. I love the idea that the toys in their physical form, dream their animated televised adventures in video. When they awake, into their plastic prisons, they half remember the super rendered full motion freedoms and adventures from the world of TV.

Each Suwappu character can be split into two parts, each half can be swapped with any other resulting in a new hybrid character. Each character has its own personality (governed by its top half) and ‘environment’ (dictated by its bottom half). This allows the creatures to visit each other’s worlds, and opens up for experimentation with the permutations of characters personality and the worlds that they inhabit. It’s possible to set up games and narratives based on the ways that the characters and their pants are manipulated.

This is not primarily a technology demo, it’s a video exploration of how toys and media might converge through computer vision and augmented video. We’ve used video both as a communication tool and as a material exploration of toys, animation, augmented reality and 3D worlds. We had to invent ways of turning inanimate models into believable living worlds through facial animation, environmental effects, sound design and written dialogue. There are other interesting findings in the exploration, such as the way in which the physical toys ‘cut out’ or ‘occlude’ their digital environments. This is done by masking out an invisible virtual version of the toy in 3D, which makes for a much more believable and satisfying experience, and something we haven’t seen much of in previous AR implementations.

We all remember making up stories with our toys when we were young, or our favourite childhood TV cartoon series where our toys seemed to have impossible, brilliant lives of their own. Now that we have the technology to have toys soak in media, what tales will they tell?

Following iPad light painting, we’ve made two films of alternative futures for media. These continue our collaboration with Dentsu London and Timo Arnall. We look at the near future, a universe next door in which media travels freely onto surfaces in everyday life. A world of media that speaks more often, and more quietly.

“The Journey” is the second ‘video sketch’ in the pair with ‘Incidental Media’ – this time looking at the panoply of screens and media surfaces in a train station, and the opportunities that could come from looking at them slightly differently.

The Journey

The other film can be seen here.

There’s no real new technology at play in any of these ideas, just different connections and flows of information being made in the background – quietly, gradually changing how screens, bits of print ephemera such as train tickets, and objects in the world can inter-relate to make someone’s journey that bit less stressful, that bit more delightful.

There’s a lot in there – so I wanted to unpack a few of the moments in the film in this (rather long!) blog post and examine them a bit.

The film can be divided into two halves – our time in the station, and our time on the train.

The train journey itself is of course the thing at the centre of it all – and we’re examining how what we know about the journey – and the train itself, in some cases – can pervade the media surfaces involved in ways that are at once a little less ‘utilitarian’ and a little more, well, ‘useful’…

The first group of interventions could be characterised as the station wrapping around you, helping you get to your seat, on your train, for your journey, with the least stress.

Let’s start at the ticket machine.


The screen supposes two things – that it knows where it is (it doesn’t move around much) and it knows where your train (in this case, “Arthur” – trains are people too!) is leaving from, and when. So why not do a simple bit of reassurance here? It’s twenty minutes to Arthur’s departure and it’s a 3 minute walk.

You’ve got 17 minutes to play with! Get a sandwich? A coffee? Or go and find your seat…

Before we do that I just want to point our something about the ticket machine itself…


There’s the screen we’ve been interacting with to get our ticket, but there’s also a LED scroller above that.

As you can see in the concept sketch below, we’ve supposed that the scroller could give reassurance to the people in the queue behind you – maybe displaying the average turn-around-time of serving tickets to travellers, so if there is a queue, you’ll know how quickly it might move.



I think when I was drawing this I had in mind the awesome-but-as-yet-unrealised scheme by Lisa Strausfeld and Pentagram NYC for a videowall in Penn Station.



I think I first saw this presented by Lisa Strausfeld at a conference some 8 or so years ago now, but it’s still wonderful. The large video wall has loads of different layers of information kind of interpolated and displayed all at once, at different ‘resolutions’.

So that if you’re approaching the station from down the street you read some overall information about the running of the station that day, and the time, and as you get closer you see news and stock prices, then closer again and you actually see the train times when you get close enough to crane your neck up at them.



Really clever, and a huge influence on us. The notion of several ‘reads’ of the information being presented on the same surface – if handled well, as in the Pentagram proposal – can be very powerful.

We’ve taken a much less high-tech approach, using the multitude of existing screens in the station, but staging the information they present intelligently in a similar way as you approach the platform and your train itself.

For instance, little messages on concourse screens about how busy the station is overall that morning…



As we get to our platform we get the message that the train is going to pretty full but the station systems know where the bulk of reserved seats are, and can give us a little timely advice about where to hunt for a free place to sit…


We’ve hinted in this image at a little bit of nice speculative quiet new technology that could be placed by the station workers: magnetically-backed e-ink signs – again displaying reassuring information about where the busy portions of the train will be.


These little inventions have hopefully got you to your train (Arthur, remember?) on time, and in a more of a relaxed state of mind. So, as we board the train we might have time to note that this is Arthur’s favourite route…


If not, it doesn’t matter. It’s not a functional improvement to your journey but these touches lead to an appreciation of the service’s scale or reach and, if you are a regular traveller, inject a bit of recognition and delight into the otherwise routine.

Once onboard, we continue to explore opportunities for these incidental, different reads of information to both inform and delight.

In the first film ‘Incidental Media’, we introduce the concept of “Print can be quick” – looking at all the printed ephemera around us and how it can be treated as a media surface for more personalised, contextualised or rapidly-updated information.

After all, most of the printed matter associated with a train journey is truly print-on-demand: your tickets, your receipts and, as in this example, the printed reservation stub placed on the seat by the train attendants.


Here we wanted to look to the reassurances and reads that one takes of the reservation stubs as you move down the carriage – either with a reserved seat to find, or perhaps without a reservation on a busy train, opportunistically looking for an unoccupied seat that might be reserved for a latter portion of the train’s total journey.

In one of our concept sketches below we’re exploring that first case – could your ticket be the missing jigsaw piece to the reservation stub?

A bit Willy Wonka magic ticket!


Privacy would be preserved by just using your first initial – printed large with salutations, attracting your eye easily to zero in on your seat as perhaps you struggle down the aisle with your baggage.

The final version used in the film takes this on board, but balances it a little more with the second use-case, that of the opportunistic search for a free seat by someone without a reservation. To answer that case, the portion of the journey that the seat is occupied for is clearly legible, whereas the initials of the traveller are only visible on scrutiny.



If it is indeed your reserved seat, on closer scrutiny you’ll also notice the weather forecast for your destination…

Again – worth noting brilliant past work in this area that’s an influence on this idea. Our friend Brian Suda’s redesign of an airline boarding pass that uses typographical hierarchy of the printed object to reassure and delight.



Here you can see that the time of your flight is clearly visible even if your boarding pass is on the floor.

Lovely stuff.

Finally, some pure whimsy!

We wanted again to examine the idea that print can be nimble and quick and delightful – creating new forms of post-digital ephemera for collecting or talking about.

First of all, using the ticket to introduce you again to Arthur, your train, and perhaps extending that to recognising the last time you travelled together.



But let’s go further.

We know that we’re going to be passing certain places at certain times, to some accuracy, during our journey.

The burgeoning amount of geo-located data about our environment means we could look to provide snippets from Wikipedia perhaps, with timings based on how they intersect with your predicted journey time – alerting you to interesting sights just as they pass by your window.



These tiny, personalised, collectable paper-spimes provide a kind of papernet augmented-reality – giving a routine journey an extra layer of wonder and interest.



As with “Incidental Media”, we’ve tried in “The Journey” to illustrate ‘polite media’ tightly bound to and complimenting one’s context. Media that lives and thrives usefully in the interstices and intervals of everyday routine and technology – indeed ‘making future magic’ instead of the attention arms race that the near-future of urban screens and media could potentially devolve into.

The Journey is brought to you by Dentsu London and BERG. Beeker has written about the films here.

Thank you to Beeker Northam (Dentsu London), and Timo Arnall, Campbell Orme, Matt Brown, and Jack Schulze!

Following iPad light painting, we’ve made two films of alternative futures for media. These continue our collaboration with Dentsu London and Timo Arnall. We look at the near future, a universe next door in which media travels freely onto surfaces in everyday life. A world of media that speaks more often, and more quietly.

Incidental Media is the first of two films.

The other film can be seen here.

Each of the ideas in the film treat the surface as a focus, rather than the channel or the content delivered. Here, media includes messages from friends and social services, like foursquare or Twitter, and also more functional messages from companies or services like banks or airlines alongside large traditional big ‘M’ Media (like broadcast or news publishing).

All surfaces have access to connectivity. All surfaces are displays responsive to people, context, and timing. If any surface could show anything, would the loudest or the most polite win? Surfaces which show the smartest most relevant material in any given context will be the most warmly received.

I recently encountered this mixing in surfaces. An airline computer spoke to me through SMS. This space is normally reserved for awkwardly typed highly personal messages from friends. Not a conversational interface with a computer. But now, those pixels no longer differentiate between friends, companies and services.

Mixing Media

How would it feel if the news ticker we see as a common theme in broadcast news programmes begun to contain news from services or social media?

I like the look of it. The dominance of linear channel based screens is distorted as it shares unpredictable pixels and a graphic language with other services and systems.

Ambient listening

This screen listens to its environment and runs an image search against some of the words it hears. I’ve long wanted to see what happens if the subtitles feed from BBC television broadcast content was tied to an image search.

It feels quite strange to have a machine ambiently listening to words uttered even if the result is private and relatively anodyne. Maybe it’s a bit creepy.

Print can be quick

This sequence shows a common receipt from a coffee shop and explores what happens when we treat print as a highly flexible, context-sensitive, connected surface, and super quick by contrast to say video in broadcast.

The receipt includes a mayorship notification from foursquare and three breaking headlines from the Guardian news feed. It turns the world of ticket machines, cash registers and chip-and-pin machines into a massive super-local, personalised system of print-on-demand machines. The receipt remains as insignificant and peripheral as it always has, unless you choose to read it.

Computer vision

The large shop front shows a pair of sprites who lurk at the edges of the window frames. As pedestrians pass by or stand close, the pair steal colours from their clothes. The sketch assumes a camera to read passers-by and feed back their colour and position to the display.

Computer vision installations present interesting opportunities. Many installations demand high levels of attention or participation. These can often be witty and poetic, as shown here by Matt Jones in a point of sale around Lego.

We’ve drawn from great work from the likes of Chris O’Shea and his Hand from Above project to sketch something peripheral and ignorable, but still at scale. The installation could be played with by those having their colours stolen, but it doesn’t demand interaction. In fact I suspect it would succeed far more effectively for those viewing from afar with no agency over the system at all.

In contrast to a Minority Report future of aggressive messages competing for a conspicuously finite attention, these sketches show a landscape of ignorable surfaces capitalising on their context, timing and your history to quietly play and present in the corners of our lives.

Incidental Media is brought to you by Dentsu London and BERG. Beeker has written about the films here.

Thank you to Beeker Northam (Dentsu London), and Timo Arnall, Campbell Orme, Matt Brown, and Matt Jones!

“Making Future Magic” is the goal of Dentsu London, the creative communications agency. We made this film with them to explore this statement.

(Click through to Vimeo to watch in HD!)

We’re working with Beeker Northam at Dentsu, using their strategy to explore how the media landscape is changing. From Beeker’s correspondence with us during development:

“…what might a magical version of the future of media look like?”

and

…we [Dentsu] are interested in the future, but not so much in science fiction – more in possible or invisible magic

We have chosen to interpret that brief by exploring how surfaces and screens look and work in the world. We’re finding playful uses for the increasingly ubiquitous ‘glowing rectangles’ that inhabit the world.

This film is a literal, aesthetic interpretation of those ideas. We like typography in the world, we like inventing new techniques for making media, we want to explore characters and movement, we like light painting, we like photography and cinematography as methods to explore and represent the physical world of stuff.

We made this film with the brilliant Timo Arnall (who we’ve worked with extensively on the Touch project) and videographer extraordinaire Campbell Orme. Our very own Matt Brown composed the music.

Light painting meets stop-motion

We developed a specific photographic technique for this film. Through long exposures we record an iPad moving through space to make three-dimensional forms in light.

First we create software models of three-dimensional typography, objects and animations. We render cross sections of these models, like a virtual CAT scan, making a series of outlines of slices of each form. We play these back on the surface of the iPad as movies, and drag the iPad through the air to extrude shapes captured in long exposure photographs. Each 3D form is itself a single frame of a 3D animation, so each long exposure still is only a single image in a composite stop frame animation.

Each frame is a long exposure photograph of 3-6 seconds. 5,500 photographs were taken. Only half of these were used for the animations seen in the final edit of the film.

There are lots of photographic experiments and stills in the Flickr stream.

The light appears to boil since there are small deviations in the path of the iPad between shots. In some shots the light shapes appear suspended in a kind of aerogel. This is produced by the black areas of the iPad screen which aren’t entirely dark, and affected by the balance between exposure, the speed of the movies and screen angle.

We’ve compiled the best stills from the film into a print-on-demand Making Future Magic book which you can buy for £32.95/$59.20. (Or get the softcover for £24.95/$44.20.)

There’s lots of text on the blog at the moment. Time to add a little bit of colour with some links that have been floating around the studio.

Following last week’s link to Reknit, friend-of-BERG Rod McLaren gave me a link to Goldenhook. It’s a French business selling knitted goods with a twist:

Golden Hook is an innovative fashion brand which allows you to create made-to-order beanies by choosing your beanie style, material, and color. You also choose the authentic grandmother who will knit your beanie from our gallery of grandma photos.

Authenticity being sold through choice – and a personal connection to whoever’s knitting your new hat. Fun, although if Goldenhook is anything to go by, Granufacture isn’t very cheap yet.

Meanwhile, from Kottke – and a great many other sites – comes news of increased sales of miniature sausages in Korea:

Sales of CJ Corporation’s snack sausages are on the increase in South Korea because of the cold weather; they are useful as a meat stylus for those who don’t want to take off their gloves to use their iPhones.

Meat styluses. I really have no sense for the Korean market: might this be a hoax? No idea; it doesn’t seem so, given the coverage. And it definitely works, as this video of someone playing Taiko Drum Master with a pair of sausages demonstrates. That’s one way to keep your fingers warm.

Matt Jones sent this to the studio mailing list last week, and it was destined for the blog from the get-go: a beautiful collaboration between Lego and MUJI Japan. It’s so simple: a model made out of a combination of Lego pieces and what look like origami squares, with pre-punched holes for joining the paper to the bricks. And: what a cheery crocodile.

Finally, via our frequent collaborator Timo Arnall comes a striking depiction of one potential Augmented future, courtesy of Keiichi Matsuda. Matsuda writes:

The latter half of the 20th century saw the built environment merged with media space, and architecture taking on new roles related to branding, image and consumerism. Augmented reality may recontextualise the functions of consumerism and architecture, and change in the way in which we operate within it.

In his video, the home becomes another space for being advertised to in – with the catch that the more advertising you choose to be subjected to, the more revenue you’ll generate. The glitches in the AR system, and the horrible Girl From Ipanema cover are the icing on an entertaining (if somewhat bleak) cake.

Last week’s links slipped over into this week’s. If you read Matt’s weeknotes for last week – week 242 – you can probably understand why. But! Better to be late than to forget them.

I loved Reknit: a site for turning unwanted woolen goods into new products. You send off an unwanted pullover, it gets unravelled, and sent back to you as something new. This month, it’s a scarf; next month, you’ll get something else based on a vote (in the running: a beanie, iPod case, cut-off gloves, or socks). If you don’t have a sweater to recycle, the site even offers to find you your nearest Goodwill store, where you can no doubt buy many new, old, scarves. This isn’t a large-scale industry, though; it’s the creator’s mother. And that’s the bit I really love, encapsulated in its tag line: this month, my mom will turn your old x into a new y. It won’t ever be a big operation, but it opens up her knitting to a slightly wider audience than the rest of her family. Lovely. Matt Brown coined something similar in the studio last week: small scale Gran-ufacture.

On a slightly more sombre note, Matt Jones sent this image to the studio mailing list, from a 2007 New Scientist article on the depletion of Earth’s natural resources. The stat that really caught our eye was the dwindling resources of Indium. Indium’s a critical component of LCD displays, and whilst, obviously, other screen technologies are available – and will continue to be developed – Matt noted that it’s a reminder that non-screen-based interactions (like those in Availabot or made possible through RFID) have an environmental value as well as a technological one.

The picture on the right is an advert I saw on the tube last week, for the University of East London. We write a lot about Augmented Reality on the blog, but I always assume we’re coming from a technologically informed/privileged position. So when I saw this on the tube, I did a double take; this is a faux-AR image of the Thames, with UEL facilities and landmarks picked out not only by map-pins, but also glassy iPhone-style bubbles. Perhaps the point of reference is meant to be mapping, but the combination of the popups with the photograph feels exactly like AR to me; the idea that AR was already a usable metaphor for advertising was very surprising. It’s also a reminder of the ability advertising has to introduce new concepts, rather than just illustrate old ones.

It’s not all serious links about Augmented Reality, or the Earth’s dwindling resources, on the studio mailing list, though; there’s also a decent amount of “here, look at this!“. It’s alright to like pretty things. I found this video from friend-of-BERG Alex Jarvis, and just had to share it. Ingenious animation, beautiful sketching; seems like the right thing to end these links with.

More on the theme of ‘Gibsonian-eversion‘ or pushing augmented reality into the physical world, this time a video speculation by Choy Ka Fai of RCA Design Interactions.

This work was part of the “Future of Etiquette” project I worked on with the year one group on the course, to a brief in part from T-Mobile’s design research team in Berlin.

Ka Fai constructed a simple apparatus using cheap laser-pointers that indicated the field of view of a digital camera to those in the surroundings.

In early design probes on the streets of Berlin, one of the most fascinating ‘protocols’ observed by passers-by was how almost universally the use of a camera created a spatial barrier between the photographer and the subject, that, at least for a short period of time, was seen as impassable.

Fascinating, in that most cameras are now digital, and there is no film to be wasted by the incursion of passer-bys in shot as perhaps there was only ten years ago. The etiquette is a hang-over from a previous technology perhaps…

The video below illustrates a period of time in Trafalgar Square, London – imagining that that invisible barrier is made visible – making clear the overlaps, frictions and interactions the cameras could create in such a highly-photographed piece of the city.

CROSSING BORDERS from KA5@RCA on Vimeo.

Matt Webb and myself were down in Bristol on Friday, for the last of our initial workshops kicking off a project named Trumbull.

During the afternoon, we had a bit of a treat, as we shared the workshop with a couple of the guys from ANTIVJ, who self-describe as a ‘video label’.

The work they showed was literally fantastic.

They map the surfaces of buildings precisely, and craft their projections accordingly, in order to then create amazing performances with light and sound – hinting perhaps at an augmented reality everted from the screen and onto the city as 21stC trompe l’oeil*.

Entrancing stuff, but my mind was really blown about 3mins 50seconds in…

AntiVJ & Crea Composite: Nuit Blanche Bruxelles from AntiVJ on Vimeo.

* c.f. our colleague Timo Arnall’s speculations on “everted A.R.”

Will Wiles of Icon magazine interviewed me briefly for his article in the current (October) issue, on developments in augmented reality and impacts of architecture and urban design.

It’s a good overview aimed at a non-technical audience, and it’s great to see the discourse about AR in a design magazine rather than the usual more tech-oriented venues.

Also, gratifying to be quoted alongside our friends Eric Rodenbeck from Stamen and Usman Haque from Pachube/Haque Design+Research.