There are a lot of mind control projects around at the moment. This concept being my favourite:
Which was recently made a reality by our friends at Neurosky.
In the last year we have done a fair bit of research into this area ourselves. We've not been able to talk about it until now as we were working towards a specific client project, but as that's not happening now, we can share some of what we found.
The headsets work by detecting the electrical activity of the brain, the millions of tiny electrical signals generated by your neurons as they fire. The brain reading itself is called an Electroencephalogram or EEG. We chose to partner with Neurosky to provide the headsets and some of the software for the project. The headsets we chose are able to detect all kinds of brainwaves, but the software provided by Neurosky single out brainwaves associated with concentration and meditation (relaxation). So we used these.
The idea was to make a mind control film. Most other applications and films we had seen used EEG as a kind of switch. i.e. "Concentrate now to switch to state B, then relax again". However, we thought that it would be better to permanently connect the state of the film to the user's mental state. We wanted the user to feel connected to the fim and eventually forget about the interface itself.
We had two parallel films; one with a conventional narrative and a more surreal second film. If you concentrated you could literally see through the main film to the second film, if you lost concentration the second film would disappear. We likened this to the way that concentrating on something in your mind, say mental arithmetic, enables you to grab it and see it clearly, if you lose concentration you lose your grasp on it and can feel it slipping away. We wanted the second film to feel like that.
Another important element was that we wanted this to be closer to a cinema experience than a game experience. We didn't want dials and graphs and things designed to make it look techie. We wanted this to be pure art. Anything that would have looked out of place in the cinema was not allowed.
Firstly though, as this was uncharted waters, we started with a load of research. What does it feel like? How sensitive is it? How do we calibrate it? What's the minimum interface we need? There is no agreed wisdom on any pf these things.
We made a test suite with some configurable settings. It was an Air App connected to Neurosky's socket server.
This is a fairly uneventful video of mind control Rasta Mouse.
First we wanted to gauge the level of concentration required by different people to show the second film. From this we could gauge how much variance there is which gave us clues as to how to calibrate the system on the fly. For this we measured people's ability to bring in a single second film at a variety of difficulty levels.
Second, we tested how people reacted to having to show multiple films over time. Was it possible? Did people fatigue? Was it unpleasant? etc.
We started with a brief called "Anti-Shyness Clothing". Could we make clothes that help you to socialise? We decided to answer this by making a t-shirt that could connect to other similar t-shirts and visually change based on data about the other person.
First we worked on the display. There are many displays used in smart clothing, mostly based around LEDs, electroluminescent thread or flexible screens. These can look amazing and are good for quick changes, complex information and grabbing attention, but we thought we'd go for a calmer approach. Nanda is influenced by Mark Weiser who promoted the idea of "calm technology". We wanted our display to be less of a replacement for a screen and more of an extension of the body, closer to a human blushing or a cat's fur rising. A slow, calm, natural feeling display. So a requirement was, no LEDs.
Nanda came up with the ingenius idea of using dye that goes transparent when warmed and conductive thread that warms when you put a current through it. Brilliantly, if you mix the dye with another normal dye to create a third colour, the heat sensitive dye will still vanish from the mix when heated. So if you mix heat sensitive blue, to normal yellow to create green, it will turn yellow when you heat it as the blue will go transparent. This way you can produce any colour.
So with our pattern of conductive thread sewn into the shirt, and painted dye pattern over the top, we had a calm natural feeling display.
Second, we wanted to enable the shirts to be able to identify each other. We wanted any shirt to explicitly recognise a specific other shirt. We opted for an RFID approach to solve this. We put and RFID transmitter and receiver in the sleeve of the shirts. So when you shake hands with someone the chips are at close enough range to connect. We actually had a pretty tricky problem to solve that had actually foxed everyone we spoke to online. In the end we solved it with a switching technique we invented, but we have to keep it quiet until a technical paper is published. Anyway, with this working, any shirt could recognise any other shirt.
The whole system was controlled by the Arduio Lillypad and a mini regular Arduino. Ok it's not pretty, its a prototype.
Thirdly, we needed data to compare between users. The eventual aim was to connect to the cloud and use live social network data. Actually this is a pretty trivial step. We made the prototype connect via Bluetooth to an Android device and exchange data. From there it's quite simple to connect to the web.
As it was though, we uploaded all the data required onto the Arduino itself. This was workable with a small group to prove the concept, but future versions will have to use live data.
So we have created t-shirts that can identify each other, change their display and connect to an internet connected mobile device.
Finally, Nanda had to test the original hypothesis that these shirts would help shy people socialise in an experiment at Queen Mary's. We had 4 volunteers and made 4 identical shirts. We represented each person's personality as a list of numbers that represented their preferences on a range of subjects, we then rated people's compatibility comparing these numbers, rather like a dating agency. The display then showed the level of compatibility when they shook hands. Green for good, red for bad. Although either way it was a talking point and a social lubricant, so had the desired effect.
Lately we've been thinking about virtual avatars and AI. What happens to your online self when you die? Could it have learnt enough about you during your lifetime to live on as you? Could an avatar be a big help in consuming the web? If an AI bot that had your taste characteristics was set free on the net, could it save you time by collating stuf for you, or doing chores etc?
We thought we were being very original here, but it seems we're not the first. this is an Australian life insurance ad.
This guy is called Bruce Duncan and he runs a thing called the Terasem Movement Foundation LifeNaut Project. They created the concept of "mindfiles", files that contain data from your life gained from data. They are mentioned in the above video, but had no idea it existed.
The life insurance ad talks about Social Network data. LifeNaut uses digital data, but not social network data yet as far as I know. So it seems to still be a young enough field to make an impact, it seems likely that social network data would be a very rich seem of information if you wanted to re-create someones personality. Anyway I spoke to Bruce and he's a really nice guy, maybe we'll collaborate.
He bought along a robot likeness of a colleague called Bina48. Her personality was based on the Mindfile of the real Bina, whilst the conversation interface was made using separate language parser. About half the session was a question and answer session with Bina48 herself, which had varying success. It was actually pretty funny and occasionally profound.
Something interesting highlighted by an audience member was that everyone seemed to be asking really profound questions about the nature of existence, although I suppose that's to be expected from your first conversation with a robot.
Being in a room at a big conference talking to a robot head felt really exciting, like we were witnessing the first alien contact or something.
He invented OCR, text-to-speech synthesis, speech recognition and loads of other stuff. He popularised the notion of the "singularity", the point in time when computers become more intelligent than humans and all bets are off. He's probably the world's best know futurist.
I like that he takes a systematic approach to futurism. He doesn't seem to act on intuition like a lot of futurists do. His predictions often seem over optimistic, but end up uncannily accurate as he looks at hard numbers.
Over the years a lot of philosophers and psychologists have said we'll never get proper "human style" artificial intelligence because there's something "other" about it, something that is too complex for any computer to simulate. I remember when I was studying cognitive science in the early nineties that it was still thought by many that a chess computer could never beat the best in the world because there was some ill-defined humanness required. Kurzweil predicted that this would in fact happen in 1996, and it did. He knew it would because of the relentlessness of Moore's law.
Another great example was the sampling of the human genome. Having spent something like 10 years to do 10% of it, researchers thought it would take decades to finish. Kurzweil, taking Moore's law into account, said it would take another year or so, and it did.
So a recurring theme was this idea that people are over pessimistic about AI and technology in general because they fail to take into account the sheer power of exponential growth. Worries about being able to create the peculiarities of intelligence get blasted away by the massive increase in things like processor speed, bandwidth, materials etc.
The recent success of IBM's Jeopardy playing supercomputer, Watson shows just this.
It beat the World's two best human Jeopardy players. According to Kurzweil this isn't due to particularly new AI techniques, but more due to raw processing power. It gained it's knowledge by reading the internet, including the whole of wikipedia. It then selected articles via a fairly simple probabilistic algorithm. It worked because it had the speed to process the data in real time. Again it's down to Moore's law. The interesting thing that Kurzweil highlight though was that this is pretty much how we work in our own brains anyway. We are fundamentally probabilistic machines with a lot of processing power.
If you plot our intelligence on a graph against machines, they pass us in 2045. So that's it then, after that point we have no way of knowing what will happen next. Machines might not care to tell us what they are up to, and we may well not understand it if they did.
On the bright side, there will be no reason why we won't be augmenting our own brains with billions of extra neurons, so we may end up having a more symbiotic relationship with machines than some people fear. And at least we'll be able to back it up.
This is a subject close to my heart. We've done a fair bit of work with smart clothes and calm displays. It was introduced by Ping Fu, she's CEO at a company called Geomagic. They make high end 3D sanning software that's used everywhere for the space programme to archeology. This was about clothes though, and she had assembled a few leading people from the smart clothes community.
First Ping herself talked about how 3D scanning and printing is being used in creative and beautiful ways.
She gave the example of a beautifully crafted laser cut prosthetic leg. As she said, they usually look more like aircraft landing gear than anything you'd want to adorn yourself with. Why shouldn't they be beautiful and custom made and custom fitted. With todays 3D scanning and printing technology its possible.
Here's a sporty little number:
Fashion designer Iris Van Herpen uses 3D printing technology and other advanced processes to make clothes. Here's one where she wanted the model to look like she'd been splashed with water:
Next up, Sheridan Martin. She does research for XBox. She made this, "You are what you tweet" dress. I think you can tweet from the keyboard on it, and I think you can tweet it. It was projecting words onto the inside of the dress at least.
Alison Lewis from Agent of Presence was cool. She was definitely on the pure aesthetic side of things. This technology doesn't have to be useful, it can just be about beauty or emotion. I like that approach. She showed a pair of 3D printed Jimmy Choo shoes, which dispelled any ideas about technology and fashion necessarily being dorky.
My favorite thing of her's though was a dress she called "Presence of Heart" that lit up in unison with the heart beat of the wearer. Dead simple and quite sweet I thought. the method used for lighting this one up was a very powerful LED and optic fibres to spread the light through the fabric.
Last was Janet Hanson from Enlighted Design. She pretty much makes all the light up clothing you ever see in pop videos and gigs.
These are all great, but they are designed for one-off or limited use. She pre-empted people asking about whether consumer versions were possible, and she yes, but there are things to consider. The main problem with the Electro-Luminescent fibre is that it's quite fragile and they don't like being bent around much. So in her designs she tries to avoid bendy areas like knees and elbows. She reckons that consumer versions will be possible, but we have to really tailor them the the material. In fact she recommended wearable accessories like necklaces or bracelets etc. We'll see.
