A Romantic Revolution In Technology

We spoke to maverick inventor Jun Rekimoto of the Sony Computer Science Laboratories on the latest technologies in human-computer interaction and artificial intelligence.

AsianScientist (Mar. 22, 2018) – We’ve lost count of the number of business ideas that involve the ‘Uberization’ of something. But nothing prepared us for the moment we heard about the Human Uber, proudly brought to you by maverick inventor Professor Jun Rekimoto.

Rekimoto is deputy director of Sony Computer Science Laboratories (CSL) in Tokyo, where he leads an intrepid bunch of computer scientists and engineers working on eliminating the seams between humans and technology.

This is how the Human Uber works: imagine you have to attend a meeting in another city or country. Instead of flying there, you pay a surrogate to strap on an iPad-like device displaying a real-time video of your face, and this person attends the meeting on your behalf. No jet-lag, minimal carbon footprint, and the best part of all? You don’t even need to change out of your pyjamas.

“Sony CSL is about seeking new and future possibilities,” Rekimoto shares, alluding to his Human Uber concept. “We can, of course, collaborate on current technological products, but we are not forced to just do product development. Our job is to find future directions [for Sony].”

Asian Scientist Magazine took the opportunity to interview Rekimoto at the the EmTech Asia 2018 conference in Singapore.


A graduate of the Tokyo Institute of Technology, Rekimoto worked at NEC Corporation and the University of Alberta in the 1980s before joining Sony CSL at the behest of founder, president and CEO, Professor Mario Tokoro, in 1993.

More recently, in 2007, Rekimoto became a professor at Japan’s top university, the University of Tokyo, where he directs the human augmentation lab at the Interfaculty Initiative in Information Studies. He calls his lab the Laboratoire Révolutionnaire et Romantique in French, which is loosely translated in English as the laboratory of revolution and romance.

If anyone has romantic notions on the revolutionary power of technology, it would surely be Rekimoto—he is a strong proponent for using technology to augment the capacity of human beings, be it cognitive or physical.

“My laboratory at the University of Tokyo is doing research on human-computer interaction, which is the technology that supports or augments humans. We use a lot of augmented reality (AR) and virtual reality (VR), like head-mounted displays, telepresence robots and drones,” Rekimoto says.

Take for example his pet project, JackIn, whose moniker is inspired by William Gibson’s science fiction novel, Neuromancer. In the story, the hacker protagonist has to insert himself or ‘jack-in’ into cyberspace. Rekimoto’s JackIn Head device allows a remote user to view video in real-time, taken by an omnidirectional camera mounted on another person’s head.

Bursting out of the telly

If a 60-inch television screen isn’t big enough for your living room, Rekimoto has the solution for you. He and his 20-member-strong research lab of students and post-doctoral fellows are using AI to enhance our movie-watching and gaming experience.

With the help of artificial intelligence (AI), they’ve developed a projector-like device that can beam images beyond the four corners of the TV set, extrapolating beyond the boundaries of the footage that is being shown.

“When you watch a TV screen, you want it to be more immersive, to be surrounded by the environment. But the content is inside of the TV screen, and so we created a special neural network that generates an image around the TV screen,” Rekimoto says.

“All of it is computer generated, and the part created by AI is inferred. AI can provide a lot of opportunities; expanding the [boundaries of what we see on the TV screen] is just one possibility.”

The future of human augmentation

Rekimoto now shuttles between Sony CSL, where he says he spends 30 percent of his time, and the University of Tokyo, where he spends the rest of his time carrying out research and teaching.

He’s quick to highlight some star researchers at Sony CSL—Professor Ken Endo, for instance, developed a prosthetic leg that was used by Japanese Paralympics team at the Rio Olympics. More quirkily, a team at Sony CSL is working on an agriculture project using sensor technology to cultivate vegetables better.

Setting his sights beyond entertainment, Rekimoto is exploring the use of AI on enhancing human learning by embedding chips inside of our brains.

“There are a lot of opportunities to enhance our learning abilities with AI. A lot of effort is required to learn new things, and your learning curve can be very steep without AI inside of you,” he says.

Rekimoto is also working on printing flexible and biocompatible electronic circuits onto a person’s skin, where they will act as sensors or communication devices.

With all these gadgets and gizmos to tinker with, it is a surprise that Rekimoto hasn’t already deployed a Human Uber to attend the conference on his behalf, and that I am talking to the living and breathing version of the man.


Copyright: Asian Scientist Magazine; Photo: Jeremy Chan
Disclaimer: This article does not necessarily reflect the views of AsianScientist or its staff.

Juliana is the founder and CEO of Wildtype Media Group, Asia's leading STEM-focused media company, spanning digital, print, custom publishing and events. Brands under Wildtype Group include Asian Scientist Magazine and Supercomputing Asia, award-winning titles available in print and online. Juliana regularly moderates panel discussions and gives talks on science communication.

Related Stories from Asian Scientist