Knocking Down The Barriers To Object Identification

Scientists in South Korea have developed a technique to identify objects by using the microphone, accelerometer and gyroscope of smartphones.

AsianScientist (Nov. 21, 2019) – By knocking a smartphone against objects, researchers in South Korea can identify objects even in dark environments. They presented their findings at the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing.

Object identification by machines typically involves the use of video cameras and computer vision to analyze the visual data. However, the use of cameras by default means that lighting plays a part in accurate identification.

In the present study, researchers led by Professor Lee Sung-Ju at the Korea Advanced Institute of Science and Technology have developed technology to identify objects by sound and impact. They called their technology ‘Knocker.’

What separates Knocker from existing technology is the sensor fusion of sound and motion. Knocker utilizes the smartphone’s built-in sensors such as a microphone, an accelerometer and a gyroscope to capture a unique set of responses generated when a smartphone is knocked against an object. Machine learning is used to analyze these responses to classify and identify objects.

The research team confirmed the accuracy of Knocker technology by using it to distinguish 23 everyday objects such as books, laptop computers, water bottles and bicycles. In noisy environments such as a busy café or on the side of a road, Knocker achieved 83 percent identification accuracy. In a quiet indoor environment, the accuracy rose to 98 percent.

The team believes Knocker will open a new paradigm of object interaction. For instance, by knocking on an empty water bottle, a smartphone can automatically order new water bottles from a merchant app. When integrated with IoT devices, knocking on a bed’s headboard before going to sleep could turn off the lights and set an alarm.

“This new technology does not require any specialized sensor or hardware. It simply uses the built-in sensors on smartphones and takes advantage of the power of machine learning. It’s a software solution that everyday smartphone users could immediately benefit from,” said Lee. “This technology enables users to conveniently interact with their favorite objects.”

The article can be found at: Gong et al. (2019) Knocker: Vibroacoustic-based Object Recognition with Smartphones.


Source: Korea Advanced Institute of Science and Technology.
Disclaimer: This article does not necessarily reflect the views of AsianScientist or its staff.

Asian Scientist Magazine is an award-winning science and technology magazine that highlights R&D news stories from Asia to a global audience. The magazine is published by Singapore-headquartered Wildtype Media Group.

Related Stories from Asian Scientist