AsianScientist (July 13, 2017) – Scientists in Korea have figured out why photos of documents taken using smartphone cameras are often wrongly oriented. Their findings have been published in the International Journal of Human-Computer Studies.
Photos of documents taken with a smartphone camera are wrongly rotated at 90 degrees. This is because software engineers designed the rotation tracking software in conventional smartphones with the following assumption: people hold their phones vertically either in portrait or landscape orientations. The camera app of smartphones typically displays the camera orientation using a camera-shaped icon, but users either overlook this feature or fail to notice its state when they take document photos.
Orientation tracking in smartphones is accomplished by measuring the direction of gravity using an acceleration sensor. When taking photos of documents, smartphones are generally held parallel to the document, and this confuses the acceleration sensor.
“Your phone fails to track the orientation if you make any rotation changes at that moment,” said Professor Lee Uichin of the Korea Advanced Institute of Science and Technology.
Lee’s team conducted a controlled experiment to discover how often orientation errors occurred in document-capturing tasks. Their results showed that landscape document photos had error rates of 93 percent. To correct this problem, the team developed a technique that corrects a phone’s orientation by tracking the rotation sensor in addition to the acceleration sensor.
The researchers also discovered that when taking a document photo, the user tends to tilt the phone, just slightly, towards the user (called a ‘micro-tilt phenomenon’). While the tilting degree is very small—almost indistinguishable to the naked eye—these distinct behavioral cues are enough to train machine-learning models that can easily learn the patterns of gravity distributions across the phone.
The team’s experimental results showed that their algorithms can accurately track phone orientation in document-capturing tasks at 93 percent accuracy. The correction software works only when the intent of photographing documents is detected and can seamlessly work with existing orientation tracking methods in Google Android and Apple iPhones without conflict. A US patent application for their method was granted in March 2017.
The research team even suggested a novel user interface for photographing documents. Just like with photocopiers, the capture interface overlays a document shape onto a viewfinder so that the user can easily double-check possible orientation errors.
“Photographing documents is part of our daily activities, but orientation errors are so prevalent that many users have difficulties in viewing their documents on their phones without even knowing why such errors happen,” said Lee.
“We can easily detect users’ intentions to photograph a document and automatically correct orientation changes. Our techniques not only eliminate any inconvenience with orientation errors, but also enable a range of novel applications specifically designed for document capturing.”
The article can be found at: Oh et al. (2017) Understanding Mobile Document Capture and Correcting Orientation Errors.
———
Source: Korea Advanced Institute of Science and Technology.
Disclaimer: This article does not necessarily reflect the views of AsianScientist or its staff.
