Facebook Patents Make Facial, Logo And Word Recognition A Reality
Earlier this year, during a three week period between April 25th and May 14th, a number of interesting Facebook patents were published. Then Facebook employees Andrew “Boz” Bosworth, David Garcia, and Soleio Cuervo applied for the patents in 2011, which give Facebook the exclusive right to develop software that records everything seen through a phone’s camera while the phone’s user takes photos. At the time, it was not immediately apparent how the software could be utilised; now, with the introduction of Instagram Video, the basic purpose of the software has become clear.
Using the software, Facebook will be able to continuously capture video through the Instagram app and any phone’s camera feature, even when the user isn’t clicking the record or capture image button. All the media captured is stored on the mobile device. If you’re still not sure what I’m talking about, an extract from one of the patent applications might help:
However, the software could have even more applications on Instagram. Not only does it take into account the quality of each image, it can also recognise faces, words, buildings and brands within the images. Based on this information, Instagram can then suggest tags for each photograph so, for example, if a user took a photo of two of their friends, Instagram would recognise the faces of these friends in its database and suggest tags accordingly.
Cleverer still, Instagram can use GPS information and recognise dialogue snippets while videos are recorded to work out what the videos are about, where they are and what they feature. For instance, if a user were to comment on a sunset as they filmed it, saying “that’s beautiful”, Instagram would know that whatever was being captured at that moment was “beautiful”. According to the Patent Automatic Photo Capture Based on Social Components and Identity Recognition, Instagram could even register someone saying “say cheese”.
The same patent request goes into detail about how the GPS and the recognition software could be used in tandem to tag images and videos with a location:
An image in one of the other patent applications, Preferred images from captured video sequence, (the third and final patent can be found here) hints at an application the automatic image capture software and selection could have on Facebook. As seen above, Charlie has been tagged in Angus’ video, from which four screenshots have been taken. Presumably, users will be informed if a friend posts a video which features them on Facebook. In the post, the four best frames in which they appear will be displayed beneath the announcement, helping them work out what the video’s about and if they want it appearing in their own news feed.
Facebook and Instagram seem to be going to a lot of trouble just to provide users with automated tagging and cover image selection, as well as slideshow-like previews of videos, unless you take into consideration the ramifications the software could have for Facebook marketers.
Before I go any further, I would like to point out that, as far as I can make out, the images automatically photographed will only be stored on a user’s phone, instead of being directly shared on Instagram and Facebook. Also, Facebook and Instagram will presumably only suggest tags, users will still have the final say over what tags appear beside images.
After the outcry last year, when Instagram revealed it had the right to sell a user’s photos without the users permission, Facebook will be very careful not to invade its users’ privacy where photos are concerned. That said, and bearing in mind that the patents make no mention of this application, Facebook would be stupid not to use all the data gathered by the software to help brands better target users.
Shops could target users who frequently take photos near, or even featuring, their shop fronts; brands could target users who take photos of their products, or even the products of their rivals; restaurants and bars could target users who are taking photos with their friends nearby; companies could even target users who say their name out loud, which is a scary thought!
Images and videos are by far the most shared forms of media on Facebook, but users don’t always tag them with information useful to advertisers. The new, recently patented software, will help brands overcome this obstacle and target customers accordingly.
In other Facebook news, Graph Search went public today, while Instagram updated its iOS app, adding Landscape Mode.
What do you think about the possible ramifications of these patents?
Contact us on Facebook, on Twitter, or leave your comments below.
Will Sigsworth
Follow us @SocialMediaF & @WillAtSMF
Or go to our Facebook page.
Using the software, Facebook will be able to continuously capture video through the Instagram app and any phone’s camera feature, even when the user isn’t clicking the record or capture image button. All the media captured is stored on the mobile device. If you’re still not sure what I’m talking about, an extract from one of the patent applications might help:
Although the camera function operates in a photo-capturing mode, the camera function may continuously capture video when the camera function is activated. For example, a viewfinder of a camera function of a mobile device can be a real-time video feed of the camera function.The software can then sift through the images and videos to work out which are the best, based on quality and subject matter. So, what’s the obvious application this software has on Instagram Video? Put simply, it will help users choose a cover frame, potentially even automating the process.
However, the software could have even more applications on Instagram. Not only does it take into account the quality of each image, it can also recognise faces, words, buildings and brands within the images. Based on this information, Instagram can then suggest tags for each photograph so, for example, if a user took a photo of two of their friends, Instagram would recognise the faces of these friends in its database and suggest tags accordingly.
Cleverer still, Instagram can use GPS information and recognise dialogue snippets while videos are recorded to work out what the videos are about, where they are and what they feature. For instance, if a user were to comment on a sunset as they filmed it, saying “that’s beautiful”, Instagram would know that whatever was being captured at that moment was “beautiful”. According to the Patent Automatic Photo Capture Based on Social Components and Identity Recognition, Instagram could even register someone saying “say cheese”.
The same patent request goes into detail about how the GPS and the recognition software could be used in tandem to tag images and videos with a location:
The image capturing process may determine […] a list of social network objects that are at or near a current location of the first user, and identify one or more social networking objects in one or more frames of the sequence of video frames by matching (e.g., by using an object recognition algorithm) images of the list of social networking objects (e.g., photos associated with each of the list of social network objects) to content in the one or more frames. For example, an object recognition algorithm may use optical character recognition techniques to identify one or more characters (e.g., “HOLLYWOOD”, “San Francisco Giants”) in one or more frames and match against image data (or identity data such as names, logos) of the list of social network objects nearby. […] For example, an object recognition algorithm can determine a match between an unknown image file (e.g., one of the frames) and an image file of a known identity (e.g., an image of Eiffel Tower) by comparing a first set of features of the unknown image file and a second set of features of the image file of a known identity.
An image in one of the other patent applications, Preferred images from captured video sequence, (the third and final patent can be found here) hints at an application the automatic image capture software and selection could have on Facebook. As seen above, Charlie has been tagged in Angus’ video, from which four screenshots have been taken. Presumably, users will be informed if a friend posts a video which features them on Facebook. In the post, the four best frames in which they appear will be displayed beneath the announcement, helping them work out what the video’s about and if they want it appearing in their own news feed.
Facebook and Instagram seem to be going to a lot of trouble just to provide users with automated tagging and cover image selection, as well as slideshow-like previews of videos, unless you take into consideration the ramifications the software could have for Facebook marketers.
Before I go any further, I would like to point out that, as far as I can make out, the images automatically photographed will only be stored on a user’s phone, instead of being directly shared on Instagram and Facebook. Also, Facebook and Instagram will presumably only suggest tags, users will still have the final say over what tags appear beside images.
After the outcry last year, when Instagram revealed it had the right to sell a user’s photos without the users permission, Facebook will be very careful not to invade its users’ privacy where photos are concerned. That said, and bearing in mind that the patents make no mention of this application, Facebook would be stupid not to use all the data gathered by the software to help brands better target users.
Shops could target users who frequently take photos near, or even featuring, their shop fronts; brands could target users who take photos of their products, or even the products of their rivals; restaurants and bars could target users who are taking photos with their friends nearby; companies could even target users who say their name out loud, which is a scary thought!
Images and videos are by far the most shared forms of media on Facebook, but users don’t always tag them with information useful to advertisers. The new, recently patented software, will help brands overcome this obstacle and target customers accordingly.
In other Facebook news, Graph Search went public today, while Instagram updated its iOS app, adding Landscape Mode.
What do you think about the possible ramifications of these patents?
Contact us on Facebook, on Twitter, or leave your comments below.
Will Sigsworth
Follow us @SocialMediaF & @WillAtSMF
Or go to our Facebook page.
Facebook Patents Make Facial, Logo And Word Recognition A Reality
![]()
Reviewed by Anonymous
on
Monday, July 08, 2013
Rating:
Anonymous
Keeping you up to date on social media, digital marketing, apps, news & reviews. Social media, and digital marketing as a whole, are rapidly and constantly changing, adapting to new developments at an ever increasing rate. Just keeping up to date can be a nightmare. Our team of writers know their field, and keep an ear to the ground. We aim to keep you informed of all recent developments in the online world, and teach you a little something along the way. So whether you just want to stay connected, or gain some knowledge for yourself, the Social Songbird will keep on singing! More articles are posted every day so keep checking back! Join the discussion on Twitter - #SocialSongbird Or contact us directly - @SocialSongbird_
Entrepreneur, international speaker on Social Media Marketing. First one in the UK to write and speak in conferences about Twitter as a marketing tool. Consultant to Corporate Companies, Government Organizations, Marketing Managers and Business Owners.
Aspiring novelist with a passion for fantasy and crime thrillers. He hopes to one day drop that 'aspiring' prefix. He started as a writer and soon after he was made Executive Editor and Manager of the team at Social Songbird. A position he held for 5 years.
Adrienne is a passionate writer with experience in editing, content creation, and social media marketing. She’s a lover of animals, helping others improve their writing, and 2000s pop culture.
Musician, audio technician, professional tutor and a Cambridge university English student. Interested in writing, politics and obsessed with reading.
Recently graduated with a BA in English Literature from the University of Exeter, and he is about to study an MA in Journalism at the University of Sheffield. He is an aspiring journalist and novelist; in his free time he enjoys playing chess, listening to music and taking long walks through nature.
Lucy is an undergraduate BSc Politics and International Relations student at the London School of Economics and Political Science.
Anna Coopey is a 4th year UG student in Classics at the University of St Andrews in Scotland. She is a keen writer and researcher on a number of topics, varying from Modern Greek literature to revolutionary theory.
Ellie is a recent BA English Literature graduate and aspiring publisher with a passion for writing. Her favourite pastimes include reading and shopping.