AEM and Google Cloud Vision – Impressions to Implementation

Inspired Digital Experiences for Manufacturing & Automotive

Whether you’re just beginning your digital transformation journey or are well on your way, we invite you to explore our partnership with Adobe and our diverse capabilities in manufacturing and automotive.

Download Lookbook

Chad Johnson’s recent observations on Google’s Cloud Vision API led us to start thinking of ways the image recognition software could be utilized within Adobe Experience Manager (AEM).  Automatic asset tagging immediately sprang to mind.  This solution reduces the burden on the end user in manually identifying and tagging the assets with relevant metadata.  Google’s Cloud Vision API is pretty amazing.  It may not completely remove the end user’s involvement in the tagging process, but certainly eases the task.  The end result is rich taxonomy structures for improved search capabilities on the DAM Assets.
AEM Implementation
Google provides an easy to use REST API for label identification (detecting individual objects on an image), landmarks detection, sentiment analysis, logo detection and amazing optical character recognition (OCR).  These services can be used within any application with access to images.


Into the details …
Google requires registration with the Vision Services and establishing an authentication scheme for authentication and/or authorization.  The simplest authentication scheme is an “API Key”.   This scheme provides a JSON file which can be referenced in the code for authentication.
We have created an AEM service to invoke Google’s API.  This service can be referenced by any component, service, or workflow process.  We extended the OOTB workflow “Update DAM Asset” to include the additional process of calling this service for every asset added to the DAM.
Our process determines the type of annotation coming back from Google (label, landmark, sentiment, or logo) and creates a tag in the corresponding taxonomy (see screenshots above).  The process also includes an OCR Vision call, which when made extracts the text identified on the image and populates the metafield “dc:description” on the DAM asset.
AEM “Smart Tags”
Adobe has also announced a native auto asset tagging feature called “Smart Tags” in its AEM 6.2 beta version which was received very well at the Adobe Summit—The Digital Marketing Conference 2016.  Though it is not available for public yet, it would be very interesting to compare it with Google Cloud Vision API’s once available. The Google Vision / AEM integration will work in any version of AEM, not just 6.2.
Conclusion
We are very excited to see what machine learning and AI are contributing to asset repositories in terms of classification. As we keep testing, we are amazed at the great results and disappointed with few misses (It seems to always fail to recognize the Nike logo). As Chad pointed out, the results are not always accurate and for a beta release of Cloud API, we are very impressed. We would now like to see how they compare with Adobe “Smart Tagging” native feature.