How to Enable Video Scanning Using Google Cloud Intelligence ?

Unleashing the Power of Google Cloud Platform for Video Analysis and Processing

The Google Cloud Platform (GCP) is a storehouse of resources, and you don’t need significant technical nous to realize its advantages and explore its many possibilities.     

Google Cloud provides many advantages for video analysis, as part of Google’s content submissions scanning services that employ video intelligence. GCP services help utilize a scalable system for scanning and filtering video and image submission with the Google Cloud Vision and Google Cloud Video Intelligence APIs.

Encouraging users to submit their images is a trend often seen in various social marketing campaigns. In these campaigns, the images and videos are used for product testimonials, content submissions, public campaigns, etc. In normal terms, processing these videos and images would consume a lot of resources. With the aforementioned APIs, you get a scalable manner to scan videos and images, filter them, and accelerate the processing of submissions. All this can be carried out through server-less procedures, further reducing the complexity.   

Here are some of the challenges involved in processing user-generated content such as videos and images:  

  • There is the scalability issue, since at the beginning of user-generated campaigns the submissions quickly grow, and then drop to nearly zero towards the end of the campaign. When this gets built into a service, the response to user activity must be scaled in.
  • There needs to be an intelligent and efficient flow in processing every video and image. Each of the submitted videos and images must be stored and processed efficiently at scale.   
  • Greater efficiency can be ensured by reducing the videos and images that need to be reviewed or evaluated before they’re processed.  For this, the application must classify each of the submissions and stop processing results that are detected as inappropriate.

To sail through these challenges Google Cloud Platform gives you a scalable platform offering the advantages of pre-trained models of machine learning (ML) available through API calls. The architecture of the system that classifies video and image submissions intelligently while filtering out submissions that are inappropriate is depicted in the following figure:

According to this architecture, all the content is stored in Google Cloud Storage. It provides scalable and durable object storage.  Cloud Storage is able to generate notification messages on the basis of events in the Cloud Storage bucket. This feature enables an action to be specified for each uploaded file. As the files are uploaded to Cloud Storage, the notification messages initiate the processing.

Reliable and Scalable Messaging Service

The Google Cloud Pub/Sub provides a reliable messaging service that is also scalable. When the content gets uploaded to the Cloud Storage bucket, the system comes up with a notification message, sending it to the Cloud Pub/Sub topic that is configured. The notification message is then delivered by the Cloud Pub/Sub. As the upload functionality is separated from processing functionality, Cloud Pub/Sub separates this application into a backend architecture based on microservices.  

Integration of Cloud Functions with Advanced APIs

Google Cloud Functions are integrated with various advanced APIs such as Video Intelligence and Cloud Vision APIs. Moreover, there are other features integrated into this such as the Google BigQuery storage server, Cloud Storage, Google Cloud Datastore and Google Cloud Spanner. With these integrated services, Cloud Functions is really effective in processing on-demand events.

Backend Processing through Various Cloud Functions

Backend processing is provided by a collection of Cloud Functions that heed to Cloud Pub/Sub topics. Every time a video or image gets uploaded to Cloud Storage, a message gets to a Cloud Pub/Sub topic. The Cloud Storage location is parsed by the backend application from this notification message. That’s how the application determines if the content is a video or an image.

Then, the application transmits a Cloud Pub/Sub message to Cloud Functions and calls the Cloud Video Intelligence or Cloud Vision API depending on if the file is a video or an image. With the API call results, the content is classified on safe search results, text, labels, and logos.

The figure below reveals the processing steps carried out by each Cloud Function. Here you can find the results stored in BigQuery, though they can be stored in any of the GCP data-storage services.          

Frontend Architecture

In the frontend architecture, you can store the video and image processing results in BigQuery. BigQuery serves as a scalable analytics engine for the Google Cloud Platform (GCP). It offers SQL-based access to a mammoth database. BigQuery is a scalable and simple manner of studying labels for the content that’s submitted.

Google Data Studio has a visual dashboard where submitted content can be filtered with tags that have been generated by the machine learning APIs. With tags, it is easy for filtering content dynamically in the Data Studio dashboards.   

In the following figure, you can see a Data Studio dashboard powered by BigQuery’s real-time data. The dashboard can be filtered by labels, content or content count. There are changes reflected in the dashboard as BigQuery processes and stores further content.

Other Video Intelligence Services

The Google Cloud Video Intelligence API helps make videos searchable. It extracts metadata with a simple REST API. It helps you search each moment and each occurrence and their significance in all the video files in your catalog. Videos in Google Cloud Storage are quickly annotated to help you identify your video’s important nouns entities and the time of their occurrence in the video. You can separate the signal from the noise through retrieval of relevant information per frame or at the shot.

The Cloud Video Intelligence API is in Private Beta and makes use of deep-learning models that are built with frameworks such as TensorFlow and are then applied to platforms such as YouTube. Developers can provide information about entities inside the video content using nouns or verbs to search videos or discover video content. Searching for a term would also help find all shots containing the searched topic in Google Cloud Storage’s video collection.

With the Google Cloud Platform, you can save precious resources and effort through a simple, server-less platform employing a host of machine learning and video intelligence capabilities. Get started with video scanning!