Image Recognition and Object Detection at Isobar

The field of machine learning has many different applications, one of which is image recognition. This is still a very new and experimental field and therefore, research and studies are making new breakthroughs every year.

We’ve been developing and setting up the infrastructure to harness the power of these cutting edge technologies to help our clients garner insights from images and videos.

What’s the point of image recognition?
Image recognition can help us uncover information locked in images and videos:
Automation: In the future, we may no longer need a human to scan through thousands upon thousands of images and videos looking for objects of interest.
Educational: You can get information about things you can see but don’t know what they are.
Diagnostic: Imagine an app that can detect potential cancerous moles by simply scanning your skin.

What is Isobar doing?
We have been developing a framework that can be deployed to AWS Sagemaker (Amazon’s ML platform) to train on any dataset of images. Our framework uses Tensorflow’s Object Detection API to train models that can be hosted in AWS Sagemaker or on a mobile device.

The images above are a very basic example of how an image classifier can be trained to detect logos of different Dentsu Aegis Network brands. A model like this can be deployed to a mobile device to detect logos in real time.

Why do we train our image classifiers in the cloud?
Training an object detection algorithm is a very resource intensive task that can take a long time….we would know. Originally we experimented with this technology by training models on our laptops – very time intensive. With a dataset of few hundred images it would take days and many crashes due to limited CPU and memory availability. To overcome this challenge Isobar partnered with AWS and deployed our framework to Sagemaker. Since we use Tensorflow for our training, we deployed everything on Sagemaker’s GPU enabled instances to maximize our training power. We can now train a dataset of many thousands of images in a few short hours.

Where do we host our image classifiers?
Isobar has developed the capability to host a model in the cloud or any mobile device, depending on how large and complex the trained classifiers are.

Mobile Device Hosting: Smaller and simpler models can be hosted directly on a mobile device to perform real-time object detection from a live camera feed. MobileNets (efficient convolutional neural networks for mobile vision applications) are so fast that they can process many frames of a live camera feed by detecting objects with hardly any lag. To accomplish this, Isobar uses Tensorflow Lite with quantization to shrink the size of a model and achieve high performance on mobile hardware.

Cloud Hosting: Larger, more complex models can have a much higher accuracy at the cost of speed and size. These types of models are not made to run on mobile hardware and therefore have to be hosted on more powerful machines. We deploy these models in AWS Sagemaker and make them available via a web endpoint. From there we can run inference and get results efficiently.

Our framework for training and hosting object detection models
The framework we’ve developed at Isobar provides the ability to train object detection algorithms with boundary boxes on CPU and GPU-enabled instances in AWS Sagemaker. We’ve also added the ability to host your trained classifiers in Sagemaker. Here’s a simple diagram for how we leverage AWS to serve our models:

You can read more about our framework and how to use it on our Github repository!