Image classification with Azure Cognitive Services (and C#)

I live next to the park and every summer pizza-delivery guys keep ringing my doorbell to deliver pizza for people in the park. So I figured I’ll train my doorbell to recognise them and disable the doorbell as soon as it identifies one. To do so, I needed image classification. This blogpost describes how to choose the proper tool and how to implement it to do that.

To build image classification you don’t need to know much about machine learning anymore. Image classification is a common problem and you can use standard SaaS services to solve such problems in any cloud.

In Azure there are two interesting services that might do the job:

• Azure Computer Vision
• Azure Custom Vision

Read this blogpost to learn the basics of image classification, what services you can use, and how you can implement them in your solution.

How does Image Classification work?

To get a computer to recognise objects in a picture, you’ll need to train the computer. This simply means you must provide a number of pictures that show the object you are trying to identify from different perspectives. The more, the better.

TensorFlow and Neural Networks

If you want to get a better understanding of how image classification works, you can build image classification with TensorFlow. TensorFlow is a free Python library built by Google. As the name implies, it works with Tensors. A tensor is a matrix. And that’s exactly what an image is: it’s a pixel-matrix.

TensorFlow uses neural networks to “learn”. The neurons are trained to confirm or falsify a hypotheses. In practice this means the images you’re “feeding” the machine learning will not be exactly the same, but it will most likely contain similarities. You can use neural networks to identify those. Under the bonnet, it’s likely that Azure Cognitive Services uses TensorFlow.

Understand that image classification is not rocket science. If you want you can build this yourself.

You can, but you don’t need to build it yourself

Recognising objects in a picture is a common problem. That’s why most cloud vendors have services you can use to do so. In Azure for example, there are two services that can do this: