Introduction

In the last decade, artificial intelligence has accomplished many significant feats: from classifying an image to tracking objects in video feeds, from beating a Go player to beating a Starcraft 2 pro and more. After such a successful decade, it is evident that academic advance is slowing down and the research lately is more incremental than fundamental. However, that also means that the AI technology is maturing and we are finding new domains where technology can be applied to.

Having seen/ read/ been amazed by these applications as a business member you maybe be starting to wonder whether it is time for you to try out these approaches in your work. In this blog post, we will share our experience in solving business problems with AI and hopefully help you avoid repeating our mistakes.

Adding ML to your IT stack VS adding ML to your business stack

Before jumping into the core matter,  we should make a distinction between two flavors of deployment issues:

Problem 1: Adding ML to the IT stack. ML is compute-heavy and requires compute-oriented software and hardware which might not be present in IT infrastructure. Therefore, how and where the algorithms are run needs to be decided, and appropriate skills need to be added to or developed in your IT department.

Problem 2: Dealing with the implications of having ML in your business processes. Most of ML methods are non-deterministic and they are evaluated on a statistical basis. That means that even when the methods work it is possible for them to deliver the wrong result. This stochastic nature of the algorithm can be a novelty to a business which depends on a fully deterministic IT system. Business processes affected by such randomness need to be identified, the impact evaluated and, if needed, policies to address it are required.

Problem 1 heavily depends on the existing infrastructure and the algorithm specifics. Moreover, it is the more obvious of the issues and has been documented extensively. To the extent that it could be solved by completely outsourcing it to the cloud - store the data, train the models, use the models, all in the cloud. There are multiple good tools to achieve that, for example Google cloud offers AutoML and Amazon AWS offers multiple AI/ML services, just to name a few.

There is much less information on Problem 2, but it can make or break the business proposition of an AI/ML application. Usually, when clients approach AI/ML developers they ask “Can you solve problem X?”, developers reply “Sure, what accuracy do you need?”. The jump from a problem to an AI algorithm is a non-trivial one and requires careful attention.

To link the business problem to an ML algorithm one needs to deeply understand where the value is added in the process. For example, AI is increasingly being applied for initial screening of medical patients and there you should be extremely cautious about false negatives, i.e. where the patient is screened as healthy when in fact he might be severely ill. The impact of the false positive in that scenario would most likely only lead to and increase in costs. Ignoring these aspects of accuracy could lead to a deadly outcome!

Therefore, framing the real-life problem into an AI solution is a significant part of the development and in this blog series, we will share our experience and tips on how to avoid some of the dangerous pitfalls.

Examples

We start by sharing two examples that we’ve encountered that are spaced nearly a decade apart. The older one will be used to set a reference point of what developers and IT managers are used to and the more recent one will shed some light on how AI algorithms are developed nowadays. Contrasting the two allows us to compare in what ways we should change how we think about development and how to be prepared to manage what is to come.

Although we focus on image recognition, the issues are shared across a wide variety of data-based ML approaches.

How production has been running and what we are used to

In 2012 we were approached by a client to develop licence plate recognition from images for their parking lot. The client gave us a dataset of 100 images and we started to work on it.

Recall that in 2012 the situation with respect to AI/ML was very different. Although deep learning had been making waves in the research community, there were much fewer software tools available, much less knowledge on how to train and use neural nets, and it was much harder to run algorithms on GPUs.

The way production computer vision was at the time - use heuristics to simplify the problem and sometimes use some neural network approach to solve that simple problem.

The solution we came up worked along these lines:

1. Extract white regions from the image;

Select the regions fulfilling certain conditions:

1. A certain range of heights and widths;
2. A specific range of aspect ratios;
3. Containing some fraction of non-white regions;
4. …

Loop over the selected regions and

1. Detect non-white contours which have letter-like features: location in the region, colour, height, width, aspect ratio and etc.

For each of the letter-like contour try to extract the letter by either:

1. Matching it to a database;
2. Using some proprietary letter detector;
3. Using some neural network;
4. A combination of the above;
5. Try combining the extracted symbols into a single license plate;
6. If all the steps succeeded return the license plate.

The solution took us a few months and was fairly restrictive:

• Required certain lighting conditions;
• The license plate must be clearly visible, close to the center of the image and not too skewed;
• A high-quality camera was required;
• Adding new features (e.g. enabling night vision ) required a full reworking of the algorithm.

In many cases, the algorithm crashed at one of the steps and did not return any result. In the cases it worked, the accuracy reached a respectable 95%. The overall accuracy was fairly low - maybe around 60% since in many cases it did not return, but given that cameras do 30 frames per second the overall solution was worth the effort. The problem was solved and has been used in production for a while.

How the AI/ML production works and what we should get used to for the next decade

More recently in 2019, we were approached by a car-wash network manager who wanted to measure vehicle queue length using cameras.

We received a sample of 1000 images and started developing the solution. After manually labeling, we developed multiple candidates, but the high-level view of the solutions was:

1. Get the image;
2. Extract image features using a pre-trained neural network;
3. Use a more data-efficient ML algorithm to map features to the vehicle count.

The development of the principal algorithm took us a few days and it was 96% accurate. However, even though we had good accuracy in our sample which the client would accept as appropriate, we did not stop there and we’ll share why later on.

Comparison

Let’s compare the two approaches:

Heuristic-based algorithmML-based algorithmRequires less dataRequires more dataThe code is complicatedThe code is simpleThe decision logic is obviousA lot of maintenanceLikely to crash than give a wrong resultNever crashes, but might give the wrong answerSlow developmentFast developmentRuns on any PCFor efficient use requires GPU-enabled PC

These contrasts led us to identify the following challenges when working with AI algorithms.

The challenges of AI-based solutions/ Why are business processes important/What has changed?

• AI development is non-deterministic. Once you’ve built a few heuristic-based image recognition solutions, you generally know what is possible and what is not. When AI is involved it is uncertain whether you’re going to have the right data, enough data, enough computing power or brainpower to deal with the math involved to solve it. Even if you follow a paper with a similar problem being solved, usually many important details are left out and take a while to figure out the hard way.
• AI results are non-deterministic. They are likely to give the correct result, but it does not mean that the likelihood of a wrong one is small. In the case of the car-wash example, in one spot a car was drawn on the pavement and it got always detected as a waiting car.
• AI does not crash. As long as you give the algorithm the input in the shape it needs it will always give you a result. And sometimes it will be wrong. E.g. the license plate algorithm had so many checks along the way that it crashed instead of giving a wrong result, making the algorithm much more usable.
• AI solutions are opaque vs transparent. License plate solution consisted of many small steps and debugging it was pretty simple since it was obvious where the system stopped working. Figuring out why the car count algorithm went broken was much harder. Maybe the error was an unseen case?
• ML-based solutions are based on the data available before the deployment stage. The same can be said about the heuristic-based solution, but heuristics are usually made such that their logic is meant to span beyond the sample set. In a related scenario, the first batch of images we got for the vehicle queue contained images during very good weather. As expected once we asked to for more samples weirder cases started appeared - the rain made some of the images unworkable and threw off our initial algorithm.

Conclusions

In this blog post, we shared our experience with two similar problems and their two different solutions. The heuristic approach has been in use for a long period of time and the AI one is slowly taking over. Contrasting how the two are developed and used hints at the challenges of the new approach. Having figured out that in this blog post we will elaborate on how such projects should be structured to achieve maximum success in the next part.