I know this sounds crazy, but one of the biggest struggles that Machine Learning engineers face is to take their models and make them available so they can be used as intended.
To be a little more specific, taking what seems to work fine in a laptop and deploying it to production so others can take advantage of it seems to be more complicated than what you may believe it should be.
This is something that software engineers have dealt with for decades. There are tools, processes, videos, books, and tricks about how to do it. But deploying machine learning models is comparatively a very young problem, and very likely outside a Data Scientist list of competencies unless they come from an engineering background.
A lot depends on the model you have, so I can't answer all the questions for you, but if you are looking to deploy an Object Detection model using TensorFlow, I can show you a couple of options that will brighten your day.
TLDR;
This GitHub repository has what you need. You can get the code and the instructions on how to run it.
You have two options:
- You can deploy your TensorFlow Object Detection model on SageMaker, or
- You can deploy on-premises.
The beauty is that you can use the same exact Docker container to do that. One container, two options to get it running.
Random notes, if you are curious
You probably don't need to read any of this to get things up and running, but in case you want a little bit more information, here is some.
Dockerizing this thing makes sure you can deploy your model pretty much anywhere you want. I'm not going to explain what's Docker, but if you are reading this, you probably know already. (And if you don't, cancel your plans for the weekend, grab the popcorn, and solve that problem).
The structure of the files and code inside the Docker container makes SageMaker happy. This container was created to run on SageMaker first, then adapted to run locally (on-premises) as well. I use it both ways. I've never tried it elsewhere, but it is just a Docker container, so it should work as long as you can talk to it through HTTP.
If you go over the README.md documentation, you'll quickly realize that I spent a long time explaining how to set up things on SageMaker (not only the container but how to configure a training job and connect all the pieces to train a model.) You don't need any of this if you are training your model elsewhere, but I wanted to show how to run the entire pipeline on SageMaker.
Finally, at the time of this writing, the TensorFlow Object Detection API doesn't support TensorFlow 2.0. The latest working version of TensorFlow is 1.15, and that's what you'll get on the GitHub repository. I'm pretty sure I'll update it as soon as 2.0 is supported, but for now, that's the best we get.