Table of contents

Recently I implement an end to end solution to better understand how a TensorFlow model could be leveraged in a production environment. There are many great resources available online that walk you through setting up and training TensorFlow models and deploying them as services. However during this short project I ran into several issues (mostly due to my lack of experience and understanding) and wanted to document my general process and issue resolution here so others can potentially learn from my mistakes and lessons.

Although I undertook this project to learn about TensorFlow model serving I wanted to tackle an end to end challenge to ensure my understanding (and that my setup works).

Project scope

This project is broken down into 3 sections/posts:

1. Build, train and save a set of TensorFlow models
2. Set up a docket container to host the TensorFlow models
3. Deploy a simple Angular user interface to consume the service exposing the TensorFlow models

Note that the second section is the one that deals with the actual hosting of TensorFlow models while the other 2 sections help set up a testing platform.  In terms of a problem for my TensorFlow model to solve I chose the MNIST digit data set. This dataset is readily available online and typically yields very good results with even the most basic models. This project will involve a set of models to classify these digits, host the models and then provide an interface for a user to draw digit for classification.

High level architecture

A single Virtual Machine is leveraged for this project and the docker containers will be used for the deployment of isolated components as illustrated below. For my implementation an Ubuntu Google Cloud Platform Virtual Machine instance was leveraged but any VM can be used.

What will be covered in my post regarding this project

1. High level overview of leveraged technology
2. TensorFlow model structure, training and hyper parameters I found to be successful
3. Detailed process of updating TensorFlow code to generate servable models
4. Configuration of the TensorFlow docker serving image to deploy multiple models
5. High level overview of my Angular app deployment and integration with the TensorFlow service
6. High level overview of deploying both the Angular and TensorFlow serving containers on a Google Cloud Platform instance

What will not be covered in my post regarding this project

1. TensorFlow and Angular code walk through - I will provide links to resources that do this much better then I and I will also be providing mt source code
2. Details walk through of setting up of hosted instance - Relevant links will be provided

Here is a link to a working demo, keep in mind the objective of this project is not to optimize accuracy. The model is finicky with off center and large/small images as it does not take advantage of convolution.

Here is a summary of the components involved in this project