Deploy Models with TensorFlow Serving and Flask

Posted on | In , , , | |

A hands-on project from coursera course Deploy Models with TensorFlow Serving and Flask

Course Certificate
  1. Write a front view handle image uplaod

base.html

1
2
3
4
5
{% extends "bootstrap/base.html" %}
{% block title %}Dog vs Cat{% endblock %}

{% block content %}
{% endblock %}

index.html 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
{% extends "base.html" %}

{% block content %}
<div class="container">
    <h1>Upload File</h1>
    <hr>
    <form method="post" enctype="multipart/form-data" class="form-inline">
        <div class="form-group">
            <input type="file" name="file" class="btn btn-default">
        </div>
        <div class="form-group">
            <input type="submit" value="Upload", class="btn btn-default">
        </div>
    </form>
</div>
{% endblock %}
  1. Use docker to deployment tensorflow serving.
1
docker run -p 8502:8501 --name=pets -v "/home/models/pets/:/models/pets/1" -e MODEL_NAME=pets tensorflow/serving

This will copy the model from /home/models/pets/ which in your desktop to the path /models/pets/1 in docker

The port 8501 is defined by docker and you can change the 8502 to any port you can you used.

  1. Use Flask to process HTTP Requests and do model inference in docker.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
import os
import string
import random
import json
import requests
import numpy as np
import tensorflow as tf

from flask import Flask, request, redirect, url_for, render_template
from flask_bootstrap import Bootstrap

app = Flask(__name__)
Bootstrap(app)

"""
Constants
"""
MODEL_URI = 'http://localhost:8502/v1/models/pets:predict'
OUTPUT_DIR = 'static'
CLASSES = ['Cat', 'Dog']
SIZE = 128

"""
Utility functions
"""
def generate_filename():
    return ''.join(random.choices(string.ascii_lowercase, k=20)) + '.jpg'

def get_prediction(image_path):
    image = tf.keras.preprocessing.image.load_img(image_path, target_size=(SIZE, SIZE))
    image = tf.keras.preprocessing.image.img_to_array(image)
    image = tf.keras.applications.mobilenet_v2.preprocess_input(image)
    image = np.expand_dims(image, axis=0)

    data = json.dumps({'instances': image.tolist() })
    response = requests.post(MODEL_URI, data = data.encode())
    result = json.loads(response.text)
    prediction = result['predictions'][0]
    class_name = CLASSES[int(prediction > 0.5)]
    return class_name

"""
Routes
"""
@app.route('/', methods=['GET', 'POST'])
def index():
    if request.method == 'POST':
        uploaded_file = request.files['file']
        if uploaded_file.filename != '':
            if uploaded_file.filename[-3:] in ['jpg', 'png']:
                image_path = os.path.join(OUTPUT_DIR, generate_filename())
                uploaded_file.save(image_path)
                class_name = get_prediction(image_path)
                result = {
                    'class_name': class_name,
                    'path_to_image': image_path,
                    'size': SIZE
                }
                return render_template('show.html', result=result)
    return render_template('index.html')

if __name__ == '__main__':
    app.run(debug=True)
  1. Rendering results in template
1
2
3
4
5
6
7
8
9
10
11
{% extends "base.html" %}

{% block content %}
<div class="container">
    <h1>Predicted Class: {{ result.class_name }}</h1>
    <hr>
    <div><img src="{{ result.path_to_image }}" class="img-rounded" width="{{ result.size }}" height="auto"></div>
    <hr>
    <a href = "{{ url_for('index') }}" class="btn btn-default">Go Back</a>
</div>
{% endblock %}