Zero to Singularity: Create, Tune, Deploy and Scale a Deep Neural Network in 90 Minutes
This notebook is part of a masterclass held at IBM Think on 13th of February 2019 in San Fransisco In this exercise you will train a Keras DeepLearning model running on top of TensorFlow.
Note: For sake of bringing the training runtime down we've done two things
Used a softmax regression model over a Convolutional Neural Network
Trained only for one epoch instead of 20
This leads to approx. 5% less accuracy
Authors
Romeo Kienzler - Chief Data Scientist, IBM Watson IoT
Please make sure the currently installed version of Keras and Tensorflow are matching the requirememts, if not, please run the two PIP commands below in order to re-install. Please restart the kernal before proceeding, please re-check if the versions are matching.
1 2 3
import keras print('Current:\t', keras.__version__) print('Expected:\t 2.2.5 ')
Using TensorFlow backend.
Current: 2.2.4
Expected: 2.2.5
1 2 3
import tensorflow as tf print('Current:\t', tf.__version__) print('Expected:\t 1.15.0')
Current: 1.13.1
Expected: 1.15.0
IMPORTANT !!!
If you ran the two lines below please restart your kernel (Kernel->Restart & Clear Output)
#some learners constantly reported 502 errors in Watson Studio. #This is due to the limited resources in the free tier and the heavy resource consumption of Keras. #This is a workaround to limit resource consumption
import keras from keras.models import Model from keras.layers import Input, Dense from keras.layers import Dense, Dropout, Flatten from keras.layers import Conv2D, MaxPooling2D from keras.datasets import mnist from keras.models import Sequential, load_model from keras.optimizers import RMSprop from keras.layers import LeakyReLU
from keras import backend as K import numpy as np
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
batch_size = 128 num_classes = 10 epochs = 1
# the data, split between train and test sets (x_train, y_train), (x_test, y_test) = mnist.load_data()
WARNING:tensorflow:From /opt/conda/envs/Python36/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /opt/conda/envs/Python36/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
Train on 60000 samples, validate on 10000 samples
Epoch 1/1
60000/60000 [==============================] - 14s 230us/step - loss: 0.3857 - acc: 0.8886 - val_loss: 0.3232 - val_acc: 0.9110
Accuracy: 0.911
1 2 3
#some cleanup from the previous run !rm -f ker_* !rm -f my_best_model.tgz
You should see an accuracy of approximately 90%. Now lets define a hyper-parameter grid including different activation functions and gradient descent optimizers. We’re optimizing over the grid using grid search (nested for loops) and store each model variant in a file. We then decide for the best one in order to deploy to IBM Watson Machine Learning.
for activation_function_layer_1 in activation_functions_layer_1: for opimizer in opimizers: model = Sequential() model.add(Dense(512, activation = activation_function_layer_1, input_shape=(784,))) model.add(Dense(num_classes, activation='softmax'))
Let's have a look at all the models and see which hyper parameter configuration was the best one. You should see that relu and rmsprop gives you > 95% of accuracy on the validation set
We will use watson_machine_learning_client python library to save the trained model to WML Repository, to deploy the saved model and to make predictions using the deployed model.
watson_machine_learning_client can be installed using the following pip command in case you are running outside Watson Studio:
from watson_machine_learning_client import WatsonMachineLearningAPIClient
2020-04-15 22:57:03,361 - watson_machine_learning_client.metanames - WARNING - 'AUTHOR_EMAIL' meta prop is deprecated. It will be ignored.
Please go to https://cloud.ibm.com/, login, click on the “Create Resource” button. From the “AI” category, please choose “Machine Learning”. Wait for the “Create” button to activate and click on “Create”. Click on “Service Credentials”, then “New Credential”, then “Add”. From the new entry in the table, under “ACTIONS”, please click on “View Credentials”. Please copy the whole JSON object to your clipboard. Now just paste the JSON object below so that you are able to use your personal instance of Watson Machine Learning.
------------------------------------ ------------------- ------ -------------- ------------------------ --------------- -------------
GUID NAME TYPE STATE CREATED FRAMEWORK ARTIFACT TYPE
e69bb10d-67ea-4fa9-9788-4bbf80edac85 k1_keras_mnist_clt1 online DEPLOY_SUCCESS 2020-04-15T22:59:44.929Z tensorflow-1.15 model
------------------------------------ ------------------- ------ -------------- ------------------------ --------------- -------------
To keep your environment clean, just delete all deployments from previous runs