Tensorflow Servingを試してみる #1

# ls -ltr /tmp/tfs/models/
total 1940
-rw-r–r–. 1 root root 1983910 Apr 4 14:17 hiragana.pb

# docker run –rm –name tfs_test -p 8501:8501 –mount type=bind,source=/tmp/tfs/models/,target=/models/hiragana -e MODEL_NAME=hiragana -t tensorflow/serving
WARNING: IPv4 forwarding is disabled. Networking will not work.
2019-04-05 04:15:10.451307: I tensorflow_serving/model_servers/server.cc:82] Building single TensorFlow model file config: model_name: hiragana model_base_path: /models/hiragana
2019-04-05 04:15:10.451708: I tensorflow_serving/model_servers/server_core.cc:461] Adding/updating models.
2019-04-05 04:15:10.451729: I tensorflow_serving/model_servers/server_core.cc:558] (Re-)adding model: hiragana
2019-04-05 04:15:10.452257: W tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:268] No versions of servable hiragana found under base path /models/hiragana
2019-04-05 04:15:11.452352: W tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:268] No versions of servable hiragana found under base path /models/hiragana
2019-04-05 04:15:12.452454: W tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:268] No versions of servable hiragana found under base path /models/hiragana

# mkdir -p /tmp/tfs/models/1
# mv /tmp/tfs/models/hiragana.pb /tmp/tfs/models/1

# docker run –rm –name tfs_test -p 8501:8501 –mount type=bind,source=/tmp/tfs/models/,target=/models/hiragana -e MODEL_NAME=hiragana -t tensorflow/serving
WARNING: IPv4 forwarding is disabled. Networking will not work.
2019-04-05 04:17:15.458502: I tensorflow_serving/model_servers/server.cc:82] Building single TensorFlow model file config: model_name: hiragana model_base_path: /models/hiragana
2019-04-05 04:17:15.458797: I tensorflow_serving/model_servers/server_core.cc:461] Adding/updating models.
2019-04-05 04:17:15.458817: I tensorflow_serving/model_servers/server_core.cc:558] (Re-)adding model: hiragana
2019-04-05 04:17:15.559948: I tensorflow_serving/core/basic_manager.cc:739] Successfully reserved resources to load servable {name: hiragana version: 1}
2019-04-05 04:17:15.559987: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: hiragana version: 1}
2019-04-05 04:17:15.560007: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: hiragana version: 1}
2019-04-05 04:17:15.560056: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: hiragana version: 1} failed: Not found: Specified file path does not appear to contain a:
– Session bundle (should have a file called export.meta)
– or, SavedModel bundle (should have a file called saved_model.pb)
Specified file path: /models/hiragana/1

# mv /tmp/tfs/models/1/hiragana.pb /tmp/tfs/models/1/saved_model.pb
# ls -ltr /tmp/tfs/models/1/saved_model.pb
-rw-r–r–. 1 root root 1983910 Apr 4 14:17 /tmp/tfs/models/1/saved_model.pb

docker run –rm –name tfs_test -p 8501:8501 –mount type=bind,source=/tmp/tfs/models/,target=/models/hiragana -e MODEL_NAME=hiragana -t tensorflow/serving
WARNING: IPv4 forwarding is disabled. Networking will not work.
2019-04-05 04:19:48.741053: I tensorflow_serving/model_servers/server.cc:82] Building single TensorFlow model file config: model_name: hiragana model_base_path: /models/hiragana
2019-04-05 04:19:48.741279: I tensorflow_serving/model_servers/server_core.cc:461] Adding/updating models.
2019-04-05 04:19:48.741296: I tensorflow_serving/model_servers/server_core.cc:558] (Re-)adding model: hiragana
2019-04-05 04:19:48.842278: I tensorflow_serving/core/basic_manager.cc:739] Successfully reserved resources to load servable {name: hiragana version: 1}
2019-04-05 04:19:48.842308: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: hiragana version: 1}
2019-04-05 04:19:48.842324: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: hiragana version: 1}
2019-04-05 04:19:48.842355: I external/org_tensorflow/tensorflow/contrib/session_bundle/bundle_shim.cc:363] Attempting to load native SavedModelBundle in bundle-shim from: /models/hiragana/1
2019-04-05 04:19:48.888869: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /models/hiragana/1
2019-04-05 04:19:49.024492: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2019-04-05 04:19:49.024724: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:285] SavedModel load for tags { serve }; Status: fail. Took 135848 microseconds.
2019-04-05 04:19:49.024774: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: hiragana version: 1} failed: Not found: Could not find meta graph def matching supplied tags: { serve }. To inspect available tag-sets in the SavedModel, please use the SavedModel CLI: saved_model_cli

OUTPUT_SERVABLE_FOLDER=”/tmp/tf”
if os.path.exists(OUTPUT_SERVABLE_FOLDER):
shutil.rmtree(OUTPUT_SERVABLE_FOLDER)

from tensorflow.python.saved_model import signature_constants
from tensorflow.python.saved_model import tag_constants

builder = tf.saved_model.builder.SavedModelBuilder(OUTPUT_SERVABLE_FOLDER)

#with tf.gfile.GFile(f'{OUTPUT_FOLDER}/{OUTPUT_GRAPH}’, “rb”) as f:
with tf.gfile.GFile(‘./hiragana.pb’, “rb”) as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())

sigs = {}
OUTPUT_TENSOR = outputName
with tf.Session(graph=tf.Graph()) as sess:
tf.import_graph_def(graph_def, name=””)
g = tf.get_default_graph()
inp = model.input #g.get_tensor_by_name(INPUT_TENSOR)
out = g.get_tensor_by_name(OUTPUT_TENSOR[0] + ‘:0′)

sigs[signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY] = \
tf.saved_model.signature_def_utils.predict_signature_def(
{“input”: inp}, {“outout”: out})

builder.add_meta_graph_and_variables(sess,
[tag_constants.SERVING],
signature_def_map=sigs)
try:
builder.save()
print(f’Model ready for deployment at {OUTPUT_SERVABLE_FOLDER}/saved_model.pb’)
print(‘Prediction signature : ‘)
print(sigs[‘serving_default’])
except:
print(‘Error Occured, please checked frozen graph’)

# docker run –rm –name tfs_test -p 8500:8500 -p 8501:8501 –mount type=bind,source=/tmp/tfs/models/,target=/models/hiragana -e MODEL_NAME=hiragana -t tensorflow/serving
WARNING: IPv4 forwarding is disabled. Networking will not work.
2019-04-10 12:16:43.464937: I tensorflow_serving/model_servers/server.cc:82] Building single TensorFlow model file config: model_name: hiragana model_base_path: /models/hiragana
2019-04-10 12:16:43.471901: I tensorflow_serving/model_servers/server_core.cc:461] Adding/updating models.
2019-04-10 12:16:43.471936: I tensorflow_serving/model_servers/server_core.cc:558] (Re-)adding model: hiragana
2019-04-10 12:16:43.473535: I tensorflow_serving/core/basic_manager.cc:739] Successfully reserved resources to load servable {name: hiragana version: 1}
2019-04-10 12:16:43.473563: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: hiragana version: 1}
2019-04-10 12:16:43.473580: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: hiragana version: 1}
2019-04-10 12:16:43.473605: I external/org_tensorflow/tensorflow/contrib/session_bundle/bundle_shim.cc:363] Attempting to load native SavedModelBundle in bundle-shim from: /models/hiragana/1
2019-04-10 12:16:43.473624: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /models/hiragana/1
2019-04-10 12:16:43.610980: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2019-04-10 12:16:43.612493: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-04-10 12:16:43.633864: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:182] Restoring SavedModel bundle.
2019-04-10 12:16:43.633926: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:192] The specified SavedModel has no variables; no checkpoints were restored. File does not exist: /models/hiragana/1/variables/variables.index
2019-04-10 12:16:43.633950: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:285] SavedModel load for tags { serve }; Status: success. Took 160316 microseconds.
2019-04-10 12:16:43.633981: I tensorflow_serving/servables/tensorflow/saved_model_warmup.cc:101] No warmup data file found at /models/hiragana/1/assets.extra/tf_serving_warmup_requests
2019-04-10 12:16:43.634055: I tensorflow_serving/core/loader_harness.cc:86] Successfully loaded servable version {name: hiragana version: 1}
2019-04-10 12:16:43.636994: I tensorflow_serving/model_servers/server.cc:313] Running gRPC ModelServer at 0.0.0.0:8500 …
[warn] getaddrinfo: address family for nodename not supported
2019-04-10 12:16:43.638015: I tensorflow_serving/model_servers/server.cc:333] Exporting HTTP/REST API at:localhost:8501 …
[evhttp_server.cc : 237] RAW: Entering the event loop …

# curl http://127.0.0.1:8501/v1/models/hiragana
{
“model_version_status”: [
{
“version”: “1”,
“state”: “AVAILABLE”,
“status”: {
“error_code”: “OK”,
“error_message”: “”
}
}
]
}