Graphpipeをこれまで使ってきましたが、モデルサーバーとして最も使われている(らしい)Tensorflow Servingを試してみようと思います。
1.dockerでの起動
モデルは下記に配置。
total 1940
-rw-r–r–. 1 root root 1983910 Apr 4 14:17 hiragana.pb
サクッと起動できるかと思いましたが、そうではないみたいですね。
WARNING: IPv4 forwarding is disabled. Networking will not work.
2019-04-05 04:15:10.451307: I tensorflow_serving/model_servers/server.cc:82] Building single TensorFlow model file config: model_name: hiragana model_base_path: /models/hiragana
2019-04-05 04:15:10.451708: I tensorflow_serving/model_servers/server_core.cc:461] Adding/updating models.
2019-04-05 04:15:10.451729: I tensorflow_serving/model_servers/server_core.cc:558] (Re-)adding model: hiragana
2019-04-05 04:15:10.452257: W tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:268] No versions of servable hiragana found under base path /models/hiragana
2019-04-05 04:15:11.452352: W tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:268] No versions of servable hiragana found under base path /models/hiragana
2019-04-05 04:15:12.452454: W tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:268] No versions of servable hiragana found under base path /models/hiragana
どうやら、モデルの配置ディレクトリ構造の制約があるようです。
# mv /tmp/tfs/models/hiragana.pb /tmp/tfs/models/1
“1”フォルダを新規作成し、pbファイルを移動させます。
WARNING: IPv4 forwarding is disabled. Networking will not work.
2019-04-05 04:17:15.458502: I tensorflow_serving/model_servers/server.cc:82] Building single TensorFlow model file config: model_name: hiragana model_base_path: /models/hiragana
2019-04-05 04:17:15.458797: I tensorflow_serving/model_servers/server_core.cc:461] Adding/updating models.
2019-04-05 04:17:15.458817: I tensorflow_serving/model_servers/server_core.cc:558] (Re-)adding model: hiragana
2019-04-05 04:17:15.559948: I tensorflow_serving/core/basic_manager.cc:739] Successfully reserved resources to load servable {name: hiragana version: 1}
2019-04-05 04:17:15.559987: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: hiragana version: 1}
2019-04-05 04:17:15.560007: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: hiragana version: 1}
2019-04-05 04:17:15.560056: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: hiragana version: 1} failed: Not found: Specified file path does not appear to contain a:
– Session bundle (should have a file called
export.meta
)– or, SavedModel bundle (should have a file called
saved_model.pb
)Specified file path: /models/hiragana/1
metaファイルを用意するか、そうでなければsaved_model.pbというファイル名でないといけないとのこと。
# ls -ltr /tmp/tfs/models/1/saved_model.pb
-rw-r–r–. 1 root root 1983910 Apr 4 14:17 /tmp/tfs/models/1/saved_model.pb
ファイル名を変えて再実行してみます。
WARNING: IPv4 forwarding is disabled. Networking will not work.
2019-04-05 04:19:48.741053: I tensorflow_serving/model_servers/server.cc:82] Building single TensorFlow model file config: model_name: hiragana model_base_path: /models/hiragana
2019-04-05 04:19:48.741279: I tensorflow_serving/model_servers/server_core.cc:461] Adding/updating models.
2019-04-05 04:19:48.741296: I tensorflow_serving/model_servers/server_core.cc:558] (Re-)adding model: hiragana
2019-04-05 04:19:48.842278: I tensorflow_serving/core/basic_manager.cc:739] Successfully reserved resources to load servable {name: hiragana version: 1}
2019-04-05 04:19:48.842308: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: hiragana version: 1}
2019-04-05 04:19:48.842324: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: hiragana version: 1}
2019-04-05 04:19:48.842355: I external/org_tensorflow/tensorflow/contrib/session_bundle/bundle_shim.cc:363] Attempting to load native SavedModelBundle in bundle-shim from: /models/hiragana/1
2019-04-05 04:19:48.888869: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /models/hiragana/1
2019-04-05 04:19:49.024492: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2019-04-05 04:19:49.024724: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:285] SavedModel load for tags { serve }; Status: fail. Took 135848 microseconds.
2019-04-05 04:19:49.024774: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: hiragana version: 1} failed: Not found: Could not find meta graph def matching supplied tags: { serve }. To inspect available tag-sets in the SavedModel, please use the SavedModel CLI:
saved_model_cli
ん、、、tagがついてないとダメってことですか。
調べてみると、pbファイルでもtensorflowにてSavedModelBuilderにて作成されたpbファイルでないといけないみたいですね。
どうにもならないので、以下のソースをもとのlerning_hiranaga.pyに付け加えました。graph_io.write_graphにて出力したpbファイルを再読み込みし、SavedModel形式で出力させています。
OUTPUT_SERVABLE_FOLDER=”/tmp/tf”
if os.path.exists(OUTPUT_SERVABLE_FOLDER):
shutil.rmtree(OUTPUT_SERVABLE_FOLDER)
from tensorflow.python.saved_model import signature_constants
from tensorflow.python.saved_model import tag_constants
builder = tf.saved_model.builder.SavedModelBuilder(OUTPUT_SERVABLE_FOLDER)
#with tf.gfile.GFile(f'{OUTPUT_FOLDER}/{OUTPUT_GRAPH}’, “rb”) as f:
with tf.gfile.GFile(‘./hiragana.pb’, “rb”) as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
sigs = {}
OUTPUT_TENSOR = outputName
with tf.Session(graph=tf.Graph()) as sess:
tf.import_graph_def(graph_def, name=””)
g = tf.get_default_graph()
inp = model.input #g.get_tensor_by_name(INPUT_TENSOR)
out = g.get_tensor_by_name(OUTPUT_TENSOR[0] + ‘:0′)
sigs[signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY] = \
tf.saved_model.signature_def_utils.predict_signature_def(
{“input”: inp}, {“outout”: out})
builder.add_meta_graph_and_variables(sess,
[tag_constants.SERVING],
signature_def_map=sigs)
try:
builder.save()
print(f’Model ready for deployment at {OUTPUT_SERVABLE_FOLDER}/saved_model.pb’)
print(‘Prediction signature : ‘)
print(sigs[‘serving_default’])
except:
print(‘Error Occured, please checked frozen graph’)
SavedModel形式のpbファイルで再度docker起動します。
WARNING: IPv4 forwarding is disabled. Networking will not work.
2019-04-10 12:16:43.464937: I tensorflow_serving/model_servers/server.cc:82] Building single TensorFlow model file config: model_name: hiragana model_base_path: /models/hiragana
2019-04-10 12:16:43.471901: I tensorflow_serving/model_servers/server_core.cc:461] Adding/updating models.
2019-04-10 12:16:43.471936: I tensorflow_serving/model_servers/server_core.cc:558] (Re-)adding model: hiragana
2019-04-10 12:16:43.473535: I tensorflow_serving/core/basic_manager.cc:739] Successfully reserved resources to load servable {name: hiragana version: 1}
2019-04-10 12:16:43.473563: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: hiragana version: 1}
2019-04-10 12:16:43.473580: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: hiragana version: 1}
2019-04-10 12:16:43.473605: I external/org_tensorflow/tensorflow/contrib/session_bundle/bundle_shim.cc:363] Attempting to load native SavedModelBundle in bundle-shim from: /models/hiragana/1
2019-04-10 12:16:43.473624: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /models/hiragana/1
2019-04-10 12:16:43.610980: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2019-04-10 12:16:43.612493: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-04-10 12:16:43.633864: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:182] Restoring SavedModel bundle.
2019-04-10 12:16:43.633926: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:192] The specified SavedModel has no variables; no checkpoints were restored. File does not exist: /models/hiragana/1/variables/variables.index
2019-04-10 12:16:43.633950: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:285] SavedModel load for tags { serve }; Status: success. Took 160316 microseconds.
2019-04-10 12:16:43.633981: I tensorflow_serving/servables/tensorflow/saved_model_warmup.cc:101] No warmup data file found at /models/hiragana/1/assets.extra/tf_serving_warmup_requests
2019-04-10 12:16:43.634055: I tensorflow_serving/core/loader_harness.cc:86] Successfully loaded servable version {name: hiragana version: 1}
2019-04-10 12:16:43.636994: I tensorflow_serving/model_servers/server.cc:313] Running gRPC ModelServer at 0.0.0.0:8500 …
[warn] getaddrinfo: address family for nodename not supported
2019-04-10 12:16:43.638015: I tensorflow_serving/model_servers/server.cc:333] Exporting HTTP/REST API at:localhost:8501 …
[evhttp_server.cc : 237] RAW: Entering the event loop …
エラーは無くなりました。
2.動作確認
とりあえずの動作確認はHTTPリクエストを投げることでできます。
{
“model_version_status”: [
{
“version”: “1”,
“state”: “AVAILABLE”,
“status”: {
“error_code”: “OK”,
“error_message”: “”
}
}
]
}
サーバーとしてはAvailableのようです。