Vespa can import TensorFlow models converted to the ONNX format. The tutorial TensorFlow: Deploy model to Vespa through ONNX shows an end-to-end example from training a Learning-to-Rank (LTR) model to deploying it to Vespa. The tutorial can be reproduced by running the Jupyter Notebook.
Key steps covered in the tutorial above:
tf_model
.tf_model.save("tf_model_file")
tf2onnx
library:
python3 -m tf2onnx.convert --saved-model tf_model_file --output tf_model.onnx
onnx
library:
import onnx
m = onnx.load("simpler_keras_model.onnx")
m.graph.input # check input format
m.graph.output # check output format
schema msmarco {
document msmarco {
field id type string {
indexing: summary | attribute
}
field text type string {
indexing: summary | index
}
}
onnx-model ltr_tensorflow {
file: files/tf_model.onnx
input input: vespa_input
output dense: dense
}
rank-profile tensorflow {
function vespa_input() {
expression {
tensor<float>(x[1],y[3]):[
[fieldMatch(text).queryCompleteness,
fieldMatch(text).significance,
nativeRank(text)]
]
}
}
first-phase {
expression: sum(onnx(ltr_tensorflow).dense)
}
summary-features {
onnx(ltr_tensorflow)
fieldMatch(text).queryCompleteness
fieldMatch(text).significance
nativeRank(text)
}
}
}