“tiny-dnn is a C++11 implementation of deep learning. It is suitable for deep learning on limited computational resource, embedded systems and IoT devices.”
Recently I have been searching for a DNN library that can easily be integrated into all kinds of applications – i.e. C++, small footprint, 32- and 64 bit, Windows (Phone), iOS, Android, Linux – and then stumbled across tiny-DNN, which seemed to fulfill most of my criteria (except missing features for RNNs). Integrating tiny-DNN proved to be very easy, basically just copy the headers in there. I’ve tried to build a small Tensorflow runtime-only library for 32-bit Windows before that and it was a frustrating experience.
My goal was to leave the existing Theano model building code untouched but write a small exporter for my own protobuf format and then import rudimentary network architecture information and the network weights into tiny-DNN. In my case this was mainly a simple multi-layer feedforward neural network.
So if you have a similar problem, this might be helpful.
In protobuf I had a “Layer” message that contained something like:
... LayerType layerType = 1; ActivationType activationType = 2; int32 numInputs = 3; int32 numOutputs = 4; ...
My network weights from Theano were stored in a numpy array “weights”, so I just exported weights and biases as follows:
layer.weights.extend(weights.flatten()) layer.biases.extend(biases.flatten()) layer.numInputs = weights.shape[0] layer.numOutputs = weights.shape[1] ... with open(output_path, 'wb') as fout: fout.write(model.SerializeToString())
Finally, rebuilding and importing the network in tiny-DNN (pbModel and pbLayer are protobuf objects):
...
tiny_dnn::network<tiny_dnn::sequential> dnnModel;
for (auto pbLayer : pbModel.layers()) {
if (pbLayer.activationtype() == pbLayer.TANH) {
dnnModel << tiny_dnn::fc<tiny_dnn::tan_h>(pbLayer.numinputs(), pbLayer.numoutputs());
}
else if (pbLayer.activationtype() == pbLayer.LINEAR) {
dnnModel << tiny_dnn::fc<tiny_dnn::identity>(pbLayer.numinputs(), pbLayer.numoutputs());
}
auto& layer = *dnnModel[dnnModel.layer_size() - 1];
auto layerWeights = layer.weights();
auto& w = *layerWeights[0];
auto& b = *layerWeights[1];
auto& pbW = pbLayer.weights();
auto& pbB = pbLayer.biases();
layer.init_weight();
for (DNNFeatureVector::size_type i = 0; i < w.size(); ++i) {
w[i] = pbW[i];
}
for (DNNFeatureVector::size_type i = 0; i < b.size(); ++i) {
b[i] = pbB[i];
}
layer.set_trainable(false);
}
}
Note that if you don’t call init_weight or set_trainable, weights will be overwritten (i.e. initialized lazily) when the next layer is added or prediction is run.
Recent Comments