For a recent problem I had to crawl and categorize
a large number of audio files from the internet.
As the bottleneck was deemed to be downstream link
I decided to use a pre-trained event
classification model and run inference on the CPU.
Another requirement was that the crawler were
running on multiple server and consistent for
several weeks. My choice of language for such
infrastructure heavy task is Rust and I looked
into ways how to deploy an inference model.
It turned out that the process for creating a
standalone binary in Rust is pretty simple. Even
better, wrapping a model pre-trained on a large
event classification dataset is possible.
This means that in the end I could just copy the
binary to any x86 machines and run them, without
the need to setup virtual environments or install
any packages.
In this post I will demonstrate how to convert a
pre-trained model to ONNX and put it into a
binary. Our goal is to create a standalone
executable taking samples from stdin and
classifying them into 527 known audio classes.