Tutorial: ResNet-50 on the Hailo NPU¶

This tutorial takes a ResNet-50 image classifier, already compiled for the Hailo NPU (resnet_v1_50.hef), and runs it on the board: from the .hef file to a live pipeline in the No-Code Flow Editor and in the MuseBox Client. The model recognises 1000 kinds of objects (the ImageNet classes).

The example .hef is compiled for HailoRT 4.19.0, the version on the board. Your own .hef files must be too (chapter 6).

You need:

the MuseBox Compiler installed on your PC (chapter 6);
the AI server running on the board (chapter 3), and ssh root@<board-ip> working from the PC;
the editor (chapter 4) or the client (chapter 5).

The files of this example are in musebox-compiler/examples/resnet_v1_50_hailo/:

File	What it is
`resnet_v1_50.hef`	the model, compiled for the Hailo NPU
`labels.txt`	the 1000 class names, one per line, in the order of the model's output
`preprocess.cpp`, `postprocess.cpp`	optional custom processing, used in the last part
`manifest.json`, `resnet_v1_50.mbx`, `resnet_v1_50_pipeline.json`	the results of this tutorial, for comparison

Run all the PC commands from the compiler folder, with its venv active (in every new terminal):

On your PC, Linux / macOS:

pc$ cd ~/musebox/musebox-compiler
pc$ source .venv/bin/activate

On your PC, Windows 10 / 11 (Command Prompt or PowerShell):

pc$ cd C:\Users\<your-user>\musebox\musebox-compiler
pc$ .venv\Scripts\activate

1. Look inside the model¶

On your PC:

pc$ mbx-compile inspect examples/resnet_v1_50_hailo/resnet_v1_50.hef --remote root@<board-ip>

The .hef is read on the board, so --remote is needed. The answer:

{
  "inputs": [
    {"name": "resnet_v1_50/input_layer1", "shape": [224, 224, 3], "data_type": "UINT8"}
  ],
  "outputs": [
    {"name": "resnet_v1_50/softmax1", "shape": [1000], "data_type": "UINT8"}
  ],
  "source": "remote_hailortcli (root@<board-ip>)"
}

This tells you that:

the model takes a 224 × 224 colour image, 8 bits per channel;
it returns 1000 values, one per class;
the last layer is a softmax: the values are already probabilities.

Warning

If source is "fallback", the board could not be reached and the values shown are placeholders. Check ssh root@<board-ip> and try again.

2. Create the package¶

On your PC:

pc$ mbx-compile init --model examples/resnet_v1_50_hailo/resnet_v1_50.hef --remote root@<board-ip> --name resnet_v1_50 --task classification --labels examples/resnet_v1_50_hailo/labels.txt

With --remote, init reads the input and output of the model from the board and fills in the manifest for you. It must print Auto-detected pre-softmaxed output tensor .... If it does not, the board was not reached and init filled the manifest with placeholder values without any error (verify passes too): check that ssh root@<board-ip> works and run init again. This creates the folder resnet_v1_50/:

resnet_v1_50/
├── manifest.json
├── resnet_v1_50.hef
└── labels.txt

The important part of manifest.json:

{
  "format_version": "1.0",
  "model_name": "resnet_v1_50",
  "task_type": "classification",
  "backend": "hailo",
  "hef_file": "resnet_v1_50.hef",
  "input": {
    "shape": [224, 224, 3],
    "format": "RGB"
  },
  "preprocessing": {
    "type": "standard"
  },
  "postprocessing": {
    "type": "classification",
    "top_k": 5,
    "apply_softmax": false,
    "labels_file": "labels.txt"
  }
}

"type": "standard": the AI server resizes each frame to 224 × 224 and converts it to RGB.
"apply_softmax": false: init saw that the output layer is called softmax1 and turned off the softmax, since the model already applies it. For a model that ends with raw scores, init sets it to true; you can force false with --no-softmax.
"top_k": 5: the AI server returns the 5 most likely classes, with their names from labels.txt.

Without access to the board, give the input yourself: --input-shape 224,224,3 --format RGB --no-softmax instead of --remote root@<board-ip>.

Every field is described in the package specification.

3. Check and package¶

On your PC:

pc$ mbx-compile verify resnet_v1_50/
pc$ mbx-compile package resnet_v1_50/ -o resnet_v1_50.mbx

verify prints:

[PASSED] Package 'resnet_v1_50/' is valid and ready for deployment!

4. Copy it to the board¶

On your PC:

pc$ mbx-compile deploy resnet_v1_50.mbx --remote root@<board-ip> --dest /root/musebox-ai-servers/ai_models

The package is unpacked into /root/musebox-ai-servers/ai_models/resnet_v1_50/ on the board. Restart the AI server so that it finds the new model. If the server runs in an ssh terminal, press Ctrl+C in that terminal; if you started it in the background (chapter 3), run pkill -f musebox-ai-server on the board (pkill -f musebox_ai for the Python server). Then start it again with the commands below. Keep running the pc$ commands in a separate terminal on your PC.

On the board:

board$ cd /root/musebox-ai-servers
board$ ./musebox-ai-server-cpp/musebox-ai-server

When the server prints ... task(s) ready., it has loaded the models. Open http://<board-ip>:8080/models in the browser: it now lists resnet_v1_50.

Steps 1 to 4 on the board:

Recorded on the Mercury+ PE3 Hailo-8 (Mercury+ MP1 SoM)

5. Run it in the No-Code Flow Editor¶

Build the pipeline by hand, as in chapter 4:

In the configuration node already on the canvas (add it from MuseBox Configuration → configuration if it is missing): URL ws://<board-ip>:8080, Board Mercury+ PE3 Hailo, then Connect / Reconnect.
MuseBox Input → PC webcam.
MuseBox Operators → Scale: width 224, height 224. Connect the webcam to it. The frames sent to the board are smaller, so the pipeline is faster.
Click Sync Models in the toolbar, then add MuseBox Tasks → Hailo Models → resnet_v1_50. Connect the output of Scale to its frame / bounding box input.
MuseBox Output → Image Rendering: connect the post process output of resnet_v1_50 to it.
Press Play.

The Image Rendering node shows the webcam with the three most likely classes and their probabilities. For the top five, connect predictions / data to MuseBox Output → Top 5. The predictions / data output carries the same results, for other nodes.

Without Sync Models, use MuseBox Tasks → Custom → Custom Model in step 4: click Fetch Models from Server and choose resnet_v1_50, or type resnet_v1_50 in Model Name / Topic and pick classification as Task Type.

Recorded on the Mercury+ PE3 Hailo-8 (Mercury+ MP1 SoM)

Or let the compiler build the pipeline for you:

On your PC:

pc$ mbx-compile pipeline resnet_v1_50 --server ws://<board-ip>:8080 -o resnet_v1_50_pipeline.json

It writes the same pipeline, with a Custom Model node for resnet_v1_50, so it needs no Sync Models. Load it in the editor with Manage → import.

6. Run it with the MuseBox Client¶

The same file runs without the browser (chapter 5):

On your PC, Linux / macOS:

pc$ cd ~/musebox/musebox-client/python
pc$ source .venv/bin/activate
pc$ python main.py --json ~/musebox/musebox-compiler/resnet_v1_50_pipeline.json --url ws://<board-ip>:8080 --source 0

On your PC, Windows 10 / 11 (Command Prompt or PowerShell):

pc$ cd C:\Users\<your-user>\musebox\musebox-client\python
pc$ .venv\Scripts\activate
pc$ python main.py --json C:\Users\<your-user>\musebox\musebox-compiler\resnet_v1_50_pipeline.json --url ws://<board-ip>:8080 --source 0

--source 0 is the PC webcam; give a video or image file to classify that instead.

What the board returns¶

For each frame, data in the answer holds the 5 most likely classes, best first:

[
  {"label": "tabby, tabby cat", "class_id": 281, "confidence": 0.8125},
  {"label": "Egyptian cat", "class_id": 285, "confidence": 0.1016},
  ...
]

class_id is the line number in labels.txt, counting from 0. The rest of the answer is described in chapter 5.

Optional: your own pre- and post-processing¶

The standard processing above is enough for ResNet-50. When a model needs something different, write it in C++: the example folder has one file for each side, and the board compiles them the first time the model is loaded. The full guide is Custom pre- and post-processing.

preprocess.cpp crops the centre of the frame instead of stretching it, so a wide webcam image keeps its proportions:

extern "C" cv::Mat preprocess(
    const cv::Mat& frame,
    int target_w,
    int target_h,
    const std::string& color_fmt,
    const nlohmann::json& manifest,
    PreprocessMeta& meta
) {
    const int frame_w = frame.cols;
    const int frame_h = frame.rows;

    // Custom center crop preserving aspect ratio on shortest dimension
    float scale = std::max(static_cast<float>(target_w) / frame_w, static_cast<float>(target_h) / frame_h);
    int new_w = static_cast<int>(std::round(frame_w * scale));
    int new_h = static_cast<int>(std::round(frame_h * scale));

    cv::Mat resized;
    cv::resize(frame, resized, cv::Size(new_w, new_h));

    int x = (new_w - target_w) / 2;
    int y = (new_h - target_h) / 2;
    cv::Mat crop = resized(cv::Rect(x, y, target_w, target_h)).clone();

    if (color_fmt == "RGB") {
        cv::cvtColor(crop, crop, cv::COLOR_BGR2RGB);
    }
    return crop;
}

postprocess.cpp converts the 1000 output values to probabilities (it applies a softmax only if apply_softmax is true in the manifest), picks the best classes and names them from labels.txt:

extern "C" nlohmann::json postprocess(
    const std::map<std::string, RawOutput>& raw_outputs,
    const nlohmann::json& request_msg,
    const cv::Mat& frame,
    const PreprocessMeta& meta,
    const nlohmann::json& manifest
) {
    // ...
    const auto& first = raw_outputs.begin()->second;
    std::vector<float> logits;
    musebox::dequantize(first, logits);
    // ... softmax if needed, then sort ...
    return {
        {"task", "classification"},
        {"custom_cpp", true},
        {"predictions", predictions}
    };
}

Note two differences from the standard processing:

the answer's data is the object returned above; the classes are in its predictions list. The editor and the client show both forms;
the number of classes comes from top_k in the request (5 if the request does not set it), not from the manifest.

To use the two files, create the package again with them, then package and deploy as before:

On your PC:

pc$ mbx-compile init --model examples/resnet_v1_50_hailo/resnet_v1_50.hef --remote root@<board-ip> --name resnet_v1_50 --task classification --labels examples/resnet_v1_50_hailo/labels.txt --preprocess examples/resnet_v1_50_hailo/preprocess.cpp --postprocess examples/resnet_v1_50_hailo/postprocess.cpp
pc$ mbx-compile verify resnet_v1_50/
pc$ mbx-compile package resnet_v1_50/ -o resnet_v1_50.mbx
pc$ mbx-compile deploy resnet_v1_50.mbx --remote root@<board-ip> --dest /root/musebox-ai-servers/ai_models

mbx-compile package adds the header musebox_plugin.hpp that the two files include. When you restart the AI server, the log shows the compilation on the board:

[PackageLoader] Auto-compiling native C++ plugin: .../ai_models/resnet_v1_50/preprocess.cpp -> .../ai_models/resnet_v1_50/preprocess.so
[PackageLoader] Successfully compiled: .../ai_models/resnet_v1_50/preprocess.so

and the same for postprocess.cpp. If the compilation fails, the log shows the first compiler errors and the model runs with the standard processing. The files are compiled again only when they change.

All the commands¶

On your PC, Linux / macOS (the board$ line runs on the board):

pc$ cd ~/musebox/musebox-compiler
pc$ source .venv/bin/activate
pc$ mbx-compile inspect examples/resnet_v1_50_hailo/resnet_v1_50.hef --remote root@<board-ip>
pc$ mbx-compile init --model examples/resnet_v1_50_hailo/resnet_v1_50.hef --remote root@<board-ip> --name resnet_v1_50 --task classification --labels examples/resnet_v1_50_hailo/labels.txt
pc$ mbx-compile verify resnet_v1_50/
pc$ mbx-compile package resnet_v1_50/ -o resnet_v1_50.mbx
pc$ mbx-compile deploy resnet_v1_50.mbx --remote root@<board-ip> --dest /root/musebox-ai-servers/ai_models
board$ cd /root/musebox-ai-servers && ./musebox-ai-server-cpp/musebox-ai-server     # restart the server
pc$ mbx-compile pipeline resnet_v1_50 --server ws://<board-ip>:8080 -o resnet_v1_50_pipeline.json
pc$ cd ~/musebox/musebox-client/python
pc$ source .venv/bin/activate
pc$ python main.py --json ~/musebox/musebox-compiler/resnet_v1_50_pipeline.json --url ws://<board-ip>:8080 --source 0

On your PC, Windows 10 / 11 (Command Prompt or PowerShell; the board$ line runs on the board):

pc$ cd C:\Users\<your-user>\musebox\musebox-compiler
pc$ .venv\Scripts\activate
pc$ mbx-compile inspect examples/resnet_v1_50_hailo/resnet_v1_50.hef --remote root@<board-ip>
pc$ mbx-compile init --model examples/resnet_v1_50_hailo/resnet_v1_50.hef --remote root@<board-ip> --name resnet_v1_50 --task classification --labels examples/resnet_v1_50_hailo/labels.txt
pc$ mbx-compile verify resnet_v1_50/
pc$ mbx-compile package resnet_v1_50/ -o resnet_v1_50.mbx
pc$ mbx-compile deploy resnet_v1_50.mbx --remote root@<board-ip> --dest /root/musebox-ai-servers/ai_models
board$ cd /root/musebox-ai-servers && ./musebox-ai-server-cpp/musebox-ai-server     # restart the server
pc$ mbx-compile pipeline resnet_v1_50 --server ws://<board-ip>:8080 -o resnet_v1_50_pipeline.json
pc$ cd C:\Users\<your-user>\musebox\musebox-client\python
pc$ .venv\Scripts\activate
pc$ python main.py --json C:\Users\<your-user>\musebox\musebox-compiler\resnet_v1_50_pipeline.json --url ws://<board-ip>:8080 --source 0