6. Your own models

The MuseBox Compiler (mbx-compile) turns your model into a MuseBox package (.mbx) and copies it to the board. After you restart the AI server, it finds the new package, and the model appears in the editor as a node.

Mercury+ PE3 Hailo-8Mercury+ AMD Hailo-8Mercury+ AMD DeepXPluto Hailo-8Pluto DeepX

Your model	Runs on
.hef (compiled with the Hailo Dataflow Compiler)	the Hailo NPU: fastest
.tflite	the board's CPU
.onnx	the board's CPU

To run on the Hailo NPU, a model must first be quantized and compiled into a .hef file with Hailo's tools; mbx-compile does not do this step. Then give the .hef to MuseBox as described in this chapter.

HailoRT 4.19.0

The board runs HailoRT 4.19.0: your .hef must be compiled for HailoRT 4.19.0, that is with the Dataflow Compiler 3.29.0. A .hef compiled for another HailoRT version may not load on the board. The same applies to .hef files from the Hailo Model Zoo: use Model Zoo v2.13, the release made for HailoRT 4.19.0 (newer ones, such as v2.14, are for HailoRT 4.20).

1. Download the Hailo Dataflow Compiler 3.29.0 (part of the Hailo AI Software Suite) from the Hailo Developer Zone (free registration). Its documentation explains how to quantize your model (.onnx or .tflite) with a set of calibration images and compile it for the Hailo-8.
2. Many common models are already compiled for the Hailo-8: download their .hef from the Hailo Model Zoo v2.13 list of Hailo-8 models (the release for HailoRT 4.19.0). The Model Zoo v2.13 also has the scripts to quantize and compile them again with your own training.
3. Package the .hef with mbx-compile (below). The ResNet-50 tutorial shows the whole path with a .hef model.

Coming soon.

Coming soon.

Coming soon.

Coming soon.

Install

Install it in its own Python environment (a venv), so that its packages stay separate from the rest of your system:

On your PC, Linux / macOS:

pc$ mkdir -p ~/musebox
pc$ curl -L https://s3.eu-central-1.wasabisys.com/musebox/SDK-RELEASES/MUSEBOX_COMPILER/musebox-compiler-latest.tar.gz | tar -xz -C ~/musebox
pc$ cd ~/musebox/musebox-compiler
pc$ python3 -m venv .venv
pc$ source .venv/bin/activate
pc$ pip install -e .
pc$ mbx-compile --help

On your PC, Windows 10 / 11 (Command Prompt or PowerShell):

pc$ cd C:\Users\<your-user>\musebox
pc$ curl.exe -LO https://s3.eu-central-1.wasabisys.com/musebox/SDK-RELEASES/MUSEBOX_COMPILER/musebox-compiler-latest.tar.gz
pc$ tar -xzf musebox-compiler-latest.tar.gz
pc$ cd C:\Users\<your-user>\musebox\musebox-compiler
pc$ python -m venv .venv
pc$ .venv\Scripts\activate
pc$ pip install -e .
pc$ mbx-compile --help

All the commands (mbx-compile --help)

usage: mbx-compile [-h] [--version] [--update] [--check-update]
                   {inspect,inspect-cpu,init,verify,package,deploy,pipeline}
                   ...

MuseBox SDK Compiler & Model Packaging Utility (Hailo · TFLite · ONNX)

positional arguments:
  {inspect,inspect-cpu,init,verify,package,deploy,pipeline}
    inspect             Inspect a Hailo HEF model streams
    inspect-cpu         Inspect a TFLite or ONNX CPU model tensors
    init                Initialize a MuseBox Model Package directory from HEF
                        / TFLite / ONNX model
    verify              Validate a model package directory or .mbx file
    package             Compress package directory into .mbx archive
    deploy              Deploy package to local server or remote board
    pipeline            Generate ready-to-run pipeline flow JSON for musebox-
                        client & no-code-flow-editor

options:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --update              install the latest release when newer (needs internet)
                        and exit
  --check-update        only tell whether a newer release exists (needs
                        internet) and exit

On Debian / Ubuntu, if venv says ensurepip is not available, install it once with sudo apt install python3-venv and run the command again.

The download creates ~/musebox/musebox-compiler (C:\Users\<your-user>\musebox\musebox-compiler on Windows), with the examples of the tutorials in examples/.

Recorded on the Mercury+ PE3 Hailo-8 (Mercury+ MP1 SoM)

Activate the venv again (source .venv/bin/activate on Linux / macOS, .venv\Scripts\activate on Windows) every time you open a new terminal to use mbx-compile. The commands of this chapter and of the compiler tutorials assume it is active.

It needs Python 3.8 or newer. ssh and scp must work towards the board (ssh root@<board-ip>). On Windows this needs the .ssh\config file described in chapter 1: mbx-compile runs plain ssh and scp, which do not connect to the board without it.

This also installs the readers for CPU models: onnx for .onnx files and, for .tflite files, ai-edge-litert (Python 3.10 or newer on Linux x86_64 / ARM64, Windows x64 and Apple silicon Macs) or tflite-runtime (Python 3.8 and 3.9 on Linux x86_64 / ARM64). Elsewhere, for example Python 3.8 / 3.9 on Windows or macOS, install TensorFlow in the venv (pip install tensorflow), which mbx-compile also uses; without any of them mbx-compile cannot read .tflite files and shows placeholder sizes marked "source": "fallback". .hef models are read on the board and need nothing extra.

Update

With the venv active and an internet connection, one command installs the latest version of the compiler:

On your PC:

pc$ mbx-compile --update

--check-update only tells you whether a newer version exists, and mbx-compile --version shows the installed one. The update is downloaded and checked before it replaces the installed version; if anything goes wrong, nothing is changed. Your venv and your own files in musebox-compiler stay as they are, and the previous version is kept in musebox-compiler/.musebox-previous/.

From model to pipeline, step by step

The steps below package an object detector, my_detector.hef, with its class names in labels.txt (one name per line). Replace them with your own model and labels file. Run the commands on your PC, with the venv active, from the folder that holds your model. If you do not have a model yet, follow the same steps with the ResNet-50 classifier of the examples (~/musebox/musebox-compiler/examples/resnet_v1_50_hailo): the ResNet-50 tutorial gives every command.

1. Look inside the model

On your PC:

pc$ mbx-compile inspect my_detector.hef --remote root@<board-ip>

inspect is for .hef models: it reads the model on the board, so the board must be on. For a .tflite or .onnx model use inspect-cpu, which reads it on the PC, for example mbx-compile inspect-cpu my_model.tflite.

All the options of inspect and inspect-cpu

usage: mbx-compile inspect [-h] [--remote REMOTE] hef_path

positional arguments:
  hef_path         Path to .hef file

options:
  -h, --help       show this help message and exit
  --remote REMOTE  SSH target of the board (e.g. root@<board-ip>)

usage: mbx-compile inspect-cpu [-h] [--backend {tflite,onnx}] model_path

positional arguments:
  model_path            Path to .tflite or .onnx file

options:
  -h, --help            show this help message and exit
  --backend {tflite,onnx}
                        Backend override (auto-detected from file extension by
                        default)

These commands show the inputs and outputs (names, sizes, types). You need them to choose the input shape and the task.

2. Create the package

On your PC:

pc$ mbx-compile init --model my_detector.hef --name my_detector --task object_detection --input-shape 640,640,3 --labels labels.txt

--task is one of:

• object_detection
• classification
• segmentation
• feature_extraction
• custom

All the options of init

usage: mbx-compile init [-h] --model MODEL --name NAME
                        [--backend {hailo,tflite,onnx}]
                        [--task {object_detection,classification,segmentation,feature_extraction,custom}]
                        [--input-shape INPUT_SHAPE] [--format {RGB,BGR,GRAY}]
                        [--conf-thresh CONF_THRESH] [--nms-thresh NMS_THRESH]
                        [--no-softmax]
                        [--pre-type {standard,letterbox,custom}]
                        [--preprocess PREPROCESS] [--postprocess POSTPROCESS]
                        [--labels LABELS] [--desc DESC]
                        [--output-dir OUTPUT_DIR] [--remote REMOTE]
                        [--num-threads NUM_THREADS] [--providers PROVIDERS]

options:
  -h, --help            show this help message and exit
  --model MODEL         Path to model file (.hef / .tflite / .onnx)
  --name NAME           Unique model name (e.g. my_yolo_model)
  --backend {hailo,tflite,onnx}
                        Backend (auto-detected from file extension by default)
  --task {object_detection,classification,segmentation,feature_extraction,custom}
                        Task type (default: object_detection)
  --input-shape INPUT_SHAPE
                        Input shape as H,W,C or N,H,W,C (e.g. 640,640,3)
  --format {RGB,BGR,GRAY}
                        Input color format
  --conf-thresh CONF_THRESH
                        Confidence threshold
  --nms-thresh NMS_THRESH
                        NMS threshold
  --no-softmax          Disable Softmax calculation in postprocessing (if
                        model already includes Softmax)
  --pre-type {standard,letterbox,custom}
                        Preprocessing strategy (default: standard)
  --preprocess PREPROCESS
                        Path to a custom preprocess.cpp (compiled on the
                        board)
  --postprocess POSTPROCESS
                        Path to a custom postprocess.cpp (compiled on the
                        board)
  --labels LABELS       Path to labels text file
  --desc DESC           Short model description
  --output-dir OUTPUT_DIR
                        Output directory path (default: model name)
  --remote REMOTE       HEF inspection over SSH: SSH target of the board (e.g.
                        root@<board-ip>)
  --num-threads NUM_THREADS
                        CPU threads for TFLite/ONNX (0 = auto)
  --providers PROVIDERS
                        Comma-separated ONNX execution providers (default:
                        CPUExecutionProvider)

This creates the folder my_detector/ with the model, the labels and a manifest.json that describes how to run it. For .tflite / .onnx add --num-threads 4.

Custom models need a custom post-processing

--task chooses the AI server's built-in post-processing, which works only with a few output formats:

• object_detection: the final boxes already computed by the model, one row [ymin, xmin, ymax, xmax, score, class] per object (Hailo models compiled with NMS on the chip);
• classification: one vector with a score per class;
• segmentation: one score per class (or the class) for each pixel; the mask is drawn in the editor and the clients;
• feature_extraction: one vector (embedding), returned as it is;
• custom: no decoding, the raw values of the outputs.

NMS (non-maximum suppression) removes the overlapping boxes of a detector. If mbx-compile inspect shows a single output that already lists the boxes (5 or 6 values each), NMS runs on the chip and the built-in object_detection works; otherwise you need a postprocess.cpp.

For any other model, which is most of your own models (YOLO before NMS, several outputs, depth…), write a postprocess.cpp that turns the output into the JSON you need: Custom pre- and post-processing. The full table is in Tasks.

If your model needs special handling of the image (--preprocess preprocess.cpp) or of its output (--postprocess postprocess.cpp), see Custom pre- and post-processing. The board compiles those files the first time the model runs.

3. Check and package

Both commands take the package folder my_detector/ created in step 2:

On your PC:

pc$ mbx-compile verify my_detector/                         # checks the folder and its manifest.json
pc$ mbx-compile package my_detector/ -o my_detector.mbx     # packs the folder into one .mbx file

All the options of verify and package

usage: mbx-compile verify [-h] target

positional arguments:
  target      Package directory or .mbx file

options:
  -h, --help  show this help message and exit

usage: mbx-compile package [-h] [-o OUTPUT] package_dir

positional arguments:
  package_dir           Directory containing manifest.json and model file

options:
  -h, --help            show this help message and exit
  -o OUTPUT, --output OUTPUT
                        Output .mbx filename

4. Copy it to the board

On your PC:

pc$ mbx-compile deploy my_detector.mbx --remote root@<board-ip> --dest /root/musebox-ai-servers/ai_models

All the options of deploy

usage: mbx-compile deploy [-h] [--dest DEST] [--remote REMOTE] package

positional arguments:
  package          Path to package directory or .mbx file

options:
  -h, --help       show this help message and exit
  --dest DEST      Destination ai_models directory (default: ./ai_models
                   locally, /root/musebox-ai-servers/ai_models on a board)
  --remote REMOTE  SSH target of the board (e.g. root@<board-ip>)

Restart the AI server on the board, so that it loads the new model:

1. stop it: press Ctrl+C in the ssh terminal where the server runs (if you started it in the background, run pkill -f musebox-ai-server on the board instead, pkill -f musebox_ai for the Python server);
2. start it again in the same terminal, with the commands below.

On the board:

board$ cd /root/musebox-ai-servers
board$ ./musebox-ai-server-cpp/musebox-ai-server

http://<board-ip>:8080/models now lists my_detector.

Steps 1 to 4 with the ResNet-50 classifier of the examples (examples/resnet_v1_50_hailo):

Recorded on the Mercury+ PE3 Hailo-8 (Mercury+ MP1 SoM)

5. Use it

• In the editor: with the Configuration node connected to the board, click Sync Models in the toolbar. Your model appears in the menu under MuseBox Tasks → Hailo Models (.hef) or MuseBox Tasks → CPU Models (.tflite, .onnx): add my_detector from there and connect it like any other task. Without Sync Models, use MuseBox Tasks → Custom → Custom Model and pick my_detector with Fetch Models from Server. If you have a custom post-processing (postprocess.cpp), see Advanced: your own nodes in the editor to show its results in the editor.

Recorded on the Mercury+ PE3 Hailo-8 (Mercury+ MP1 SoM)

• Generate the pipeline:

On your PC:

pc$ mbx-compile pipeline my_detector --server ws://<board-ip>:8080 -o my_detector_pipeline.json

Load that file in the editor (Manage → import) or run it with the client (chapter 5).

With --cascade face_detection the faces found by the built-in face detector are sent to your model: for example an emotion or age model.

All the options of pipeline

usage: mbx-compile pipeline [-h] [-o OUTPUT] [--server SERVER]
                            [--cascade CASCADE]
                            package

positional arguments:
  package               Path to model package directory, .mbx file, or
                        manifest.json

options:
  -h, --help            show this help message and exit
  -o OUTPUT, --output OUTPUT
                        Output pipeline .json filename
  --server SERVER       MuseBox server WebSocket URL (default:
                        $MUSEBOX_SERVER_URL or ws://127.0.0.1:8080)
  --cascade CASCADE     Upstream detector to cascade from (e.g.
                        'face_detection')

Complete examples

The package specification lists all the manifest.json fields. These tutorials go from the model file to a running pipeline:

• ResNet-50 classification on the Hailo NPU
• Age and gender on the CPU (TFLite)
• MobileNetV2 classification on the CPU (ONNX)