3. Starting the AI server¶

The AI server runs the models on the board: the No-Code Flow Editor and the clients send it images and receive the results. It must be running while you use them.

Two servers: C++ and Python¶

The board has two versions of the AI server, in /root/musebox-ai-servers:

Server	Where	Start it with	Use it when
C++	`musebox-ai-server-cpp/musebox-ai-server`	`./musebox-ai-server-cpp/musebox-ai-server`	you use the editor or the clients: this is the default
Python	`musebox-ai-server-python/` (package `musebox_ai`)	`python3 -m musebox_ai`	you also want to call the models from your own Python script on the board

Both use the same AI engine, so they behave the same way: same options, same models (ai_models/), same settings (config/), same license and the same ports. The Python server adds a Python interface to the engine (see Using the models from Python).

Run only one of the two at a time: they use the same ports and the same AI accelerator.

Start the server¶

C++ serverPython server

On the board:

board$ cd /root/musebox-ai-servers
board$ ./musebox-ai-server-cpp/musebox-ai-server

On the board:

board$ cd /root/musebox-ai-servers/musebox-ai-server-python
board$ python3 -m musebox_ai

All the options (--help)

Both servers have the same options: ./musebox-ai-server-cpp/musebox-ai-server --help and python3 -m musebox_ai --help print:

MuseBox AI Server v1.9.0
Usage: musebox-ai-server [options]
  --models-dir <dir>      shared ai_models directory (env MUSEBOX_MODELS_DIR)
  --config-dir <dir>      shared config directory with *.json (env MUSEBOX_CONFIG_DIR)
  --config <file>         legacy single config.json merged on top of the config directory
  --host/--port           WebSocket bind address (default 0.0.0.0:8080)
  --zmq-host/--zmq-port   ZeroMQ bind address (default 0.0.0.0:5555), --no-zmq to disable
  --lazy-load             load models on first request
  --camera <device>       board camera for frame_id / source=camera requests (/dev/videoN, index or video file)
  --no-camera             disable the board camera
  --license <file>        license file (env MUSEBOX_LICENSE; default <config-dir>/license.lic, /etc/musebox/license.lic)
  --activate              send the activation request (asks name, company and email the first time) and exit
  --update                install the latest release when newer (needs internet) and exit
  --check-update          only tell whether a newer release exists and exit
  --mqtt / --no-mqtt      MQTT bridge (plus --mqtt-host, --mqtt-port, --mqtt-username, ...)
  --rabbitmq / --no-rabbitmq  RabbitMQ worker (plus --rabbitmq-host, --rabbitmq-port, ...)
  --ros2 / --no-ros2      ROS 2 bridge (plus --ros2-node-name, --ros2-qos, --ros2-image-topic, ...)

The server loads the models and waits for requests. It is ready when it prints WebSocket server listening on ws://0.0.0.0:8080. Leave this terminal open: the server runs as long as it stays open (see below to keep it running in the background). Run the pc$ commands of the next chapters in another terminal on your PC. It listens on:

Port	Protocol	Used by
`8080`	WebSocket and HTTP	No-Code Flow Editor, clients (default)
`5555`	ZeroMQ	clients with `--zmq`

MQTT, RabbitMQ and ROS 2 are off by default (--mqtt, --rabbitmq, --ros2).

If the server stops and asks you to activate the board, the board has no valid license yet: go back to chapter 2.

Check from the PC that the server answers. Open http://<board-ip>:8080/models in the browser: it shows, in JSON, the AI tasks the board offers ("models": [...]).

To stop the server, press Ctrl+C in the terminal where it runs (the ssh terminal connected to the board, where you started it): after a few seconds the board$ prompt comes back. To keep it running after you close ssh, start it inside tmux or screen, or in the background:

C++ serverPython server

On the board:

board$ cd /root/musebox-ai-servers
board$ nohup ./musebox-ai-server-cpp/musebox-ai-server > server.log 2>&1 &

On the board:

board$ cd /root/musebox-ai-servers/musebox-ai-server-python
board$ nohup python3 -m musebox_ai > server.log 2>&1 &

The server's messages then go to server.log in that folder. To follow them, on the board run tail -f server.log; press Ctrl+C to stop following them: the server keeps running.

Ctrl+C does not stop a server started this way. To stop it:

On the board:

board$ pkill -f musebox-ai-server     # C++ server
board$ pkill -f musebox_ai            # Python server

Using the models from Python¶

With the Python server you can also skip the network and call the models directly from a Python script on the board:

import cv2, musebox_ai

engine = musebox_ai.Engine()                  # loads the settings, checks the license, loads the models
print(engine.tasks())                         # ['FaceDetection', 'ObjectDetection', ...]

frame = cv2.imread("photo.jpg")
reply = engine.run("ObjectDetection", frame)  # the same reply the server sends to the clients
for obj in reply["data"]:
    print(obj["boundingBox"]["label"], obj["confidence"])

musebox_ai.Server(engine, port=8080).run()    # optional: serve the editor and the clients too

Run the script from /root/musebox-ai-servers/musebox-ai-server-python, or add that folder to PYTHONPATH. Only one engine can run at a time on the board, so do not start a server alongside it: use musebox_ai.Server in the script, as in the last line, if the editor or the clients need it too.

Built-in AI tasks¶

Task	What it does
Face Detection	finds the faces (box, confidence, 5 key points)
Face Landmarking	3D landmarks of each face
Face Recognition / Add Face	recognises the people you registered
Object Detection	finds 80 kinds of objects (COCO)
Mono Depth	depth map from one camera
Classification	what the image shows (ImageNet, 1000 classes)
Segmentation	class of each pixel (PASCAL VOC)

Board camera¶

To use the camera plugged into the board instead of the PC webcam, start the server with --camera /dev/video0, or enable it once for all in /root/musebox-ai-servers/config/camera.json ("enabled": true, "device": "/dev/video0"; edit it on the board, for example with vi). http://<board-ip>:8080/camera/frame.jpg shows the latest frame.

Recorded on the Mercury+ PE3 Hailo-8 (Mercury+ MP1 SoM)

The images then never leave the board, so this is the fastest way to run a pipeline.

Settings¶

The files in config/ are read at start-up (restart the server after a change). Both servers use them:

File	Content
`models.json`	which model each task uses, thresholds, board name
`camera.json`	board camera
`license.json`	license activation (chapter 2)
`mqtt.json`, `rabbitmq.json`, `ros2.json`	the other protocols

Update the server¶

With the board connected to the internet, one command installs the latest version:

C++ serverPython server

On the board:

board$ cd /root/musebox-ai-servers
board$ ./musebox-ai-server-cpp/musebox-ai-server --update

On the board:

board$ cd /root/musebox-ai-servers/musebox-ai-server-python
board$ python3 -m musebox_ai --update

--check-update only tells you whether a newer version exists. The update is downloaded, checked and tried on the board before it replaces the installed version; if anything goes wrong, nothing is changed. Your settings, models and license stay as they are. The previous version is kept: musebox-ai-server.previous next to the C++ program, .musebox-previous/ in the Python folder.

A server that is running keeps the old version: stop it and start it again.

Next: 4. The No-Code Flow Editor