Sporadic Bluetooth crash, crashes ORM completely

[JaapvanEkris]

While rowing on EXR, ORM crashed on me mid-session. It was a weird crash as EXR didn't even attempt to reconnect. I found the following in the logs:

Apr 29 18:44:52 roeimachine npm[615]: Uncaught Exception: TypeError: hardwareErrorCallback is not a function
Apr 29 18:44:52 roeimachine npm[615]:     at handleHardwareError (/opt/openrowingmonitor/node_modules/ble-host/lib/internal/adapter.js:844:3)
Apr 29 18:44:52 roeimachine npm[615]:     at HciSocket.onData (/opt/openrowingmonitor/node_modules/ble-host/lib/internal/adapter.js:1182:31)
Apr 29 18:44:52 roeimachine npm[615]:     at HciSocket.emit (node:events:519:28)
Apr 29 18:44:52 roeimachine npm[615]:     at /opt/openrowingmonitor/node_modules/hci-socket/lib/hci-socket.js:105:9
Apr 29 18:44:52 roeimachine npm[685]: 2026-04-29 18:44:52 sigHandler: Unhandled signal 15, terminating
Apr 29 18:44:52 roeimachine npm[615]: OpenRowingMonitor shutdown at 18:44, at 2543.14435 seconds,distance 9172.7m
Apr 29 18:44:52 roeimachine npm[615]: /opt/openrowingmonitor/node_modules/ble-host/lib/internal/adapter.js:844
Apr 29 18:44:52 roeimachine npm[615]:                 hardwareErrorCallback(hardwareCode);
Apr 29 18:44:52 roeimachine npm[615]:                 ^
Apr 29 18:44:52 roeimachine npm[615]: TypeError: hardwareErrorCallback is not a function
Apr 29 18:44:52 roeimachine npm[615]:     at handleHardwareError (/opt/openrowingmonitor/node_modules/ble-host/lib/internal/adapter.js:844:3)
Apr 29 18:44:52 roeimachine npm[615]:     at HciSocket.onData (/opt/openrowingmonitor/node_modules/ble-host/lib/internal/adapter.js:1182:31)
Apr 29 18:44:52 roeimachine npm[615]:     at HciSocket.emit (node:events:519:28)
Apr 29 18:44:52 roeimachine npm[615]:     at /opt/openrowingmonitor/node_modules/hci-socket/lib/hci-socket.js:105:9

Something to handle on ble-host's side, and perhaps on our side to prevent it crashing the entire app?

[Abasz]

tThis is something new and I am not sure what caused it...

Of course we can and should defend against it but I think this is an upstream bug... I have looked at the code and while the stack suggests the issue is that on initialisation of the HciAdapter no one is passing a hardwareErrorCallback (and you had a hardware error) and it was tried to be called, but actually to me it seems that the code it-self is handling this correctly upstream.

I will have to trace through this in-depth.

The quick fix is handling the crash of the hci device but the problem with that is that I have no idea what happens to the hci device in such case (i.e. whether we can recover by reinitialise or we need to shut down orm gracefully). I have to think this through.

Did you have to reboot rpi to have this working again or you just restarted ORM?

[JaapvanEkris]

tThis is something new and I am not sure what caused it...

I've seen it once before, on 0.9.6, but that was about half a year ago (please note, I row about 10K a day, so it is quite some hours making this truly exotic). Somehow the BLE connection drops but I think it stays up on the OS level. EXR doesn't consider it a dropped reconnection, and after the crash, EXR crashes again as bluetooth is occupied/busy.

Of course we can and should defend against it but I think this is an upstream bug... I have looked at the code and while the stack suggests the issue is that on initialisation of the HciAdapter no one is passing a hardwareErrorCallback (and you had a hardware error) and it was tried to be called, but actually to me it seems that the code it-self is handling this correctly upstream.

It was a nasty crash. On my production machine crashes are extremely rare, and even when they happen the recorders still dump their status. This was such a brutal crash they din't save anything at all. Unexpected to be honest.

The quick fix is handling the crash of the hci device but the problem with that is that I have no idea what happens to the hci device in such case (i.e. whether we can recover by reinitialise or we need to shut down orm gracefully). I have to think this through.

I think allowing a reboot with HCI not available would perhaps help (as the monitor comes back up), as it would keep users inside the kiosk, allowing to complete distance, trigger a full reboot etc..

Did you have to reboot rpi to have this working again or you just restarted ORM?

In essence, yes.

[Abasz]

I have a high suspicion that this is an upstream bug:

I beleive the root cause is a missing parameter in the module export at the bottom of lib/internal/adapter.js. The hardwareErrorCallback argument is accepted by HciAdapter internally but is never forwarded from the exported factory function.

As a result, hardwareErrorCallback is always undefined inside HciAdapter. When handleHardwareError is triggered by the controller, it calls hardwareErrorCallback(hardwareCode) which throws TypeError: hardwareErrorCallback is not a function, crashing the Node.js process.

I filed a bug report, but since we are using my fork I will create a fix for this and a PR.

I will also think through how we could defend against on our side to be on the safe side.

[Abasz]

I can cleanly reproduce this error now (I mean cleanly that the error is unmistakably what I have described above). I also fixed the upstream bug in my local repo, I will push the changes shortly.

But since such hardware error also thorws we need to handle the throwing in our code. I will try to attempt a clean restart of the full Ble and we shall see what happens (i.e. does that work or we need more on the hardware side like full restart of the Rpi). Either way rowing can continue as this crash will be isolated to the Ble peripheral even if it cannot be restarted from the program.