Unauthorised Bluetooth device keeps connecting to Mac

Question

Recently I started experiencing frequent random lags with a Bluetooth mouse I have connected to my iMac: the tracking speed would suddenly drop for a few seconds before returning back to normal.

One time, while it was happening, I quickly clicked the Bluetooth icon in the menubar to see what was happening, only to see some unfamiliar device with some random name presumably connected to my machine. It has no indication of what it is, only an option to Disconnect. My iMac is not set to be "discoverable" and I was never prompted to allow the connection with that device. It also seems to keep changing the name, so it's not always the same, but the symptoms always are.

I installed a bunch of Bluetooth monitoring tools, from the Xcode dev tools page, and I am able to catch the activity of that random device (it keeps connecting every few minutes, but without any apparent pattern), but not identify the nature of that activity or find a way to stop it from happening.

How do I find the culprit (a human hacker or a rogue device somewhere in the building) and make it stop doing what it's doing?

Answer 1

After almost an entire day of experiments and investigation, I think I might have identified the nature of this strange behaviour. The biggest clue (and originally, the most concerning discovery, for obvious reasons) was the relation of the activity of those random devices, with almost 100% certainty, to browser being open.

That's right: the Bluetooth connections weren't happening unless I was actively working in the browser, navigating between pages. This was true for both Safari and Chrome (not Firefox however) but was not the case with Incognito/Private Browsing windows.

Bottom line: it appears that the issue was caused by misbehaving Handoff feature and those random devices trying to connect could have been other people's iOS devices or Macs.

I have no knowledge of how Bluetooth and Bluetooth "sniffing" work, let alone the way Handoff makes use of those, so I can't say what exactly went wrong. By design, Handoff is supposed to only work between the Apple devices that use the same iCloud account, which definitely wasn't the case here.

For the time being, I simply turned Handoff off in the iMac's System Preferences, as well as performed a factory reset of the Bluetooth environment, just in case. I can't yet say if switching Handoff back on after the reset is safe, but will update this answer once I confirm that.

Answer 2

Randomly started having the previous keyboard try to connect to this mac mini. I use the keyboard in another room, and I guess I never cleanly unpaired it. Anyways, Disabling Handoff instantly fixed the issue. Glad I don't need that feature right now.