"I probably had USB Debugging enabled as I developed an Android app in
the past and tested it on my device. However I remember that you must
trust the computer before using it, so maybe this doesn't make my
device more vulnerable?"
Android has had ADB authorization for some time (since Jelly Bean, RTM July 2012).
Source: New Android 4.2.2 Feature: USB Debug Whitelist Prevents ADB-Savvy Thieves From Stealing Your Data (In Some Situations)
Worried about an ADB-savvy thief stealing the precious data off your
stolen phone or tablet? Well, Android 4.2.2 makes doing that a little
harder now, with the addition of a USB debugging whitelist feature
into the OS. The way it works is pretty simple - when you connect your
PC to your Android device via USB, Android gets your PC's RSA key (an
identifier token). In Android 4.2.2, when you have USB debugging
enabled, this now causes a prompt to appear on connection, seen below.
The prompt asks you to confirm that you wish to allow USB debugging
from the connecting PC, and allows you to add it to a whitelist such
that the prompt won't ask you to confirm for that computer again. So,
if a thief steals your phone (assuming you have a password / gesture /
face unlock set), they won't be able to get all fancy with ADB and
start dumping your personal data onto a hard drive. And that's good.
Here's the message a potential thief would get from ADB upon
attempting to list a connected device from an unauthorized PC
('offline'), and the message after that PC has been authorized
('device').
The security aspect of this feature relies on a few prerequisites,
though, otherwise it kind of loses its usefulness.
First, you obviously need some kind of barrier to entry to the phone:
if the thief gets the prompt, that doesn't do you much good. So, your
lockscreen needs a password, gesture, or face unlock set up, so the
prompt won't appear. That's easy enough, and something many people
already do.
The next part may not exactly appeal to you, the Android-tinkering
enthusiast. If your phone is rooted, has an unlocked bootloader, or is
running custom firmware (eg, a custom recovery like ClockworkMod
Recovery), there's a good chance you've created a potential workaround
for this USB debug whitelist, and that a determined data thief could
still find a way to your precious information. We won't go into
specifics, but suffice to say, if your bootloader isn't locked, the
USB debug whitelist can effectively be nullified.
This is just one of a number of changes in Android 4.2.2
There was a bug that could be exploited in Android versions from 4.2.2 to 4.4.2 to bypass Android’s secure USB debugging, this allowed attackers to access adb prior to unlocking the device. This bug was fixed in Android 4.4.3.