Non-Sloppy Focus-Follows-Mouse

I've managed to get focus-follows-mouse working without window-raising how I like it. However, it exhibits "sloppy" behavior by default. As in, it doesn't switch focus to the desktop when I hover over the desktop. All other windows are properly focused when I hover over them. How would I get the same behavior with the desktop?

Thanks.