Mouse click only works once with iterm2/tmux/mosh

Question

I'm trying to set up iTerm2 + mosh + tmux to do my remote work. I can get everything to connect up, but for some reason, the mouse response is funky. If I split my tmux window so I have two panes and click to switch between them, the first time I do this, everything is find. The second time and thereafter, it doesn't work at all. I can't scroll or do anything else mouse related either. This only happens after I click. If all I do is scroll in the two windows, no problem.

The problem doesn't come up if I use SSH instead of mosh, but the whole point of this attempt was to switch over to mosh full-time. The problem also doesn't happen if I don't use tmux and just have a single vim window. I can click around in there all day.

Does anyone know what setting could be wrong or even where to start? It's obviously related to tmux, but I haven't been able to find a setting that could be causing the problem.

tmux.conf

bind -n M-Left select-pane -L
bind -n M-Right select-pane -R
bind -n M-Up select-pane -U
bind -n M-Down select-pane -D

set -g mode-mouse on
set -g mouse-resize-pane on
set -g mouse-select-pane on
set -g mouse-select-window on