The approach I use combines a bit of Ashish's answer with piec's; I have alt-left and right arrow bound to a quick little shell callout that moves the window one to the left or the right, unless it is the first or last window, respectfully. I did this because, when you issue a swap +1 at the last window (or swap -1 at the first window), it will still swap, instead of looping back around again like you might expect:
0:one 1:two 2:three 3:zero*
0:zero* 1:two 2:three 3:one
0:zero* 1:one 2:two 3:three
So, the commands I use stop working when the window has reached the edge of the list:
bind-key -n M-Left run-shell 'tmux list-windows | head -n 1 | grep -q active || tmux swap-window -t -1'
bind-key -n M-Right run-shell 'tmux list-windows | tail -n 1 | grep -q active || tmux swap-window -t +1'
This can easily be combined with base-index and renumber-windows to have a list of windows that start at an arbitrary number and never has any gaps.
If you are using base-index 1 like me and you don't think you'll ever go above 999 windows, you can use a little trick to make it roll properly, though the commands bloat a bit:
set -g base-index 1
set -g renumber-windows on
bind-key -n M-Left run-shell 'if tmux list-windows | head -n 1 | grep -q active ; then tmux move-window -t 999 \; move-window -r \; refresh-client -S ; else tmux swap-window -t -1 ; fi'
bind-key -n M-Right run-shell 'if tmux list-windows | tail -n 1 | grep -q active ; then tmux move-window -t 0 \; move-window -r \; refresh-client -S ; else tmux swap-window -t +1 ; fi'
This works by temporarily moving the last window to the unused index-0 and then calling move-window -r to renumber them starting from 1 again. It works similarly when moving the first window to the end; by picking a huge number you'll never use, it ensures that when move-window -r fires again everything will be numbered like you'd expect. If you're wondering about refresh-client -S, that's necessary because sometimes, while the reordering from move-window will work properly, the status bar won't update until further changes are made. By forcing a refresh of just the status bar (-S), you avoid this.
The only issue I can find with this approach is that swap-window will implicitly alter the last-used window to the one you swapped with. Thus, if you are on window #1, switch to window four and move it back one, you'll find that your last-used window is the new # 4 (formerly #3) instead of #1. There doesn't seem to be a way around this.