Every time Marcus took his shirt off, someone would say "There goes the shirt!" This started because Marcus took his shirt off when doing almost any physical activity. In fact, he probably has it off 10% of the time in the AML videos. Whether he was jumping on the trampoline, wrestling, skateboarding, or working outside, his shirt was off.
Adam noticed this trend, so every time Marcus took his shirt off, he'd say the catchphrase.