Yes, it would make a difference.
USB 2 has a theoretical max of 480 Mbit per second. Actual available bandwidth is about 30-40MB/sec, depending on the chips and the drivers used. Part of this is due to 10% bandwidth reserved for USB1 communication, part of it is because the USB protocol has a lot of overhead.
SATA is less flexible (e.g. you can only connect harddisks, port multipliers and CD/DVD/BR-ROMS), but it has much less overhead. It also works at a much higher speed (SATA-II at 3.0 Gbit per second). The result is a maximum thoughput around 270MB/sec. (Actual test values from a ICH9 and SATA drives on tech sites).
35MB/sec vs 270MB/sec. That makes SATA-II over 7 times as fast on thoughput alone.
And since there is also less overhead the latency will also be lower.
TLDR; Connect it via SATA. Or if you want it externally, get an eSATA connection.
3In practical terms USB 2 is 25mb-30mb/s and SATA 2 is close to 300mb/s. – cybernard – 2013-12-25T07:35:36.053
Related: Using FireWire 800, does it makes sense to use a SSD hard disk as opposed to a 7200rpm?
– gronostaj – 2013-12-28T21:01:23.240