To explain what is happening with your escaped apostrophes, we'll examine your second example (also see single quotes, or strong quotes):
$ echo '\'Hello World\''
> # expects you to continue input
Here, you've left the quotation hanging, as you've stated. Now trim the end and change it to:
v v v
$ echo '\'Hello World # Echo two strings: '\' and 'Hello World'.
\Hello World ^
The "Hello World" sub-string wasn't quoted here, but it behaved as if it was strong quoted. Using your example again, trim the end differently this time:
vv v (plain apostrophe)
$ echo '\'Hello World\' # Will echo: '\' and 'Hello World''
\Hello World' ^^ # Note that the trailing ' char is backslash escaped.
The "Hello World" sub-string again behaves as if it were strong quoted, with only the added apostrophe (escaped, so no longer a single quote) at the end.
When another single quote is added to the end (your original example) the string is left hanging and waiting for a close-quote.
echo \''Hello World'\'
– math – 2012-11-13T08:11:54.637