Wow! So much waffle! And this from a guy who's waffled more than anyone else today!
The simple answer to the original question is plainly visible if you use any vi in a TTY: it waits to see if there's a character coming immediately after the escape.
I've repeatedly found that delay to be a nuisance, temporarily confusing and alarming me as the editor gets behind my typing, and I curse the day DEC decided to use the escape character in their key sequences. It throws off my train of thought!
As for why it's such a long delay, I can only imagine it's designed to work with 75 baud serial lines, or maybe laggy ssh sessions.