Speaking as a vendor who provides a popular PDF Converter for SharePoint that requires Office to be installed on the server for some formats I'd like to add my 2 cents.
In the past when I was in charge of change control for a large financial institution I would have raised my eyebrows if any vendor required office to be installed on the server. Oh how I loved my fancy Ivory Tower.... the irony is not completely lost on me.
Fact is that sometimes we need to look at what the business requires, and if the business requires perfect fidelity when converting documents that use the latest and greatest office formats then you can't escape going down the route of installing Office on the server.
Now, plenty of people have tried and failed to make this stable, and sometimes even employing dedicated people that go around rebooting hung servers. The thing is, if you know what you are doing and deal with all the pitfalls, of which there are many, then it is actually possible to successfully and reliably use Office on the server in a way that scales well.
I don't want to make this into an elaborate sales pitch, but we have hundreds of (high profile) customers and I have yet to hear from a customer who had a crash, outage or any kind of downtime related to our software.
If you don't know what you are doing and just code up some COM automation with Office directly in your SharePoint app then you are bound to run into problems, but if you do your job well, run everything in a separate process, allow optional offloading to non SharePoint servers then there is nothing wrong from a technical perspective with running office on the Server.