We would like to have self-hosted slack alternative (mattermost, rocket.chat), but hiding it behind VPN is uncomfortable for some our users.
I would feel more safe if it wasn't public, but how risky it really is?
I believe this question can be applied to all dev applications such as: gitlab, redmine etc.
You can evaluate by taking a look at the update tempo and security-specific release information for each package you are interested in.
The common case for folks who run third party/open-sourced apps publicly without firewall or VPN protection is that they fail to keep them and their dependencies updated and then some time after they fall out of date their systems are compromised.
The time between announcement of security issues and compromise attempts is shortest with something like Wordpress, where attacks can start within minutes.
But all open source apps have security issues that require a steady tempo of updates, and will have attackers looking for signatures that signal vulnerabilities.
Whether this posture is a significant risk for you is highly context-dependent, both on the impact of compromise and exposure, the attraction attackers may have to you, and on the costs- financial, human, operational- of alternative approaches.
Usually folks have information sensitive to the company in chat, and of course dev tools have custody of important assets. But from an attacker's perspective there are a lot of fish in the sea, and many highly vulnerable systems go uncompromised.
In the end, though, this is why folks wind up using SAAS services, unless they have a strategic need to self-host and are able to do so at a more professional level than hobbyist. For some definition of eventually, hobbyist hosting will eventually, inevitably, wind up in compromise.
... hiding it behind VPN is uncomfortable for some our users.
Complex software is always likely to have bugs and vulnerabilities. If your company's information is sensitive, but VPN feels like an overkill, consider running your web services through HTTPS with client certificates (I believe Mattermost and Rocket.chat both work on top of web protocols).
Your users will have the inconvenience to install their certs into their browsers, but you will effectively reduce possible attack vectors to webserver's implementation of asymmetric crypto.
