Your question whether you should move to Web Apps is mainly opinion-based, but from an objective standpoint: Web Apps and VMs are not interchangeable, nor are they intended to be. Web Apps (part of Azure App Service) are specifically targeted at web/app tier, and offering everything you'd need for that (including scaling, certificate management, etc.). App Service won't support 100% of what you're working with, so you'd either need to stay with what you have or create an environment where you're using both VM and App Service.
Web Apps fully support VNets, so even if you do split your app environment between the two, you can maintain secure communication between the tiers.
Web Apps aren't for simple applications per se. But you are somewhat limited in what you can install. For example, you cannot install MySQL. Nor would you want to, as you would have no control over system resources and attached disks to manage something like that reliable.
You also have specific port access for Web Apps. That is, only ports 80 and 443. Along with web sockets support.
The Web App environment is multi-tenant (another reason why you can't install something heavy like MySQL). Your app tier will run on any number of instances (or be moved to a different VM in the cluster as needed).
Web App environment is Windows-only (which usually doesn't matter when dealing with running code such as node, .net, php, Java, and python (the current list of supported languages).
It's true that Web App service is fully managed, and that you only worry about your code. Along with version control integration, it's near-trivial to update your app (e.g. push new build to github, and have a new version of your app roll out automatically). But given there's usually a need for database and other complex services, this is where you could take advantages of a VM (if an Azure-provided service isn't what you want). You'd still have to manage the VMs you create, but you would be able to offload a subset of your app to Web Apps.
I know you're not using Web / Worker Roles (which fit between Web Apps and VMs), but I wrote up a comparison between the two on StackOverflow here. This gives a bit more insight into how you could use Web Apps.