The easiest way would be deploying Windows 10 LTSB. Enterprise LTSB is a long-term support version of Windows 10 Enterprise released every 2 to 3 years. Basically, this is Windows 10 without the whole app stuff; No Microsoft Store, no Microsoft Edge, no apps at all.
This obviously requires an entirely different ISO. They are supported with security updates for 10 years after their release, and intentionally receive no feature updates.
If you already have a Windows 10 Enterprise E3 (per device) plan via Volume Licensing, you're allowed to download and install Windows 10 Enterprise LTSB.
More details: http://download.microsoft.com/download/2/D/1/2D14FE17-66C2-4D4C-AF73-E122930B60F6/Windows-10-Volume-Licensing-Guide.pdf
If you can't or don't want to change the OS, you should be able to uninstall all apps via PowerShell:
Run as admin:
Get-AppXProvisionedPackage -Online | Remove-AppxProvisionedPackage -Online
Get-AppxPackage -allusers | Remove-AppxPackage
Run as user:
Get-AppxPackage | Remove-AppxPackage
Please keep in mind that those commands will uninstall all apps except Cortana, Edge and Microsoft Store. You'll loose the calculator as well by doing so, just for your information.