I don't have much experience in computer security and I am developing a server for a project. I'm using a tcp connection and UDP packets for some real time data transmission. I want both of these channels encrypted using a self signed cert. What I have planned so far is.
- Multiple servers are started for redundancy.
- Each server generate a self signed cert when starting up and registers the public key in a service discovery agent (ETCD maybe).
- There is an API( which is https, using a cert signed by a valid CA). Lets call this management API
- Clients first query management API, which responds with an IP:Port and the public key of the desired server to connect.
- Clients connect(TCP TLS) to the server IP with the public key as CA (which validates the TLS connection).
- The server then generates a new AES GCM session, generates a 256 bit key and sends the key through the TCP TLS connection.
- Each UDP packet to and from the server is encrypted using this key (using AES GCM)
- Client uses the initialized tcp connection and UDP channel to communicate with the server.
Questions:
- Is this approach secure ?
- What are security vulnerabilities of this plan ?
- If there are vulnerabilities, how can I fix them?
- is AES GCM mode is a good fit for UDP packet encryption?
Notes :
- Server is written using java (Netty), client is cpp(boost asio).
- The UDP packets contain time sensitive data, therefore low latency is a priority.
- I have looked into two encryption algorithms (AES, Salsa). planning to use AES for UDP encryption because running "gnutls-cli" benchmarks in EC2 reveals AES is way faster on modern CPUs (source).
- UDP data is VOIP data.