A webserver could use the contents of the User-Agent header for logging purposes, to control access ("deny all bots") or return alternative responses ("mobile-friendly pages"). The validity of a header value depends on the application, it is not possible to write on rule that works with everything.
Consider the Cookie header. One site could purely use it to store a session identifier such as sessionid=01234567890abcdef. A different site might choose to store locales, such as lang=nl-NL. You could apply a character blacklist, but then someone might exploit a bug in your application which results in lang=--help being treated as an command-line option.
For CGI programs, web browsers will usually invoke the script and pass headers via environment variables such that the script can use this information as well. This is not limited to the User-Agent header, the values of Cookie might be more interesting for the script. The "form input" as you describe is different from the headers. Headers come before the message body.
Now about the Bash bug that leads to the "Shellshock" security issue, all you need is control over at least one environment variable. As you can see, direct CGI access on a webserver is one example where that condition is satisfied. You could also be affected when the program (PHP, Cgit, ...) invokes the shell, directly (via a shell script) or indirectly (via a shell command).
