When you look at the documentation, you are looking for the terms "character set" and "charset"
By default, the [Incremental:All] parameters are set to use the full
printable US-ASCII character set (95 characters)
The 95 characters do not include umlauts.
But later, the documentation reads:
If you've got a password file for which you already have a lot of
passwords cracked or obtained by other means, and the passwords are
unusual, then you may want to generate a new charset file, based on
character frequencies from that password file only:
john --make-charset=custom.chr mypasswd
What you do is create a .pot file with the characters you want to include, update your .conf file for a new character set file, then run
john --make-charset=custom.chr