This is because DES-based crypt (AKA 'descrypt') truncates passwords at 8 bytes, and only checks the first 8 for the purpose of password verification.
That's the answer to your direct question, but here's some general advice implied by your context:
Fortunately, from my reading, MD5
in /etc/login.defs
is actually md5crypt ($1$), which, while a little outdated and declared deprecated by its author, is still far superior to DES-based crypt (and definitely much better than a raw, unsalted hash like plain MD5! Most unsalted hashes can be cracked on commodity GPUs at rates of billions per second)
It looks like SHA256
(actually sha256crypt) and SHA512
(actually sha512crypt) are also there. I would pick one of those instead.
If you set your password to password
or something under each scheme, you can visually verify whether or not my conclusion that they're the -crypt variants is correct (examples here are taken from the hashcat example hashes, all 'hashcat', some wrapped for readability):
Not recommended - unsalted or legacy hash types, much too "fast" (cracking rates) for password storage:
MD5 - 8743b52063cd84097a65d1633f5c74f5
SHA256 - 127e6fbfe24a750e72930c220a8e138275656b8e5d8f48a98c3c92df2caba935
SHA512 - 82a9dda829eb7f8ffe9fbe49e45d47d2dad9664fbb7adf72492e3c81ebd3e2 \
9134d9bc12212bf83c6840f10e8246b9db54a4859b7ccd0123d86e5872c1e5082f
descrypt - 48c/R8JAv757A
OK - much better than unsalted, no truncation, but no longer sufficiently resistant to brute force on modern hardware:
md5crypt - $1$28772684$iEwNOgGugqO9.bIz5sk8k/
Better - relatively modern hashes with large salts and work factors:
sha256crypt - $5$rounds=5000$GX7BopJZJxPc/KEK$le16UF8I2Anb.rOrn22AUPWvzUETDGefUmAV8AZkGcD
sha512crypt - $6$52450745$k5ka2p8bFuSmoVT1tzOyyuaREkkKBcCNqoDKzYiJL9RaE8yMnPgh2XzzF0NDrUhgrcLwg78xs1w5pJiypEdFX/
Of these, only descrypt truncates at 8. The last two are your best bet.
(Side note: the digits-only salts in the md5crypt and sha512crypt examples above are just side effects of how hashcat creates example hashes; real, healthy salts are usually drawn from a much larger keyspace).
Note also that I'm only listing the hash types that are supported by /etc/login.defs on this platform. For general use, even sha256crypt and sha512crypt have been superseded - first by bcrypt, and then later by truly parallel-attack-resistant hashes like scrypt and the Argon2 family. (Note, however, that for interactive logins that should complete in under one second, bcrypt is actually more resistant to attack than the latter)