It seems that the author of a project can upload arbitrary binary files on the release page, and I don't see a way to verify that the binary is actually built from the source code without malicious modifications.
Surprisingly after some search, I did not find discussions on this potential issue. Is it that I've missed something, or people simply just trust those binaries?