I do not believe that you can use a different .htpasswd file per project to manage users in a single Trac install (when using Trac's internal web server). I know that it is something that is being considered by the Trac community (see Comprehensive Solution for Multiple Projects for more information).
That being said, you do have the option of running Trac through Apache with Python to host multiple projects which opens the door a bit. See Multiple Projects in Trac through Apache. The short version is that in Apache, you'll do the following:
ScriptAlias /project /path/to/trac.cgi
<Location "/project">
SetEnv TRAC_ENV "/path/to/trac/project"
</Location>
<Location "/project/login">
AuthType basic
AuthName "project - trac"
AuthUserFile "/path/to/svn-auth-file"
Require valid-user
</Location>
If you need to keep Trac as its own webserver, you can definitely set different permissions for different projects even when all Trac projects share the same .htpasswd file. That way a user can be TRAC_ADMIN or WIKI_ADMIN of one Trac project while not having those permissions in a second Trac project. You can do this from the command line using the trac-admin tool. For example:
trac-admin /path/to/projenv permission add bob REPORT_DELETE
Or you can accomplish the same from the GUI (when logged in with a user with TRAC_ADMIN privileges to a specific project, go to Admin->Privileges).
For more on using the command line trac-admin tool for setting permissions (users and groups) see Trac Permissions. A more fine-tuned option for managing permissions using permission policies can be found at Trac Permission Policies.