Make sure user2 has been added to the same group as user1:
grep user2 /etc/group
Should return something like:
user1:502:user2,user3
Ensure that there are write permissions on the parent directory:
chmod g+w mydir
Finally add a sticky bit to said directory:
chmod g+s mydir
If you want further help, you would need to post the output of ls -lhad mydir for the specific parent directory you are having problems modifying files in and the group information for user1 and user2 in /etc/groups (as posted above). You can also check out this CentOS Guide, it's written for CentOS 5 but is still applicable for most distros. You can also use the following command to add write permissions for all files and directories within a specific parent directory (so mydir/myfile gets group write access as well):
chmod -R g+w mydir