This depends on which side of the router they reside. If they are both on the LAN side (as you seem to suggest), then all the 'router' is doing is switching packets. This means the TTL would remain the same; 64.
If the other machine was on a different network, say the WAN side of the router; then the TTL would decrease by one. Each time a router makes a decision on which network to route a packet to, it decreases the packet's TTL by one. Similarly for packets generated in response (which should again start with a TTL of 64).
I'm not an expert, but I think switch in a single subnet should know all hosts and how to reach them. TTLs are useful when the route from sender to target is not known ahead of time: This stops packets being routed in circles or bouncing back and fore between two routers. Eventually broken paths will be recognised and fixed via other mechanisms.
If they're on the same network, why are they connected through a router? (Or by "router" do you mean a router that is also a switch and the two computers are actually connected by the switch portion?) – David Schwartz – 2013-10-22T06:27:44.650