Your biggest limitation is the number of concurrent network connections that can be made to the client OS (Windows XP). Windows XP will only support a maximum of 10 connections while server will support an unlimited amount.
On Licensing
WinXP
The 10 concurrent connections are included as part of the CAL you receive when purchasing WinXP.
Server
Server (2003/2008) will come standard with 5 user/connection CALS meaning from a legal standpoint, if you will have more than 5 users or connections accessing services served up by the server you will have to buy additional CALS (legal standpoint only, server will still work).
On Remote Desktop
Win XP
Will only support a single session. If someone is logged in on the console, and then you log in using Remote Desktop, you will take over the desktop session.
Server
Will accept up to 2 Remote Desktop connections, and one console session for a total of 3 sessions. You can buy additional Terminal Services licenses to support more users (licenses are tracked and you cannot expand the number of Remote Desktop/Terminal Server users without buy additional licenses).