Disclaimer The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.
Visual Studio 2003 stores Reference paths for projects in the .User file. Not in the csproj file. I can't see why they wanted to store that information in a separate file. The settings pertain only to the specific project. Anyway, if your application's build is dependant upon having the reference paths set, make sure you are adding the .User files to your source control system. The csproj file is not enough.
Normally you can find the .User file in the same directory as the project file... Unless it is a web application/service. Then you will find the file in the VSWebCache folder. Which makes it very difficult to put that file under source control. I don't see a good solution to this problem. Maybe porting the build to use NAnt. Although that would be burdensome if we just wanted to debug the project.
I guess the best solution is to architect your project so that it isn't dependant on a reference path.