I work in insurance and this question comes up all the time. And whether or not you should keep physical damage coverage on a car is something that in the end only the individual can answer. In other words..it depends. If something happens to total the car out--even if it's "only" a couple of thousand dollars--are you in the position to repair/replace on your own? If you take off the coverage for comprehensive and collision and something happens, it's all on you. A few thousand dollars may not mean much to some. To others trying to come up with that amount would work a real hardship. Keep in mind that whenever a vehicle is totaled the insurance company will offer you the actual cash value of the vehicle at the time of the incident less your deductible. This is assuming you are responsible for repairing your vehicle; if it's the other guy's fault then his insurance should cover you. Unless he isn't insured. And that's why you carry uninsured motorist coverage. Right?
Earlier the question came up about reporting an incident to your insurance company. It's a good idea to do it, even it the fault is clearly with the other party. I've seen it happen more than once that the other party (and the other party's insurance) will drag their feet a little, or try to get off cheaper than they first stated, or both. Sad but true. Your company can go to bat on your behalf if need be. And (last but not least) if you are involved in an accident with someone and the police show up, go ahead and get a copy of the police report even if you're as innocent as the morn. It can be a very handy thing to have.
There--I've said my piece.