If you have a local Columbia or a Tomberlin dealer, they can order any part you may need. If you're in the boonies or a DIY guy who likes to tinker, you can order online from several sites. You don't say which parts you are looking for but for older vehicles, you could try www.vintagegolfcartparts.com or www.kraftimplement.com . If you have questions on the older units, either of those two sites are super helpful and easy to talk to. For newer vehicles you could try www.cartpros.com or www.columbia.parts .
For the cost question, most dealers go off recommended list pricing from the manufacturer (although it isn't required), but sometimes you can get a deal. Right now www.columbia.parts has no drop ship fee, free shipping on orders over $200, plus a minimum of 5% off all parts (they discount more for dealer to dealer sales or government sales), and they are donating 10% of all parts sales to the Honor Flight Network for the month of May.