American express card charged without permission?

Yesterday we needed a limo last minute for an event and a limo company told me that they just needed my AMEX card to hold it, nothing would be charged and that I could pay everything in cash when I saw them 3 hours later that same day. My husband was also calling companies and had already booked one & paid a deposit while I was on the other line without my knowing. So I called back to the 1st limo company within 25 minutes, greatly apologized and said that I needed to cancel our limo reservation and explained how my husband had paid a deposit and had already reserved another limo. The guy was very disappointed but never said anything about being charged for anything. Today I get a voicemail from the "cancelled" limo company saying that they are putting a $200 charge on my AMEX because I had inconvenienced their driver by rearranging his day at the last minute. Can I dispute this ? I never signed anything or gave any permission to put any charges on this card, I was promised it was just to hold the limo reservation. I'm not sure how I greatly inconvenienced their driver when I had called back within 25 minutes to cancel. Please help, thank you so much !!!