First of all, many thanks to jnesoff for the pictures. They encouraged me to change the battery because my headset only lasted about 2 hours.
But I made one mistake and I'm writing here to make it a little easier for others. My comment refers to the first picture. I tried to take the foam off and ended in cutting it into small pieces. In the process, I noticed that this is not the right way to do it, because the foam has a backing film that can be quite easily removed from the headphone housing in one piece. So when you are in this step, don't go between the foam and the backing film with the cutter knife, but try to get hold of the backing film and remove it straight away.
I was able to find the battery easily on ebay (Germany) - I'm sure it's also available in other countries. The part number is AHB403029.
Take pictures of the individual steps with your mobile phone, it helped me during assembly.
If you have the tools at hand, the whole thing can be done in about 30 minutes.