I recently had this same problem. I took a straightened paperclip and bent the end 90 degrees (with pliers) so that it looked like an "L" with a tiny "foot" - about 5mm or 0.2 inch. I left most of the other end of the paperclip coiled up as it was a good "handle".
This was an excellent tool for taking the cover off my iPhone. You can poke it into the tiny square hole on each side of the cover and twist so the foot turns outwards and the cover just releases. It released almost too easily!