old iPhone5 battery works fine, new battery flashes apple logo

My battery in my iPhone 5 is going dead really quick so I bought a 3rd party battery. I took out the old battery and put the new one in, but my iPhone 5 won't turn on now. When I plug it in to an outlet or my Windows computer, the phone just flashes the Apple logo. I've tried putting the phone into DFU setting and then plugging it in with iTunes open, but when iTunes tries to restore the phone, the phone turns on and still flashes the Apple logo.

I took the new battery out, put the old battery back in, and it still works fine. So I don't understand the problem - is the new battery just dead, even though it makes the Apple logo flash, showing it must have some power in it?