Did you replace the battery with a new one or just put back the original? For such an old device, I would certainly suspect the battery too. However, there are some really poor quality batteries on the market and depending on where you bought it (or will buy it), you have to be careful.
If you need to order one, I would certainly recommend iFixit batteries.
Now, if the battery you replaced was new and from a reputable supplier like iFixit (as opposed to eBay), there is a possibility that you have an issue with your charging circuit on your logic board. That can be caused by using fake chargers and cables, especially car chargers from the dollar store or gas station.