So, I sold the wheel, quite a few months ago now actually, but only now I remembered to do an update, because maybe, it’ll be useful for someone. The guy I sold it to, found the problem and it was something with the sensor which counts how many times wheel rotates, I’m guessing that’s going to be the optical enconder? It’s a translation, so it might not be accurate, but he said that the problem was that a few traces were broken on the sensor’s board and one of the connectors was moving a bit. TL:DR; If you have a problem similar to mine and a new power adapter doesn’t help, chances are the problem lies in the optical enconder.

So, an update after a month. I ordered a new power adapter from Aliexpress, instead of doing some more testing. It finally arrived and the wheel is still dead. Checked the voltage on the adapter with a multimeter, that's seems fine. Chances that it has the same problem as the previous one I guess are small, so most likely it's something in the inside of the wheel, which is really sad. Thanks for trying to help and I at least learned a few things.

That's the thing, I never seen 2003 parts in real-life so I don't know if I could use it for 2006 because I can't find the parts I need for 2006.

The problem was the memory card. I took it out and now PSP works.