You should stick with Sierra 10.12.x on this system. High Sierra and newer uses a newer file system APFS which has issues with SATA based systems like yours. Don’t even go with Catalina!
Yes, you can upgrade to a SSD MacBook Pro 17" Unibody Hard Drive Replacement as well as upgrade the RAM to 16 GB MacBook Pro 17" Unibody RAM Replacement. I don’t recommend replacing the optical drive with a drive. SSD’s are much cheaper today than what we had back in 2011 and are much larger! Best to just get the size you need.
Given the age of your system you should replace the battery as well [gekoppeld product ontbreekt of is uitgeschakeld: IF162-035-2] that should help keep the MagSafe charger from running as long. I would recommend making sure your charger is in good shape and the MagSafe connector is clean.
Sadly, both your WiFi and Bluetooth services can’t be internally upgraded, you’ll need external dongles to offer newer services.