Sounds like the camera app itself is broken. Try [guide|160620|safe mode.|new_window=true] If that doesn't work, you'll probably have to factory reset it unless you have CFW.
If safe mode failed and you have CFW, you can try deleting the camera app, then updating the system to restore it. Launch FBI, then go to Titles > Nintendo 3DS Camera > Delete Title And Ticket. Press A to confirm, then update the console (you can use safe mode or the regular system update). The update will redownload and install the app, leaving you with a working camera