Your headphone jack is sort of built into the rear case, and doesn't have a plug, but instead has contacts that are pressed against 5 gold square shaped contacts on a cable attached to the logic board. It sounds like there might be a dirty or intermittent connection there.
This image might help you:
http://guide-images.ifixit.net/igi/UIUAkltfFOyvmacw.huge You can see the jack built into the panel on the top left of the picture, and the cable it contacts with built into the top right corner of the logic board at the very bottom of the picture.
Following this guide should get you deep enough into the phone to clean the contacts/inspect the jack.
[guide|2980]