Well, contrary to what some may believe, the new PSP is most likely not going to be named PSP 2, or PSP-4000 or any other numerical advancement. Instead it will be called PSP Go! and will feature a sliding screen. Rumors indicate a possibility of touch screen (though personally, I doubt it). Sony will also have 8 or 16GB of internal memory (from what I hear, flash memory). It also seems like they may be using a new type of Memory Stick (assuming additional memory will be allowed in addition to the internal memory). One of the main differences is the dropping of the UMD drive. Instead, Sony plans to offer game downloads at launch. There will still be only one analog stick.
Sources:
http://www.gamedaily.com/articles/news/psp-go-revealed-e3-2009-says-report/
http://www.joystiq.com/2009/05/01/rumor-psp-go-to-use-memory-stick-hg-micro-or-something-like-it/
http://www.crunchgear.com/2009/04/28/touchscreen-slider-psp-go-confirmed-for-this-fall/
http://www.joystiq.com/tag/psp2
http://www.boygeniusreport.com/2009/04/29/is-the-psp-go-the-new-umd-less-psp/
I sure hope the final PSP Go! doesn't look as awful as the mockups. It's too early to tell about hackability, unfortunately.