This is typically caused by one of two things:
1. IIS 7 is running in Classic Mode, not Integrated Mode.
2. The web.config is missing the handler and/or module registrations. Compare yours to a stock one in a fresh download of GSP.
If this doesn't help, use a tool like Firebug to see if there are any javascript or network errors (like requests that return 404).
Roger Martin
Creator and Lead Developer of Gallery Server Pro