I'm not sure what cleaning stuff(s) you're using specifically, or how/why you got it on the grip area of the frame, but I've had rubberized Talon Grips on almost all of my carry pistols for the past year and have had absolutely ZERO problems with them peeling off. On my Glock 19, I did manage to wear through the grip tape material on one upper corner by using it with a Safariland GLS OWB paddle holster, as the holster contacted the frame kinda tightly there, but I don't fault the Talon Grips for that. My gal also has a Traction Grips brand set (also rubberized) on her Walther PPS M2 and no problems thus far, seems to be of the exact same material and quality as the Talon Grips.
A lot of it depends upon the application. You have to be very, VERY thorough with cleaning the frame before you go sticking on the grips, and then using a hair dryer afterwards and pressing it onto the frame into every nook n' cranny is absolutely ESSENTIAL to getting it to stay on there.
Can't speak from experience with the sandpaper type ones, though. Those things are too abrasive for my delicate, fragile little mitts, and I'm not keen on having actual sandpaper rubbing up against my torso when I carry concealed.