Have rock climbed at more than 4,000m a few times. Once, on the Diamond CO, I felt pretty f*cked trying to do something too hard having walked straight from the valley floor the previous day. As others have said, you should be fine as long as you take time to acclimatise (a couple of nights sleeping at that altitude before targeting the 50m overhanging off widths).
TBH I wouldn't bother with Diamox if you are only going to 4,000m. Not everyone gets on with it anyway and I've been on trips where those who are taking it have suffered the most from altitude. But it can be good, especially for helping you sleep. Just tried it the once and couldn't be ar*ed with the side effects - tingly toes and fingers and peeing a lot.