yes you should definitely be doing other stuff - but mainly for injury prevention.
Shoulders(all 3 heads and the rotary cuff), scapula retraction, wrist extensors, finger extensors, triceps. These are the main things to work on that will keep you climbing strong in the long run.
It sounds like a lot and in a way it is, but none of the exercises need to take too long, none of them need to be that intensive, and it's totally worth it. I do this stuff 2 hours total a week and it's plenty.