get a lot of calluses from weight lifting and unfortunately they always tear off and flap about when i climb leaving sore bits
1)stop doing weights, you'll end up looking like nadiras n who wants that
2)stop using hand sized holds n love the crimp
i used to do weights n had this problem so i stopped, n my elbows can breathe again as well