I do believe it is £3.50 at any time. You don't have to join, that's the public entrance fee. And yes, it's at U-Sport, just on the left as you walk down. It's worth checking the evening opening times as they close the pool for uni Swimming / Canoeing practise on some evenings.
I go fairly often to "swim" i.e. chill out in the sauna for ages and dabble my toes in the pool for 5 minutes
. Soothes those aches allright. Unfortunately the sauna has gone a bit shit recently (only 70'c, WTF), but the steam room makes up for it.