[jwi]

Hi all,

I have made my own benchmarking form on https://docs.google.com/forms/d/1doSSysItq2iiTZ92eu3J7iaLgg9Pe5M_8mo3hlgOAAk/viewform

Before I release the form to the general public (or a more general public at any rate) I'd be happy if someone took the time to look at it and see what they think. Are the instructions clear? Is there something missing? (I would for instance like to add an endurance or strength test for lower back / core but I can't think of one that is simple enough)

I don't mind people answering and guessing the answer to mandatory questions they don't know the answer to.

[Sasquatch]

How much weight can you deadlift? 

Filled out and submitted

[jwi]

Thanks!

I was thinking about deadlift, but I guess most climbers I know don't have a clue to what they can deadlift, and it require quite a lot of special equipment to measure....

[slackline]

Looks good, very useful having height/weight in both metric/imperial and font/vgrades (some didn't bother reading the text or were too lazy to do the conversion themselves for the last survey).

This will likely seem pedantic (not that that will surprise anyone), but I'm strongly opposed to categorising continuous variables such as time, size of holds and additional weight for various reasons such as loss of statistical power and precision of estimates in models, arbitrary categorisation, and quite importantly the false assumption that within categories relationships are linear which they won't be (someone who can have 11kg of additional weight will be different from someone who can have 20kg of additional weight).  These are all nicely summarised with references/citations here*.

I don't mind people answering and guessing the answer to mandatory questions they don't know the answer to.

I guess the rationale here is to have as complete a set of data as possible, one of the problems when I came to try and model/analyse the data of the last survey was that there just weren't enough people who'd completed all fields (and in statistical models you generally need a minimum of 10, but ideally 20 individuals for each predictor you are including in your model if you want meaningfully accurate estimates without massive confidence intervals).  However, I'd strongly caution against including guesses to unknown answers, you may end up with people sticking their grades in but not knowing any of the training performance (or
not being inclined to find them out) and guessing at all of them.  Its a situation of garbage in, garbage out, although I guess in defence of this you could go with John Tukey who said...

Far better an approximate answer to the right question, which is often vague, than the exact answer to the wrong question, which can always be made precise.

...but I think these are perhaps the right questions already.


Good luck

* Obviously this has never stopped people doing it in the past and the practice still continues, it doesn't mean its a sensible thing to do.

[jwi]

Thanks!

I had some input from an experienced coach for comp-climbers.  He suggested that I added more detail to hours trained, which seems like a good idea.

Edge depths in tricky.  For the half crimp the rounded part of the edge should not be measured if the fingers are not in contact with the surface there, but most people hold either pinky or index a bit more open than the other fingers when using the half crimp grip.

[jwi]

I'd also like to point out that the survey is in a beta-state, and is going to change based on comments and what people enter into the form.

[jwi]

I doubt there is much meaning in collecting data that vastly higher precision than can be measured.  Most people measuring themselfs will not know weather the edge is 15 or 16 mm, or how much their clothes are weighting, if the edge is slightly in-cut or sloping etc etc.

For comparison it probably makes sense to pool the data into even fewer cohorts

[slackline]

I kind of disagree, its piss easy to take a tape measure to an edge, stand on some scales or use a stop watch to measure time in seconds.

The issue of how rounded/sharp an edge is is indeed trickier.

For comparison it probably makes sense to pool the data into even fewer cohorts

Its very easy to go from continuous > categories > even fewer categories, but impossible to go in the reverse direction once you've collected your data.

[shark]

Edge depths in tricky.  For the half crimp the rounded part of the edge should not be measured if the fingers are not in contact with the surface there, but most people hold either pinky or index a bit more open than the other fingers when using the half crimp grip.

Perhaps the Beastmaker holds should be adopted as the climbing training equivalent of the Greenwich Meridian.

[jwi]

A big thanks to the test-pilots who tried to fill in the form!

So far there is very little variance. The five that filled in the form are all men with almost exactly the same climbing grade on everything and they can hang from about the same size edge, doing approximately the same numbers of pull-ups (in my coarse stratas), and have all, bar one, average to good oxygen uptake.

[jwi]

Edge depths in tricky.  For the half crimp the rounded part of the edge should not be measured if the fingers are not in contact with the surface there, but most people hold either pinky or index a bit more open than the other fingers when using the half crimp grip.

Perhaps the Beastmaker holds should be adopted as the climbing training equivalent of the Greenwich Meridian.

I don't have access to a Beastmaker, and I think that is true for many. (I have contacts in far flung places that I hope are willing to distribute the test).  But if there is lots of data to be had I am all ears.

For those who do not have access to a 16 mm edge to do the testing on I added the option of the smallest metoliuos rung (which I don't have access to right now either, but from memory it should be slightly harder to hang than a 16 mm flatty - but not by much).

Do you have a suggestion for a BM-grip that is about equivalent to a 16 mm flatty? I can add them for people that don't have access to either.  Are there several kinds of BM? Which is the most popular?

[shark]

Edge depths in tricky.  For the half crimp the rounded part of the edge should not be measured if the fingers are not in contact with the surface there, but most people hold either pinky or index a bit more open than the other fingers when using the half crimp grip.

Perhaps the Beastmaker holds should be adopted as the climbing training equivalent of the Greenwich Meridian.

I don't have access to a Beastmaker, and I think that is true for many. (I have contacts in far flung places that I hope are willing to distribute the test).  But if there is lots of data to be had I am all ears.

For those who do not have access to a 16 mm edge to do the testing on I added the option of the smallest metoliuos rung (which I don't have access to right now either, but from memory it should be slightly harder to hang than a 16 mm flatty - but not by much).

Do you have a suggestion for a BM-grip that is about equivalent to a 16 mm flatty? I can add them for people that don't have access to either.  Are there several kinds of BM? Which is the most popular?

Maybe not such a good idea as getting equivalents to the beastmaker is tricky because not only are they rounded at the lip but also rounded at the back so your fingers stick in further.

The number 5 slot is about 25mm and the number 10 edge is around 6/7mm. The middle 14 slot is around 16mm but obviously you can't hang it two handed and similarly the pockets don't accommodate all your fingers.

http://ukbouldering.com/media/images/bm.jpg


For benchmarking, though not necessarily training, Eva's board is a better internationally available option.

Well constructed survey BTW   

[abarro81]

Do you mind me asking what the point in collecting this data is? What will you gain from having it? My thought would be that people would look at it to say "oh, to climb 8c I need to be able to hang this edge with x kgs on" and focus on that when we all know that's not quite how getting better at climbing works.

[John Gillott]

Edge depths in tricky.  For the half crimp the rounded part of the edge should not be measured if the fingers are not in contact with the surface there, but most people hold either pinky or index a bit more open than the other fingers when using the half crimp grip.

Perhaps the Beastmaker holds should be adopted as the climbing training equivalent of the Greenwich Meridian.

I don't have access to a Beastmaker, and I think that is true for many. (I have contacts in far flung places that I hope are willing to distribute the test).  But if there is lots of data to be had I am all ears.

For those who do not have access to a 16 mm edge to do the testing on I added the option of the smallest metoliuos rung (which I don't have access to right now either, but from memory it should be slightly harder to hang than a 16 mm flatty - but not by much).

Do you have a suggestion for a BM-grip that is about equivalent to a 16 mm flatty? I can add them for people that don't have access to either.  Are there several kinds of BM? Which is the most popular?

Maybe not such a good idea as getting equivalents to the beastmaker is tricky because not only are they rounded at the lip but also rounded at the back so your fingers stick in further.

The number 5 slot is about 25mm and the number 10 edge is around 6/7mm. The middle 14 slot is around 16mm but obviously you can't hang it two handed and similarly the pockets don't accommodate all your fingers.

http://ukbouldering.com/media/images/bm.jpg


For benchmarking, though not necessarily training, Eva's board is a better internationally available option.

Well constructed survey BTW   

? I make the number 10 edge about 14/16mm rather than 6/7. Similarly, the number 5 is > 25mm. As stated on the Lopez threads, the 10 edge on a Beastmaker is a little bit conditions dependent re producing reliable measures of progress on a session-by-session basis, but does this matter so much? A weekly average is probably good enough?

[jwi]

Do you mind me asking what the point in collecting this data is? What will you gain from having it? My thought would be that people would look at it to say "oh, to climb 8c I need to be able to hang this edge with x kgs on" and focus on that when we all know that's not quite how getting better at climbing works.

Curiosity. Most importantly perhaps, I try to construct a useful self-evaluation test which people can use to track their own progress on certain aspects of climbing and training for climbing.

However certain physicals variables have explanatory power for the variance in climbing performance; this is known. Having data on this can help people identifying glaringly weak or strong points.

I am also curious about certain correlations which I have found in the literature (e.g. number of pull-ups is correlated to sport climbing performance for women but not for men).

One of the things I am most curious about is the distribution of training hours put into bouldering and roped climbing, and how this correlates with stated goals (my hypothesis is that they are not correlated at all, at least not for old hands).

[jwi]

After more input from people I have added a requested form for estimating the continuous climbing grade, the lactic threshold, the route "stamina", bouldering "stamina", and how hard individual moves the climber can do.

[shark]

Maybe not such a good idea as getting equivalents to the beastmaker is tricky because not only are they rounded at the lip but also rounded at the back so your fingers stick in further.

The number 5 slot is about 25mm and the number 10 edge is around 6/7mm. The middle 14 slot is around 16mm but obviously you can't hang it two handed and similarly the pockets don't accommodate all your fingers.

http://ukbouldering.com/media/images/bm.jpg


For benchmarking, though not necessarily training, Eva's board is a better internationally available option.

Well constructed survey BTW   

? I make the number 10 edge about 14/16mm rather than 6/7. Similarly, the number 5 is > 25mm.

I wasn't including the bevel/chamfer which illustrates the point about the difficulty in getting an equivalent.

[Sasquatch]

One of the things I am most curious about is the distribution of training hours put into bouldering and roped climbing, and how this correlates with stated goals (my hypothesis is that they are not correlated at all, at least not for old hands).

If you really want to understand this relationship, you may want to add more questions regarding the individuals climbing history.  I.e. how long have they been climbing?, what is their lifetime best?, when they achieved that best how many hours were they training?, How many years were they at or near that max?, etc. 

[jwi]

Stevie is right....

This is how relative pullup-strenght is correlated to the grade the respondents can redpoint within 5 tries:

Spearman rank-correlation coefficient=0.7610944.
The alternative hypothesis, that the stronger a climber is on pullups, the worse he* is at fast redpoints may be true with probabillity 1- 0.9859 = 1.41% probability.

(Admittedly N is too small yet to state anything really)

https://plus.google.com/photos/111272204514144637665/albums/5851384292854470257?authkey=CJbB5YaetbCqggE

-----------
Note: only male respondents so far.

[slackline]

  Thats not how hypothesis testing and probabilities should be interpreted.

(Scatter plots would show the relationship better, histograms like that don't show if the one of the two people who can hang 5.5mm edges is the person who can RP Sport 8c)

[jwi]

Thats not how hypothesis testing and probabilities should be interpreted.

(Scatter plots would show the relationship better, histograms like that don't show if the one of the two people who can hang 5.5mm edges is the person who can RP Sport 8c)

Of course not.  I only shown the histograms, so far.  Scatterplots make no sense yet, to few respondents.

[slackline]

Scatterplots make no sense yet, too few respondents.

Then there seems little point in calculating/reporting (non-parametric) correlation at this point either.


  An aside, since the plots look like they might have been done in R is this very useful tutorial on using ggplot2 I came across the other day, a brilliant graphics engine if you are plotting things in R. 

[jwi]

Scatterplots make no sense yet, too few respondents.

Then there seems little point in calculating/reporting (non-parametric) correlation at this point either.

I was facetious I'm afraid.  I have done half of the scripting, and checked it on the responds so far, and the p-value just jumped out from the various cor.tests. (I very much doubt they are that strongly correlated in reality.)

[slackline]

If you're doing multiple two-way comparisons you will of course need to adjust your threshold for accepting statistical significance to account for the problem of multiple testing as it increases the chance of making a Type I Error (ie. false positive).

BTW your Alternative Hypothesis (H₁) is that the stronger a climber is on pull-ups, the higher their redpoint grade.  Its antonym is the Null Hypothesis (H₀) that there is no relationship between how strong a climber is on pull-ups and their red-point grade.  A p-value tells you how likely you are to observe the results by chance alone (ie. if the Null hypothesis is true).  But this is not the place to be giving statistics classes, as its all written in many books and web-sites far better than I have the time or inclination to explain it.

[jwi]

I think I am aware of the difference between two-sided testing, one-sided testing, and is also aware the risk of false positives in large datasets. But thanks anyway!

(I have not studied more statistics than absolutely necessary to get by, but I have a decent understanding of probability-theory, at least in the context of stochastic processes, especially if they are martingales with respect to a filtration)

[slackline]

Ok, just that your statement about the alternative hypothesis (even under a one-sided test) was confusing, as all a one-sided test does is stipulate the direction of an effect, its opposite is still the null, and not an effect in the other direction.

Its also not the size of a dataset that increases the risk of a false positive (large datasets actually decrease the chance of this, although small differences will be statistically significant in larger datasets), rather the number of tests that are being performed.

Anyway, I'll shut up and say no more, its your work not mine. 

[jwi]

1   Benchmarking


1.1  What should be tested

...

1.2  What can we learn from tests?


Demonstrating strength is different from having or acquiring strength.

   Every climber needs to understand the results of the benchmarks according to his or hers own training background. Someone that is demonstratively good at holding on to small flat edges might be strong on that because a) climbed a lot on small flat edges, or b) have trained a lot on small flat edges and has achived a high neural recruitment ("technique") specific for the testing apparatus.

   Compare yourself to others, and compare to your own results from before, but remember that lots of the variation can be explained by familiarity with the testing apparatus.


2   Survey


2.1  What kind of climbers took part of the survey?


21 male climbers and 0 female climbers with the following characteristics filled in the self-evaluation questionaire:

Sport RP: 7c/+
Sport OS: 7a+/b
Boulder RP: 7B+
Boulder flash: 7A
Trad X: 5c+/6a
Trad OS: 6c+/7a
Number of pull-ups: 17
Pull-up strength/weight ratio: 32% of body-weight added
16 mm open crimp, strength/weight ratio: 44% of body-weight added
Minimum edge open crimp: 8.6 mm
No repeaters (10s on, 5s rest) on 16 mm open crimp: 5.5
Static hang 16 mm open crimp: 33 s
Hanging knee raises: 23
Cooper test: Average (for untrained), VO2max 45 mls/kg/min

Code: [Select]

   Quantiles:______________________________________________________________
   _Test___________________________________0%____25%____50%____75%___100%__

    Number of pull-ups                     8     13     18     23    28
    Pull-up strength/weight ratio          0.035 0.169  0.312  0.368 0.926
    16 mm open crimp, strength/weight ratio0.036 0.276  0.512  0.584 0.913
    Minimum edge open crimp                5.5   6.75   8      9.875 14
    Intermittent endurance 16 mm open crimp4     4      5      6     9
    Static endurance 16 mm open crimp      12.5  25     30     45    60
   _Abs_stamina____________________________2.5___13_____20_____30____50____


3   Results


I have only counted grade-differences in full french letter grades.  E.g.  the difference between 8b and 8c is 1 grade, the difference between 8b and 8b+ is .5 grades.



3.1  Tradclimbing


The difference between the sport and safe trad o/s grades was on average 0.6 full french grades, and the quantiles are given by


 0%   25%   50%   75%  100%
-3.00  0.00  1.00  1.75  3.00


   The difference between the scary but relatively safe trad and safe trad o/s  grades was on average -0.82 full french grades. and the quantiles are given by


 0%  25%  50%  75% 100%
-2.0 -1.0 -1.0 -0.5  0.0


   The difference between the risky and safe trad o/s grades was on average -2.1 full french grades. and the quantiles are given by


 0%  25%  50%  75% 100%
-4.0 -2.5 -2.0 -1.0 -1.0



   The difference between the sport and (safe) trad o/s grades was on average -3.3 full french grades. and the quantiles are given by


  0%  25%  50%  75% 100%
-6.0 -4.5 -3.0 -2.0 -1.0


   Assuming linear behaviour over the grades the typical climber that onsight 7b/5.12b on safe trad is onsighting between 6b+ and 7a (5.10a-5.11d) on risky trad. Values outside these indicates an outlying behaviour.
   

Code: [Select]

     __________________________________________________________________
     | Trad OS (safe)||Sport OS grade||  Trad X|  |Trad R | |Trad PG/R |
     |_______________|25%____75%___|_25%__75%_|_25%___75%_|25%___75%__|_

     |       5c       |5c   6a+/b  | 4a+   5a  |4c+   5b  | 5b   5b+  |
     |       6a       |6a   6b+/c  | 4b+   5b  |5a+   5c  | 5c   5c+  |
     |       6b       |6b  6c+/7a  | 4c+   5c  |5b+   6a  | 6a   6a+  |
     |       6c       |6c   7a+/b  | 5a+   6a  |5c+   6b  | 6b   6b+  |
     |       7a       |7a   7b+/c  | 5b+   6b  |6a+   6c  | 6c   6c+  |
     |       7b       |7b  7c+/8a  | 5c+   6c  |6b+   7a  | 7a   7a+  |
     |       7c       |7c   8a+/b  | 6a+   7a  |6c+   7b  | 7b   7b+  |
     |_______8a_______|8a___8b+/c__|_6b+___7b__|7a+___7c__|_7c___7c+__|_


3.2  Stamina and strength-endurance for sport climbing


Be aware that there is yet little data to support these tables. However, they are reasonably close to a similar table an experienced coach I know (who work with beginners and world class climbers alike) use to prescribe levels.

Code: [Select]

    _______________________________________________________________________
  | Sport RP|| Fast RP  ||   Circuits| L|apping routes|| Treshold| |Continuous
  |
   |__________|25%__75%_|_25%___75%_|25%_____75%___|25%___75%_|_25%___75%_|_

   |    7a    |6c    6c  | 6a    6c  |6a      6b    |5c    6a  |5b    5c  |
   |    7b    |7a    7a  | 6b   7a  | 6b      6c    |6a    6b  | 5c   6a  |
   |    7c    |7b    7b  | 6c   7b  | 6c      7a    |6b    6c  | 6a   6b  |
   |    8a    |7c    7c  | 7a    7c  |7a      7b    |6c    7a  |6b    6c  |
   |    8b    |8a    8a  | 7b   8a  | 7b      7c    |7a    7b  | 6c   7a  |
   |____8c____|8b____8b__|_7c___8b__|_7c______8a____|7b____7c__|_7a___7b__|_


3.3  Stamina and strength for bouldering


So far this table looks loop-sided to my eyes, perhaps because of lack of data, or perhaps because my perceptions where wrong. Time will tell.

Code: [Select]


   ________________________________________________________
   | Boulder RP||Boulder quick send||Stamina laps|| Max level  |
   |____________|25%______75%_____|25%____75%__|25%___75%_|_

   |     6b     |6a        6b      |5b    5c   | 6b    6b  |
   |     6c     |6b        6c      |5c    6a   | 6c    6c  |
   |     7a     |6c        7a      |6a    6b   | 7a    7a  |
   |     7b     |7a        7b      |6b    6c   | 7b    7b  |
   |     7c     |7b        7c      |6c    7a   | 7c    7c  |
   |     8a     |7c        8a      |7a    7b   | 8a    8a  |
   |_____8b_____|8a________8b______|7b____7c___|_8b____8b__|


3.4  Physical tests


Unsurprisingly the tests shows that climbing performance is correlated with results from physical tests, even though the number of participants is yet to low to do any robust modelling.

[jwi]

Unsurprisingly, from the data I collected so far it is dead easy to guess someones max RP grade from their performance on circuits, 10 laps,  threshold- and continuous climbing level.

[Muenchener]

Interesting analysis jwi, thanks

   The difference between the risky and safe trad o/s grades was on average -2.1 full french grades.

This makes intuitive sense wrt the uk trad grading system: e.g. E3 can be anything from dangerous 6a to safe 6c

[jwi]

I have sent a long and confusing e-mail to the kind people who filled in my questionnaire.

Look in your spam-box...

Hopefully I will be able to give reasonable confidence intervals for predicted performance when I get more data (and when I have translated the questionnaire to spanish and french).

Cheers,