I've been to that area of the crag several times and been joined by large groups of local youths mooching about in the woods. I very much doubt they'll stop going there on the crag side of the fence, and as has often been argued with Anston, the presence of climbers may well be a positive influence.
I also doubt that continuing to go would jeopardise access to the rest of the crag. As has been pointed out, the Parish Council have an explicit commitment to engaging with climbers and allowing access.
Network Rail's concern will be purely safety based, and hence the fence helps mitigate that risk. I can't imagine they'll be too bothered by people keeping outside of the fence / off the tracks.
All in all, I would say continued discrete sessions are fine.
On a related note, Carlos I can't decide whether your Insta posts calling people out are great fun / much needed in British climbing, or a bit much. I mean, who appointed you as the judge, jury and executioner? Albeit your heart is clearly in the right place on this at least.
I love your enthusiasm don't get me wrong, we could do with more of it a lot of the time, although maybe best to ask questions first, shoot later?
PS I'm the crag moderator for Anston on UKC but I think the access panel comes from somewhere else; I don't seem to be able to change it.