To me, polution or consumption of finite resources is just a side effect, and not a neccessary or causative part of that knowledge and expertise growth that drives economic growth. We could potentially totally decouple economic growth from environmental degradation. We could just use renewable energy and recycled materials to provide for ourselves in an ever more effective way.
I think you're forgetting slightly about arable land and potable water, neither of which are endlessly available.
I'm not forgetting about finite resources such as that. There is only so much that people can eat and drink. Economic growth needen't be seen as providing more and more wastage of food and water. Arable land and potable water can be managed in a sustainable way or mismanaged in an unsustainable way. Longer term, continued economic growth actually depends on them being managed in a sustainable way. We could have a global economy that provided everyone one with a decent lifestyle AND used arable land and potable water in a sustainable manner. Getting to that point could entail a lot of economic growth from where we are now. Ofcourse economic growth COULD be entirely malign, just involving people making torture equipment and burning each others' houses down or whatever, but it needn't be, it could be entirely symptomatic of doing things well.