I am not as pessimistic about the future capabilities, and definitely not as sure as you are (hence this post), but I see what you describe as a possibility. Definitely there is a lot of overhang in terms of augmentation: https://www.oneusefulthing.org/p/it-is-starting-to-get-strange
I think I touched on these points, that some things are easy and others are hard for LLMs, in my other post, https://www.lesswrong.com/posts/S2opNN9WgwpGPbyBi/do-llms-dream-of-emergent-sheep
I am not as pessimistic about the future capabilities, and definitely not as sure as you are (hence this post), but I see what you describe as a possibility. Definitely there is a lot of overhang in terms of augmentation: https://www.oneusefulthing.org/p/it-is-starting-to-get-strange