(This was originally posted as a comment in the Open Thread Summer 2026, but I garner that ‘sorted by new’ comments on the Open Thread are not an optimal place to cast a question.)
The Question:
I am curious what opinions/writings exist on the ethics and plans for AI presence in previously AI-unrelated research directions: does anyone have suggestions for useful places to start?
The Context:
I am a PhD student working between physics (hep-th) and mathematics (math.AG / math.CO). This path found my by the offer of solving abstract problems that interest me, rather than having a genuine impact beyond the creation of new knowledge; in fact, the trajectory was set in part by my determination not to produce what could be used for short-term development of technology[1].
From outside (and even from inside), it is hard to tell exactly what impact AI is having on mathematical research. Headlines with counterexamples to old conjecture(s), statements about the desired future, and mixed reports on the sophistication of approaches[2] are interesting, but do not tell the story of the many researchers unsure of what place this has now and what place it will have in the future. Browsing LessWrong, much discussion is focused on those AI-related issues that are more existential or otherwise of greater moral weight. Nonetheless, I believe that part of thinking about AI safety is to think of how to rework existing institutions[3] so they do not completely crumble under the weight of those tools that already exist, much less those tools that do not exist yet.
At the time of choosing my trajectory, I believed that the rapid development of technology is scary, and that is hard to trust that its rapid development will lead to the best outcomes! I did not wish for my curiosity-driven work to be transformed by people I do not know into tools I do not recognize or endorse. (I speak in past tense, though my opinions have not changed so much in generality, with the particular exception that we have come to need rapid development of those technologies that mitigate the negative effects of other ones.)
There could be some debate of whether it is worth saving institutions (e.g. academia and its investment in basic research) in the face of bigger problems, or perhaps better solutions to the problems that such institutions currently solve. I believe the answer is that it is worth it, but will save the thoughts for another time.
(This was originally posted as a comment in the Open Thread Summer 2026, but I garner that ‘sorted by new’ comments on the Open Thread are not an optimal place to cast a question.)
The Question:
The Context:
At the time of choosing my trajectory, I believed that the rapid development of technology is scary, and that is hard to trust that its rapid development will lead to the best outcomes! I did not wish for my curiosity-driven work to be transformed by people I do not know into tools I do not recognize or endorse. (I speak in past tense, though my opinions have not changed so much in generality, with the particular exception that we have come to need rapid development of those technologies that mitigate the negative effects of other ones.)
For example, in the First Proof Second Batch benchmark, there is a stark difference between the sophistication of the harness used by team B and the prompt used by team C.
There could be some debate of whether it is worth saving institutions (e.g. academia and its investment in basic research) in the face of bigger problems, or perhaps better solutions to the problems that such institutions currently solve. I believe the answer is that it is worth it, but will save the thoughts for another time.