I think the average undergrad / new grad student (especially my past self) overvalues hard ambitious projects and undervalues easy straightforward projects. A good easy project should teach you lots of things that will be applicable in the future, and should contribute to the field at least a little bit, but should otherwise be almost as easy as possible.
To pick one example among many, “replicate some other paper but do better analyses on what happens as you tweak the architecture” is a huge family of easy, unsexy projects that will rapidly catapult you to being a world expert on a subject. I think if a mentor is suggesting easy projects that will teach you useful stuff, that should be a quite positive sign, not a reason for an “oh that’s boring” reaction. And if you look at those easy projects and think “but the things it would teach me aren’t the things I want to learn,” try to think of an easy project that would teach you the things you want to learn.
On ranking mentors, I think people have good instincts about trading off expertise as a researcher versus goodness as a mentor, but they probably overvalue connections and status. If that status didn’t come from banger research, you should generally ignore it.
I’m not sure I agree—I think historically I made the opposite mistake, and from a rough guess the average new grad student at top CS programs tends to look too much for straightforward new projects (in part because you needed to have a paper in undergrad to get in, and therefore have probably done a project that was pretty straightforward and timeboxed).
I do think many early SERI MATS mentees did make the mistake you describe though, so maybe amongst people who are reading this post, the average person considering mentorship (who is not the average grad student) would indeed make your mistake?
Yep, I think theLaw of Equal and Opposite Advice applies here.
One piece of advice which is pretty robust is — You should be about to explain your project to any other MATS mentee/mentor in about 3 minutes, along with the background context, motivation, theory of impact, success criteria, etc. If the inferential distance from the average MATS mentee/mentor exceed 3 minutes, then your project is probably either too vague or too esoteric.
(I say this as someone who should have followed this advice more strictly.)
I’d guess this varies by field? I think this would be bad advice in mech interp—there’s a lot of concepts and existing mech interp theory that you need to understand a bunch of good projects, and people new to the field are often bad at explaining these (and, importantly, I think I have decent judgement about whether a project is any good). But I’d guess this is decent advice in some areas of alignment.
My thoughts from more or less the other side:
I think the average undergrad / new grad student (especially my past self) overvalues hard ambitious projects and undervalues easy straightforward projects. A good easy project should teach you lots of things that will be applicable in the future, and should contribute to the field at least a little bit, but should otherwise be almost as easy as possible.
To pick one example among many, “replicate some other paper but do better analyses on what happens as you tweak the architecture” is a huge family of easy, unsexy projects that will rapidly catapult you to being a world expert on a subject. I think if a mentor is suggesting easy projects that will teach you useful stuff, that should be a quite positive sign, not a reason for an “oh that’s boring” reaction. And if you look at those easy projects and think “but the things it would teach me aren’t the things I want to learn,” try to think of an easy project that would teach you the things you want to learn.
On ranking mentors, I think people have good instincts about trading off expertise as a researcher versus goodness as a mentor, but they probably overvalue connections and status. If that status didn’t come from banger research, you should generally ignore it.
I’m not sure I agree—I think historically I made the opposite mistake, and from a rough guess the average new grad student at top CS programs tends to look too much for straightforward new projects (in part because you needed to have a paper in undergrad to get in, and therefore have probably done a project that was pretty straightforward and timeboxed).
I do think many early SERI MATS mentees did make the mistake you describe though, so maybe amongst people who are reading this post, the average person considering mentorship (who is not the average grad student) would indeed make your mistake?
Yep, I think the Law of Equal and Opposite Advice applies here.
One piece of advice which is pretty robust is — You should be about to explain your project to any other MATS mentee/mentor in about 3 minutes, along with the background context, motivation, theory of impact, success criteria, etc. If the inferential distance from the average MATS mentee/mentor exceed 3 minutes, then your project is probably either too vague or too esoteric.
(I say this as someone who should have followed this advice more strictly.)
I’d guess this varies by field? I think this would be bad advice in mech interp—there’s a lot of concepts and existing mech interp theory that you need to understand a bunch of good projects, and people new to the field are often bad at explaining these (and, importantly, I think I have decent judgement about whether a project is any good). But I’d guess this is decent advice in some areas of alignment.