… is that a prototypical problem from that benchmark? Because if so, that is a hilariously easy benchmark. Like, something could ace that task and still be coding at less than a CS 101 level.
(Though to be clear, people have repeatedly told me that a surprisingly high fraction of applicants for programming jobs can’t do fizzbuzz, so even a very low level of competence would still put it above many would-be software engineers.)
people have repeatedly told me that a surprisingly high fraction of applicants for programming jobs can’t do fizzbuzz
I’ve heard it argued that this isn’t representative of the programming population. Rather, people who suck at programming (and thus can’t get jobs) apply to way more positions than people who are good at programming.
I have no idea if it’s true, but it sounds plausible.
Rather, people who suck at programming (and thus can’t get jobs) apply to way more positions than people who are good at programming.
I have interviewed a fair number of programmers, and I’ve definitely seen plenty of people who talked a good game but who couldn’t write FizzBuzz (or sum the numbers in an array). And this was stacking the deck in their favor: They could use a programming language of their choice, plus a real editor, and if they appeared unable to deal with coding in front of people, I’d go sit on the other side of the office and let them work for a bit.
I do not think these people were representative of the average working programmer, based on my experiences consulting at a variety of companies. The average engineer can write code.
So that example SWE bench problem from the post:
… is that a prototypical problem from that benchmark? Because if so, that is a hilariously easy benchmark. Like, something could ace that task and still be coding at less than a CS 101 level.
(Though to be clear, people have repeatedly told me that a surprisingly high fraction of applicants for programming jobs can’t do fizzbuzz, so even a very low level of competence would still put it above many would-be software engineers.)
I’ve heard it argued that this isn’t representative of the programming population. Rather, people who suck at programming (and thus can’t get jobs) apply to way more positions than people who are good at programming.
I have no idea if it’s true, but it sounds plausible.
I have interviewed a fair number of programmers, and I’ve definitely seen plenty of people who talked a good game but who couldn’t write FizzBuzz (or sum the numbers in an array). And this was stacking the deck in their favor: They could use a programming language of their choice, plus a real editor, and if they appeared unable to deal with coding in front of people, I’d go sit on the other side of the office and let them work for a bit.
I do not think these people were representative of the average working programmer, based on my experiences consulting at a variety of companies. The average engineer can write code.
If you want to take a look I think it’s this dataset (the example from the post is in the “test” split).