It seems like in realistic versions of your promts you describe the job position extremely briefly, and add statements about the culture, as well as other things, but real job descriptions primarily describe the job and requirements for candidates, which is almost absent in all of your prompts. Doesn’t seem like realistic job descriptions to me.
In this paper I was also examining the robustness of existing hiring bias evaluations when adding realistic detail, which limited our degrees of freedom. The dataset from the evaluation had a bunch of IT industry resumes, but across a wide range of experience and skillsets. I considered adding job descriptions, but the majority of candidates wouldn’t be well matched for any given specific job, which would limit our ability to evaluate many candidates under a single prompt for simplicity.
I agree that it would be good to extend this work to complex and realistic job descriptions.
It seems like in realistic versions of your promts you describe the job position extremely briefly, and add statements about the culture, as well as other things, but real job descriptions primarily describe the job and requirements for candidates, which is almost absent in all of your prompts. Doesn’t seem like realistic job descriptions to me.
That’s a fair point.
In this paper I was also examining the robustness of existing hiring bias evaluations when adding realistic detail, which limited our degrees of freedom. The dataset from the evaluation had a bunch of IT industry resumes, but across a wide range of experience and skillsets. I considered adding job descriptions, but the majority of candidates wouldn’t be well matched for any given specific job, which would limit our ability to evaluate many candidates under a single prompt for simplicity.
I agree that it would be good to extend this work to complex and realistic job descriptions.