To anyone currently going through NeurIPS rebuttals fun for the first time, some advice:
Firstly, if you’re feeling down about reviews, remember that peer review has been officially shown to be a ridiculous random number generator in an RCT—half of spotlight papers are rejected by another review committee! Don’t tie your self-worth to whether the roulette wheel landed on black or red. If their critiques don’t make sense, they often don’t (and were plausibly written by an LLM). And if they do make sense (and remember to control for your defensiveness), then this is great—you have valuable feedback that can improve the paper!
Read this guide to get a sense of what rebuttals are about
Generally, be nice and polite, even if your reviewers are really annoying
You have three goals here:
Improving the paper! Often reviewers raise some good and useful points, and ultimately one of the key goals is doing good research and communicating it well to the world
Convince reviewers to like you, so they increase their score
For unreasonable reviewers who dislike you, your goal is to convince the area chair (and other reviewers) that this person is wrong and unreasonable. This means you still should write a careful and well-argued rebuttal, even to onbnoxious reviewers, but have a different target audience in mind.
Meta: The way the process works is that the area chair makes the final decision, and has a lot of discretion to overrule reviewers, but by default if lazy will go by the average reviewer score. You want to either increase average reviewer score, or convince the area chair to ignore the bad ones. Convincing a reviewer is just a means to the end.
One of the key things to do in the rebuttal is to improve the paper. Realistically, you can’t upload a new version, so your actual goal is to convince people that you have improved the paper. It is an adversarial setting and people will generally assume you are lying if you just give empty words, especially saying you will do X by the camera ready. So the key question to ask is how can you show proof of work? Running experiments and reporting the results is one good way (or even just saying that you’ve done them).
A common piece of feedback is “this is badly written”.
This common because it’s often true! Writing papers is hard (some advice).
If you receive this feedback, try to fix it (eg give an LLM the reviews and your paper, and maybe my post, and ask it to give concrete feedback on how to improve things, along with quotes). This will improve the paper even if you don’t get in
One difficulty is that even if you improve the writing a bunch, this is hard to convince anyone of in the rebuttal, since they’re normally not willing to re-read in detail (and NeurIPS doesn’t even let you re upload).
My best strategy is to make a long changelog of what you improved, to signal high effort, and put it in a top-level comment
If the reviewer complained about a specific paragraph or section, copy in the reworded version of that
It often helps to add an appendix with a glossary for key terms, ideally both intuitive and technical definitions
I recommend the following process:
Copy all reviews to a google doc.
Go through and comment on each complaint in each review, sorting them into misunderstandings, disagreements with you, presentation issues, and technical issues—either do this while on a call, or async
Brainstorm how to address each complaint—prioritise the important ones
Write a bullet point outline, and try to get feedback
Write it up nicely and send.
You typically want to have a comment per review, and a top level comment covering critiques from multiple reviewers. For some insane reason NeurIPS 2025 removed your ability to make a top level comment—copy and paste between reviewers I guess?
Picture the area chair as your audience for the top level comment. You want to begin with a paragraph about the strengths of your paper, as noted by reviewers, supported by reviewer quotes—imagine you’re writing something the area chair can copy and paste into a meta-review about accepting you. Reviewer quotes are key for any positive claims as no one will trust you to be honest.
If one reviewer hates you, the top-level comment is a good opportunity to try to discredit them by emphasising how other reviewers disagree, as politely as possible. For example, we appreciated the constructive critique from bad reviewer that X, and have changed Y to fix it. But we are glad to see that good reviewers A and B thought Z followed by quotes supporting Z from the good reviewers, where Z contradicts X as much as possible.
Some technical complaints are best addressed by doing new experiments—you have 1-2 weeks to do this but should ask yourself how long it’ll take and whether this is the best use of time. Time is constrained and you want to maximise returns per unit time, and new experiments often take much longer than writing or conceptual rebuttals—prioritise these carefully.
Thanks for this helpful framework, it’s also useful for people who are submitting rebuttals not for the first time :). Sadly NeurIPS and ICML no longer allow a top-level comment (for silly technical reasons).
To anyone currently going through NeurIPS rebuttals fun for the first time, some advice:
Firstly, if you’re feeling down about reviews, remember that peer review has been officially shown to be a ridiculous random number generator in an RCT—half of spotlight papers are rejected by another review committee! Don’t tie your self-worth to whether the roulette wheel landed on black or red. If their critiques don’t make sense, they often don’t (and were plausibly written by an LLM). And if they do make sense (and remember to control for your defensiveness), then this is great—you have valuable feedback that can improve the paper!
Read this guide to get a sense of what rebuttals are about
Generally, be nice and polite, even if your reviewers are really annoying
You have three goals here:
Improving the paper! Often reviewers raise some good and useful points, and ultimately one of the key goals is doing good research and communicating it well to the world
Convince reviewers to like you, so they increase their score
For unreasonable reviewers who dislike you, your goal is to convince the area chair (and other reviewers) that this person is wrong and unreasonable. This means you still should write a careful and well-argued rebuttal, even to onbnoxious reviewers, but have a different target audience in mind.
Meta: The way the process works is that the area chair makes the final decision, and has a lot of discretion to overrule reviewers, but by default if lazy will go by the average reviewer score. You want to either increase average reviewer score, or convince the area chair to ignore the bad ones. Convincing a reviewer is just a means to the end.
One of the key things to do in the rebuttal is to improve the paper. Realistically, you can’t upload a new version, so your actual goal is to convince people that you have improved the paper. It is an adversarial setting and people will generally assume you are lying if you just give empty words, especially saying you will do X by the camera ready. So the key question to ask is how can you show proof of work? Running experiments and reporting the results is one good way (or even just saying that you’ve done them).
A common piece of feedback is “this is badly written”.
This common because it’s often true! Writing papers is hard (some advice).
If you receive this feedback, try to fix it (eg give an LLM the reviews and your paper, and maybe my post, and ask it to give concrete feedback on how to improve things, along with quotes). This will improve the paper even if you don’t get in
One difficulty is that even if you improve the writing a bunch, this is hard to convince anyone of in the rebuttal, since they’re normally not willing to re-read in detail (and NeurIPS doesn’t even let you re upload).
My best strategy is to make a long changelog of what you improved, to signal high effort, and put it in a top-level comment
If the reviewer complained about a specific paragraph or section, copy in the reworded version of that
It often helps to add an appendix with a glossary for key terms, ideally both intuitive and technical definitions
I recommend the following process:
Copy all reviews to a google doc.
Go through and comment on each complaint in each review, sorting them into misunderstandings, disagreements with you, presentation issues, and technical issues—either do this while on a call, or async
Brainstorm how to address each complaint—prioritise the important ones
Write a bullet point outline, and try to get feedback
Write it up nicely and send.
You typically want to have a comment per review, and a
top level comment covering critiques from multiple reviewers.For some insane reason NeurIPS 2025 removed your ability to make a top level comment—copy and paste between reviewers I guess?Picture the area chair as your audience for the top level comment. You want to begin with a paragraph about the strengths of your paper, as noted by reviewers, supported by reviewer quotes—imagine you’re writing something the area chair can copy and paste into a meta-review about accepting you. Reviewer quotes are key for any positive claims as no one will trust you to be honest.
If one reviewer hates you, the top-level comment is a good opportunity to try to discredit them by emphasising how other reviewers disagree, as politely as possible. For example, we appreciated the constructive critique from bad reviewer that X, and have changed Y to fix it. But we are glad to see that good reviewers A and B thought Z followed by quotes supporting Z from the good reviewers, where Z contradicts X as much as possible.
Some technical complaints are best addressed by doing new experiments—you have 1-2 weeks to do this but should ask yourself how long it’ll take and whether this is the best use of time. Time is constrained and you want to maximise returns per unit time, and new experiments often take much longer than writing or conceptual rebuttals—prioritise these carefully.
Thanks for this helpful framework, it’s also useful for people who are submitting rebuttals not for the first time :). Sadly NeurIPS and ICML no longer allow a top-level comment (for silly technical reasons).
I deserved all the smoke sent my way this time lol. Next time!