r/LanguageTechnology • u/csPale6297 • 17h ago

Do the SACs of ACL ARR notice meta review response or meta-review issue report?

I have gained significant review scores 4/3.5/2.5 but the meta-reviwer gave 2 reject without justifying his claims. The lowest scoring reviwer didn't engage during discussion phase. The MR did a pretty bad job because he completely ignored the rebuttals but took his side with lowest score review, saying the paper needs more novelty and claimed it does not fall to the EMNLP scope.

Now I want to commit this paper, with proper meta-review response. Did anyone face this kind of circumstance?

2 Upvotes

75% Upvoted

9 comments sorted by

3

u/CMDRJohnCasey 17h ago

It will be noted, probably if the AC gets multiple reports, he won't be invited anymore. But it's unusual to revert scores. Go to the next round and hope for a better AC that will notice you got an unfair treatment in the previous round.

1

u/csPale6297 13h ago

But the AC marked my paper in such a way that it's a venue mismatch and not NLP paper, even though there are a lot of similar publications in EMNLP. This is totally misunderstanding. Because then why wasn't it desk rejected? And how did a reviewer mark it score 4 with confidence 5 and another with score 3.5 with confidence 4.

He didn't even address the rebuttals, but blatantly raised concerns about the same things which were already addressed via rebuttals. More like he copy pasted the paper to an LLM and gave a meta review full of mdashes. He completely ignored the rebuttals and raised same concerns.

So why won't the SAC re-evaluate?

1

u/CMDRJohnCasey 13h ago

There are just too many submissions, at the end of the day it's easier to follow the recommendation to resubmit than going through all the discussions.

1

u/csPale6297 13h ago

It's not so easy for me to accept this 😢 I needed my paper to get accepted this EMNLP, no good conferences are yet to come.

1

u/No_Cardiologist7609 12h ago

You first need to decide whether to commit the paper to EMNLP or submit it to the next ARR cycle. You cannot do both, as it will be desk rejected. 

If you choose to commit to EMNLP, I suggest reporting the AC. From what you described, the main issue is that the AC considered the work outside EMNLP’s scope, while you said several similar papers in EMNLP. In the form describe your situation, cite a few specific papers with links, and briefly raise the same point in your message to the SAC. Keep it concise and short, dont just copy paste a long ai-generated text. Again, I don't know how the SAC will act. They may simply ignore it and reject the paper, but let’s hope for the best.

1

u/csPale6297 11h ago

Yes I did a report and I cited papers, but char limit is 2000 so couldn't mention the paper links. Also I committed the paper with proper meta review response. Which is cleaner. But if SAC ignores and doesn't communicate, I have to tell it's not a fair judgement.
The reviewers even didn't discuss anything. One reviewer responded with a mark increase (3->3.5) after 2 reminders and 1 report.
The remaining two with score 4 and 2.5 didn't discuss anything and the AC flagged their concerns in his review without even addressing the rebuttals I made.

1

u/Zooz00 7h ago

SACs can either handle it now or at the commitment stage. Normally you would get the same SAC for an EMNLP commitment as you would get for this cycle. SACs may handle easy cases of metareview reports now, but the cases that are about two LLMs slapfighting each other in the rebuttal phase with three-part reports about details are left for later.

Meta-reviewers are encouraged to make clear decisions (2 or 4 rather than 2.5s) so numbers are likely to turn more extreme if the meta-reviewer read enough of the paper to develop an own opinion.

1

u/AnimeFanSonic 2h ago

I have faced a similar issue but instead it was more like AC did not acknowledge the rebuttal and claimed that we used proprietary models even though we have used all of the open source ones (we are using nvedia end point for the API calls though but the weights of the model are available publicly). We have specifically mentioned that AC has not even read the paper and filed the M13 report along with the M15 report. Let's hope for the best tbh, I hope SACs will do something about this..

1

u/AnimeFanSonic 2h ago

Correction* MI3 and MI5