This page will be updated with questions (and answers) as they arise. Please check back regularly. If you do not find an answer to your question here, you are welcome to contact the IJCAI-24 program chair at pcchair@ijcai24.org, but please make sure that you have read IJCAI-24 CFP and the current FAQ first.

[Many parts are inherited from FAQ of previous IJCAIs]

Reviewing FAQ

A copy of the guidelines for reproducibility <https://ijcai24.org/reproducibility/>, and the conflict of interest policy <https://www.ijcai.org/IJCAI_Conflict_of_Interest_Policy.pdf> are all available for consultation.

What is Phase 1 of reviewing?

The reviewing process consists of two phases. In Phase 1, every paper will be reviewed by two reviewers and if a paper receives two reviews that are not sufficiently positive, it will be rejected without any opportunity to submit an author response.  By submitting a paper, authors acknowledge that they are aware of the possibility of receiving a rejection notification after Phase 1.

What feedback to be expected for authors of papers rejected in Phase 1?

Authors of papers rejected in Phase 1 will receive two reviews.

What level of quality is expected from an IJCAI paper?

The IJCAI series of conferences started in 1969, and since then IJCAI has remained the premier conference bringing together the international AI community. In classic AI areas, with a long tradition at IJCAI, people submit their best papers to IJCAI.  For areas that have grown a lot in the last few years (NLP, vision, machine learning), a rule of thumb to decide whether a paper is good enough for IJCAI, is to ask whether it is good enough for conferences such as ACL / ICCV / ICML.

Can I ask questions to the authors that they should answer during the response phase?

The purpose of the rebuttal is to ask questions about specific issues that 1) could directly influence your evaluation of the paper, and 2) do not require providing new results. Typical questions include requests to clarify or justify particular issues, or about important relationships to other works.
The purpose of the rebuttal phase is not to enter a discussion about the ways the authors could extend or revise their paper by, for instance, asking for extra experiments or theoretical results. You must judge the current submission,  not the possible future versions of the paper.

Who will see the rebuttal?

The rebuttal will be visible to all members of the program committee involved in the paper (reviewers, meta-reviewers, senior meta reviewers, (associate) pc chairs).
There is also the option to enter confidential comments that will not be visible to the reviewers, but will be accessible to the meta-reviewers, senior meta reviewers, and (associate) pc chairs.

What happens to papers that are non-anonymous or overlength or misformatted?

All papers violating the rules specified in the CFP must be rejected. Most papers that obviously violated these requirements have been desk rejected before the reviewing phases. However, in case you find an obvious violation, please inform the workflow chairs (at workflow@ijcai24.org) immediately. You can also mention it in your review, though you must not take it into account in your judgment as decisions about violations are made by the PC-chair directly in order to ensure consistency and fairness.
Please refrain from using search engines such as Google to try to identify the authors.  This is not good practice.

I am relatively new to reviewing. Where can I find more information about writing high quality reviews?

Please start by looking at our review form.
We highly recommend reading Mistakes reviewers make (by professor Niklas Elmqvist), see https://niklaselmqvist.medium.com/mistakes-reviewers-make-ce3a4c595aa2
It provides a very balanced view on reviewing that applies to AI and its subfields. (One little detail: when Elmqvist writes about user study (coming from an HCI perspective), you might want to think of experiment (in AI or ML).)
The program chair also highly recommends going through the slides that Hendrik Blockeel and Jesse Davis used for a tutorial on reviewing, and that was adapted for IJCAI-ECAI 2022, and is still highly relevant for IJCAI 2024.
https://dtai.cs.kuleuven.be/introduction-to-reviewing-for-ijcai.pdf

I was assigned a paper which is outside my area of expertise. Why? And what should I do?

The assignment algorithm considers your bids, the subject areas, TPMS, as well as the reviewer workload, etc. It aims at optimizing the global assignment. If you did not bid, or entered positive bids for too few papers, or mainly on very popular papers, the algorithm may not have found the best papers for you. Still there are many other papers that need to be reviewed, and we rely on your expertise to provide an opinion.
Warning: If you see that a paper does not at all match the subject areas entered for that paper, please inform  workflow@ijcai24.org and put it in your review, as this is considered bad practice.
If a paper is somewhat outside your core area of expertise, your evaluation of the work as an informed outsider can still be useful. Nevertheless, in exceptional cases where you believe you are unable to provide any well-informed review, please inform workflow@ijcai24.org as soon as possible. We will then try to assign it to someone else. Obviously, the longer you wait, the more difficult it is to reassign. (The same holds if you discover a conflict of interest, or if you cannot review a paper for any other reason.)

Can I delegate a review to a subreviewer?

In principle, the answer is NO, unless there is really no other option because of unforeseen circumstances (like illness). The reason is that for large conferences like IJCAI, it is extremely hard to control the conflicts of interest if subreviewers are allowed. In any case it is always the PC member / reviewer  that remains responsible for the review.

I have reviewed a (re)submission for a previous conference, shall I avoid reviewing it for IJCAI 2024?

Please continue to review it. We don’t regard this situation as a COI. Reviewers for previous versions of the submission are in an ideal position to judge whether the current submission has been really improved. Note that submissions that have been rejected at any peer-reviewed conference within the past 12 months must declare the resubmission and attach the previous review as well as an explanation about how authors have improved the work. The resubmission file will be visible only after the rebuttal during the discussion period.

The paper is mainly about an application. When should we accept it?

As mentioned in the call for papers, IJCAI  2024 welcomes papers on AI techniques for novel application domains. Keep in mind though that IJCAI is the place to publish research papers.  Therefore, if the use of AI, or the AI technique used, is novel for this kind of application, it can be considered, provided that the application is formalized in such a manner that the formalization can be applied to solve other applications in an innovative manner as well.”  If the paper doesn’t completely satisfy the previous criterion but it clearly explains why the used technique works well, again it can be considered. In those cases, acceptance depends on how important the application is, how significant the improvement is, and to which extent it can be generalized to other problems. If the paper doesn’t satisfy the previous criteria but the AI technique used has a major impact in the application domain and in industry or society, it may be considered for publication at IJCAI 2024.

On the other hand, simply applying a standard AI or ML technique (like neural nets or decision trees) on standard datasets from an application domain clearly does not constitute a valid contribution for IJCAI 2024.

A preliminary version of the paper has already appeared at arXiv, or been presented at a workshop or a conference without formal proceedings?

That is not a problem as long as the papers  are presented in non-archival venues (that is, as long as there are not formally published proceedings with a publisher, in general with a DOI, ISBN, or ISSN ).  ArXiv submissions are also ok.

A short paper/extended abstract of the paper is already published. Should I recommend rejection?

No, if the short paper is not longer than a two-column two-page paper. If the ‘short’ paper is longer than that, but no longer than half of the IJCAI paper, then the extension must be significant enough.

Submissions

Can I submit a paper to IJCAI if it is already on arXiv?

Yes, you can.

A preliminary version of my paper has already been presented at a workshop or a conference without formal proceedings. Is it ok to submit it to IJCAI?

Yes. This applies more generally to papers presented in any non-archival venue (that is, as long as there are no formally published proceedings with a publisher, in general with a DOI, ISBN, or ISSN).

How strict is the author anonymity requirement for submissions?

​​IJCAI requires that all submissions be anonymized. To facilitate this, we require that the authors follow both the letter and spirit of the normal anonymization guidelines. Please ensure also that the PDF meta-data does not reveal the authors’ identities.  Any paper that obviously violates this requirement will be rejected without review.

Can I submit the title and abstract of a paper which is under review for a conference whose notification date is between January 10 and January 17?

Yes. You are allowed to submit papers where the notification will occur between the IJCAI abstract and full-paper deadlines. However, if the paper is accepted during this period you must withdraw the IJCAI submission. If the paper is rejected during this period then it is considered as a resubmission and you are required to upload the resubmission information.

My paper has been rejected from a peer-reviewed conference within the past 12  months. Do I have to attach the reviews and the response letter?

Yes. Authors must declare whether their paper has been rejected from another peer-reviewed conference in the last 12 months before the submission. They will be requested to upload the latest rejected version (anonymized if necessary) with the reviewer’s comments; a cover letter responding to the reviews is optional. 

Are there separate deadlines for supplementary file and/or resubmission file?

Yes. Please consult the dates on the conference website for this information.

When can reviewers read my resubmission information?

Your resubmission information will be visible only to reviewers after they enter their review, and after the rebuttal. 

When does a paper qualify as a student paper?

A paper is considered a ‘student paper’, and thus a potential winner of the ‘Best Student Paper Award’, when the first author is a student who, at the time of submission, has not obtained any PhD degree yet. When submitting a paper, this should be indicated by marking the appropriate checkbox.

Is there a maximal length for titles and abstracts to be submitted on the website?

Title: It is highly recommended not to go above two lines in the IJCAI format. Three lines is a strict maximum.

Abstract: There is no word limit for abstracts. However, 200 words is a nice upper limit that authors should always apply.

What does ‘abstract submission’ mean? Do I have to submit an abstract if I intend to submit a full paper? Can I submit an abstract only?

By ‘abstract submission’ we mean that you should register your paper by providing a title and a short abstract. This abstract is intended to help program committee members get papers that match their expertise. There are neither abstract-only submissions, nor any paper-only submissions: in all cases, you must submit an abstract, and then you must submit a full paper. (See abstract submission and paper submission deadlines on the website).

When submitting a paper I have to agree with the IJCAI 2024 submission policy, which says that no additional author will be allowed after the Author Information deadline has passed. Will I be allowed to change the author-order if the paper is accepted?

Yes. You will be allowed to change the author-order if the paper is accepted. You will not be allowed to add or remove authors.

What must my co-authors do if I submit to IJCAI-24?

Please ask your co-authors to log in to the submission system and fill in author information. Please make sure that you use the email address they prefer to use in CMT.   As an author or co-author, once you log in, you can see all papers under your name or account  (if you have multiple accounts/email addresses, you should check all your accounts). If an author finds any submission, for which they did not give their consent to submit, they should report to the program chair immediately.

Can I submit my paper elsewhere during the IJCAI 2024 reviewing process? 

Authors are not allowed to submit their papers to venues with formally published proceedings during the  IJCAI 2024 review period. These restrictions do not apply to workshops and similar specialized presentations with a limited audience and without published proceedings.

The CMT site for some special tracks is not open yet.  How can I submit?

The dates of  January 10th, 16th, 17th, and 24th are the key deadlines for the MAIN track. Other tracks have later deadlines. For many of these, the submission is not yet open. Please refer to the conference website for their submission deadlines.