Quick access:

Document Contents and Format

For the reports, we will be using the ACM conference proceedings template on Overleaf. You can create your accounts on the website and share the document with each other and work on it collaboratively.

You can also use the (university-hosted) version of Overleaf called ShareLaTeX. They do not have the official ACM template, but you can import it yourself from GitHub repository.

You can install LaTeX to your own computer (each), download the GitHub template page, and use some sort of GitHub backhand to collaborate. I would not recommend this option, especially to beginners, hence using local LaTeX and being able to compile documents is significantly hard.

Application

To apply for this seminar, please write a short application letter and submit it via Moodle. If you are applying as a group of up to three members, combine all your applications in one document and upload it only once. The document should not exceed 1 page per person (2 pages for two people, etc.). There are no other formal requirements.

Content of the application:

Literature Review Document

The literature review is the basis for your final report and your first scientific document. The goal is that by preparing it, you become familiar with your seed paper and the broader research area it belongs to. It should also get you used to scientific writing tools and style. Therefore, grading focuses more on what you found and how you present it. We want you to show that you are on a good track in terms of grounding your topic and finding supporting papers. The more advanced requirements such as structure, coherence, and storytelling have more focus (more grading points) in the final report.

Take a look at this template to get an idea of the purpose, structure, and contents of a literature review. Note however that the literature reviews in this seminar are quite short, and thus you don’t need to cover all the aspects mentioned in the template (and obviously you don’t have to justify a research proposal). What we want to see is (1) an introduction that defines your general topic and it’s scope, (2) a main part with short summaries of the seed paper and the related papers you found, and (3) a short conclusion.

The page limit for the literature review is 1 page in writing, plus 0.5 pages per additional group member. Try to get close to the limit, but do not exceed it. References are not included in this limit. Include your seed paper and at least one closely related paper for each group member.

Peer Review Document

The peer review document is free-form and should contain answers to the predetermined questions given below. The goal of this exercise is

Our review process roughly simulates that of an academic conference. At a conference, submitted papers are peer-reviewed because (a) the conference’s program committee wants to know if the paper is relevant for the conference and meets their quality standards, and (b) the authors of the paper want to understand why the paper was accepted/rejected and get feedback on how to improve their paper.

If you want to learn more about the entire academic reviewing process, you can read this recent document. Note, however, that this document is targeted at full conference papers, including the process called rebuttal, which we will not employ. We adapted the process to fit the needs of this seminar. Everything relevant for you is described in the following. Read the explanations carefully and fill out the questionnaire for the report you are reviewing. Unlike a real conference, this process is not double anonymous.

Questions to be Answered

Summary

Summarize the report. Explain the topic, related papers, key ideas, and research directions presented and discussed in the report.

One or two paragraphs. Assume that the program committee has not read the report and only relies on your summary to grasp what the report is about. This is also to show the authors that you understood what they wanted to say.

Strengths

Consider the significance of key ideas expressed, the writing quality, etc. What are the strong aspects of the document? Have the authors captured important directions around the topic and written coherent text?

Weaknesses

Consider the significance of key ideas expressed, the writing quality, etc. What are the weak aspects of the document? Have the authors clearly described the topic and the papers? Does the introduction paint a clear picture? Are there any missing points or research directions that should have been captured but are missing?

Short bullet lists are not enough; please clearly describe or cite papers you think are important but are not covered. Be constructive so the authors will be able to address your criticisms. Don’t be mean, and only critique things that you reasonably believe students are capable of doing in the time allotted for this seminar.

Rating

Based on your assessment of the strengths and weaknesses, give your overall rating of the report. If this seminar were a conference, would you accept this report and publish it here? Select from the following options and explain what the most important factors for your decision are.

Don’t worry, this rating will not directly influence the authors’ grade. On the contrary, constructive criticism can help the authors improve their reports and therefore their grades. The supervisors will make their own judgment when grading the report.

Confidence

Declare how sure you are about your assessment using the options below. Also, state in your own words how well you understood the content of the report and how familiar you are with the general field.

Additional Comments

Provide a list of detailed feedback for the report to help the authors improve it. Refer to specific paragraphs, sentences, line numbers (if available), and figures. Be constructive, i.e., help the authors understand what the issue is and what the solution could be. This includes:

Final Report Document

For your final report, you will build on what you learned during your literature research for a deeper discussion of your topic. In addition to your existing review of the literature, the report should place greater emphasis on the broader context, offer critical reflections on the selected papers and the field as a whole, and include your own ideas for addressing current challenges and advancing the state of the art. You have a lot of creative freedom here, but make sure to tell a clear and engaging story that ties your chosen papers together in a meaningful way.

We recommend using your literature review as the starting point and expanding from there. It’s totally fine to add or swap out papers if you’ve come across new ones that fit better. You can also structure the report however it works best for you, but generally, it should include:

The page limit for the final report is 6 pages in writing, plus 2 pages per additional group member. Try to get close to the limit, but do not exceed it. For the draft submission, it is okay if you haven’t reached the target length yet. You can use images and tables, but they also count towards the page limit. References and appendices are not included in this limit.

Presentation

The content of the presentation can be derived from the grading scheme below.

There are no formal requirements for your slides. You can use any template you like. Presentation times (make sure not to exceed them!):

Grading Scheme

We score your submissions to determine your final grade. Below you can find how different aspects are weighted relative to each other.

Literature Review (20/100)

(2 pt) Language and formal aspects

(9 pt) Scientific aspects

(9 pt) Argumentative aspects

Peer Review (10/100) (individual)

These are evaluated on an OK/NOT OK level. A peer review is deemed OK if you have read and understood your peer’s report and provided appropriate, specific, and helpful feedback.

Final Report (40/100)

(5 pt) Language and formal aspects

(14pt) Scientific aspects

(21pt) Argumentative aspects

Presentation (25/100)

Participation (5/100) (individual)

Preparing and asking at least 2 questions for other presentations in their Q&A sessions. All activities, such as emails with supervisors and being active in Moodle discussions.