Peer Review Instructions

Here you can also download following text as a pdf.

Suggestions and constructive criticism are vital for everybody’s work. Coming from a group with same or similar interests (so called “peers”) they can even be more valuable. For several years now we have been using peer feedback as an effective learning method – and we hope to inspire you to use it as an instrument for quality assurance in science.
Following instructions can also be found in the submission process. We will provide you with all relevant information so you will know what to expect after submission.

Please upload your finished article no later than the given deadline. As for the second stage, you will rate and comment two articles of each of your fellow students according to the criteria described below. After the peer review you will exchange all received ratings and valuations with your team and use the comments to improve your work. Results of the peer review will also be used for the conference contribution selection in February. However, they do not represent the final evaluation.

Conditions of submission:
Upload the finished solution of the task as a .pdf (max. 20 MB). Every student is asked to submit a document even if they worked on it in a team. Only one pdf document is allowed for each student to ensure equal and comparative assessment. Upload the pdf after reading all the following instructions.

Anonymity of submissions:
This review is limited regarding anonymity. To ensure fair evaluation please do not disclose your identity in the document! Remove all traces e.g. names, location or time which could lead to the revelation of your identity. The system will keep your identity safe. Please also anonymize the name of the pdf document.

Criteria of evaluation:

  • Content quality and information content:
    (concept, scientific linking/embedding, completeness) Please provide a summary of the following questions before evaluation of this criterion:
    . What is the paper about?
    . What are the main points?
    . What are paper’s strengths and weaknesses?
    Afterwards assign points to the work for this criterion – 6 stands for excellent work and important contribution. 5 – good work. 4 – quite important. 3 – not insignificant. 2 – rather weak significance. 1 – questionable work. 0 – absolutely no relevance.
  • Relevance for theory and practice:
    What is the added value for practitioners or for scientific progress?
    6 – outstanding importance. 5 – big importance. 4 – quite important. 3 – not insignificant. 2 – rather weak significance. 1 – of little importance. 0 – absolutely no relevance.
  • Originality and degree of innovation:
    What is the paper’s degree of innovation?
    6 – pioneering. 5 – work that paves the way. 4 – one step ahead. 3 – shows status quo. 2 – better work under that topic exists. 1 – already mentioned frequently. 0 – outdated.
  • Quality of presentation:
    writing style, comprehensibility, presentation of content/dramaturgy
    6 – excellent writing. 5 – good writing. 4 – readable. 3 – should be revised. 2 – needs considerable work done. 1 – more than half of the work should be revised. 0 – inacceptable.
  • Structure and form of presentation:
    scientifically oriented structure, balance of text and images, consistent structure, standardized formatting and citation, visual quality of graphics and images, readability
    6 – perfect and exemplary. 5 – almost perfect and appealing. 4 – few abnormalities. 3 – some abnormalities or quality issues. 2 – several errors. 1 – great number of errors. 0 – uncaring.
  • General recommendation for publication:
    Should the contribution be published?
    6 – definitely accept (very high quality). 5 – accept (good quality). 4 – likely accept (acceptable quality). 3 – only accept after revision (rather low quality). 2 – likely reject (low quality). 1 – reject (insufficient quality). 0 – definitely reject (insufficient work).
    Which suggestions for improvement do you have for the sections discussion and conclusion?

Two Peer Reviews
For this task, you are being asked to review two student contributions aside from submitting your own draft. For the evaluation, you have to check the criteria. Please make an effort to evaluate the work accurately and equitably after reading the submitted document carefully. Pay attention to giving constructive and preferably positive feedback and to point out suggestions for improvement. Remember that the student is in the same learning situation as you are. In the comment, you should substantiate your assessment.

Grading of task
Your submission will be evaluated with a maximum of 70 out of 100 points according to set criteria. Check for fulfillment of criteria before submission. For the fair evaluation of two other contributions you will receive 30 out of 100 points. Gradings (randomly chosen) can be moderated by lecturers or tutors especially if the individual evaluations differ strongly. The evaluation process is usually finished within one week of submission.

Use of feedback
Use the received feedback and helpful comments for the revision of your paper! Exchange the received reviews in your team and discuss the outcome. Use the criteria for revision and for the presentation at the conference!

Wishing you success
Elisabeth Katzlinger, Martin Stabauer, Michael A. Herzog & Leonore Franz.

Peer Assessment

Peer Review is an established tool used in journals and scientific magazines. Its benefits are widely accepted [1]. Also, in higher education settings, it has been widely used across all disciplines [2]. Along with the shift from pure content delivery to competency building, Peer Review trains mainly Blooms cognitive domains synthesis, analysis, and evaluation [3, 4] as it especially helps to improve students’ assessment and reflective skills. Therefore, Peer Review is a catalyst for improved self-assessment [5].

In a joint Master’s class with students from four courses at three universities instructed by four teachers, interuniversity student groups are expected to write a scientific paper within one semester. The class consists of students from the following Master programs: Digital Business Management (Johannes Kepler University and University of Applied Sciences Upper Austria), Cross Media and Risk Management (both from Magdeburg-Stendal University). After handing in the first draft of their finals paper the student groups receive feedback from their class peers. This Peer Review is an indispensable course sequence, as it is most valuable to the students learning process.

Most of the group work is done (collaboratively) online, so is the Peer Review. By submitting their group paper via the workshop activity on the LMS platform Moodle each student is randomly allocated to be reviewed and graded by other participants of the course. In the end, every student has to anonymously evaluate two papers which made up about six reviews for each paper. Students have to rate and feedback according to the:

  • content quality and information content,
  • relevance for theory and practice,
  • originality and degree of innovation,
  • quality of presentation,
  • structure and form of presentation,

and were asked to give a general recommendation for publication.

The next step for the students was to incorporate peer feedback before they handed in their papers for final submission. Our research confirms, that the revision phase is highly beneficial to the learning process when student groups have the opportunity to improve their papers.

 

 

  • Kelly, S. Tara, and A. Khosrow, “Peer Review in Scientific Publications: Benefits, Critiques, & A Survival Guide.” EJIFCC25.3, 2014, pp. 227–243.
  • M. Ashenafi, “Peer-assessment in higher education – twenty-first century practices, challenges and the way forward” in Assessment & Evaluation in Higher Education Vol. 42, Iss. 2, 2017, pp. 226-251.
  • W. Anderson, et al., “A taxonomy for learning, teaching and assessing: A revision of Bloom’s taxonomy.” New York. Longman Publishing. Artz, AF, & Armour-Thomas, E., 2001.
  • Nicol, A. Thomson, C. Breslin, “Rethinking feedback practices in higher education: a peer review perspective” in Assessment & Evaluation in Higher Education Vol. 39, Iss. 1, 2014, pp. 102-122.
  • Reinholz,“The assessment cycle: a model for learning through peer assessment”a Center for STEM Learning, University of Colorado, Boulder, CO, USA, 2015.