STEP 5: Grading & Quality Control (Optional)
Grading contributions to OSM can be a challenge. If participating in a graded assignment, students must be compensated fairly for their work. Graded or not, instructors should always be conscious that the quality of work contributed to OSM is of a high standard.
One way to tackle these concerns is to apply a rubric. For our rubric, we suggest grading each student on Completeness, Quality and Coverage. The suggested rubirc is featured here:
- Completeness: The overall grade for the assignment is weighted on the completeness of the task. If only half the assigned work is completed it will be weighted at 50%.
- Quality: This element grades the overall quality of the tracing, for instance, are the corners of the buildings square, and do they match the building footprint accurately, or, are roads connected properly.
- Coverage: Did the student exhaust all of the possible features in the area/cell they chose? We want to avoid gaps in the infrastructure in OSM. Sometimes students will move from cell to cell, and essentially “cherrypick” the easiest features in each. Assigning a grade for good coverage disencourages this activity.
By enforcing a rubric like this, you can ensure higher quality work from your students and to the OSM community.
You can download an excel version of this sample rubic. Please feel free to customize according to your project needs.
To help manage and monitor the contributions to OSM, the website Overpass-Turbo can be used. This website can identify the contributions to OSM by a specific student by searching for that students OSM display name. To do so, follow the steps below:
- Go to http://overpass-turbo.eu/
- Zoom to the area you wish to check
- Use the Wizard to search for the student’s OSM display name (user:osmname)
- The number of points, lines and polygons created by the user will be noted in the bottom right corner and the features will be highlighted on the screen.
Grading Workflow using Overpass Turbo and JOSM (Java OpenStreetMap Editor)
It is very difficult to individually assess student work by simply panning through the live OSM map, or even identifying their individual work on Overpass Turbo. We have developed a multi-step grading workflow that makes use of:
- Overpass Turbo
- JOSM (Java OpenStreetMap Editor)
- Grading Rubric (above)
If you are new to JOSM (Java OpenStreetMap Editor), please see the LearnOSM site for instructions on how to use it.
INSERT GRADING WORKFLOW INSTRUCTIONS HERE
Informal Quality Control
Conveying the notion of quality control and quality assurance is a large part of mapping. Even if you are not working in a traditional classroom setting, or some place where the idea of grading is inappropriate, you must still take the quality of the traced map material into consideration. Consider spot checking the groups work and/or a peer review of the work completed by the group with students fixing mistakes they find.