Some feedback from having done a review of a pyOpenSci package.
It wasn’t clear whether comments should be inline with each review topic or just at the end. I found it easier to note comments inline, and have any additional comments at the end.
There’s a slight formatting issue when I copied the raw Markdown from the template to the editor that I used (Simplenote) and then back again, where the check boxes changed formatting. This might be how my editor handles Markdown.
It might make sense to put the JOSS requirements after the functionality requirements
There was some repetition about packaging guidelines and other categories
In terms of accessing the reviewing template and instructions, I didn’t have any particular issues. However I think it would be good if links to the reviewing instructions and templates were included in the GitHub comment (and thus the email notification) which asks the reviewer to do the review.
thank you @npch !! this is great feedback. i’ll try to compile all of this feedback for discussion in our next meeting. In the meantime @jlpalomino and chris <i don’t think chris is in the forum yet as i can’t find his username> … any thoughts to add to Neil’s comments given you also just did a review??
I second Neil’s comments and do not have any additional comments about the review process from the reviewer’s perspective. The criteria/questions for the reviewer were straight-forward and clear in terms of what the reviewer was asked to consider. Regarding the raw Markdown formatting, Chris and I used HackMD and it copied well into the editor.
I also agree with all that Neil said. My experience reviewing https://github.com/pyOpenSci/software-review/issues/7 was fine and kind of smooth, I believe the problems were more related to my lack of experience reviewing a third party software than the review process per se.
I found very useful to do the review first by opening issues upstream and then summarizing the review the in the issue in the PyOpenSci submission. That way all the actionable items I expect from the submitter are in the issues upstream, as they are closed I can just check the boxes and approve the package.
I am not super clear about what was redundant but we should definitely eliminate overlap where we can! any specific suggestions? Please feel free to submit a PR to address any of the above items as well @npch thank you for this feedback.
for being willing to pilot this new approach of adding upstream issues.
@marskar did this approach work for you? any feedback?
see htis issue where we initially discussed this approach
i think we’d like to eventually update our documentation to address how to
create these issues and
link back to them.
Filipe did this nicely but he is an github expert!! i know that most people i work with who have less experience may not be familiar with issue tracking / linking to PR’s etc!!! Thank you both for working through this with us all!!