I’ve been working on a command-line tool for working with Figshare
and I’ve kind of hit a brick wall; I tweeted about it (https://twitter.com/nicholdav/status/1177593328780677126) and @lwasser suggested I bring the discussion here.
So, any all feedback welcome: do you think this would be useful, does it fall under the PyOpenSci umbrella, can meet the requirements for review?
To start with, there is an RFigshare tool that’s part of ROpenSci:
and it (admirably and as expected since it’s part of ROpenSci) meets all the review criteria, tests, docs, CI, etc.
The goal for any figshare command-line tool, I think, should be to make interfacing with Figshare transparent to the user as much as possible.
The ROpenSci package achieves this, basically by pretending to be a web app, i.e. like when you sign in to Discourse using your GitHub credentials. In other words, to interface with the Figshare API, the RFigshare devs pretend that they are a web app, and use the OAuth web app workflow to get authorization from the user to work with their data.
I’ll come back to this below, but let me say there’s a couple existing Python packages that I found very helpful in getting me started
I think neither is actively maintained, based on this discussion
and neither has tests/non-docstring docs/continuous integration
I also remembered upon searching PyPI again (https://pypi.org/search/?q=figshare) that there’s also this package:
which looks like it might actually solve some of the issues that I’m having … but the README says it’s only been tested with institutional figshare and mostly for downloading data. (I want to upload data for sure, and not require the user to have an institutional account.)
And that brings me to issues I’m having:
Since these are command-line tools, not full-blown web apps, they require some hacky tricks to work with the OAuth web app workflow:
- pop open a browser, and get the user to authorize the command-line tool
- in OAuth 2.0, somehow get the authorization token returned by Figshare from a redirect url
This is what (the now deprecated) OAuth2Client does:
and people have used this approach in scripts
This hacky approach to having a command-line tool pass as a web app works because you can specify the redirect url as ‘http://localhost/8080’
The problem is that Figshare requires the redirect url to be https not http.
Notice this isn’t a problem for the figshare packagr from rmcgibbo and RFigshare, because they still use the OAuth 1.0 workflow, which I thought Figshare maintains for backwards compatability.
I thought it would make sense to use an OAuth 2.0 workflow (not 1.0) since that is what the most current Figshare API uses, and it should be more convenient / secure, but it turns out to not be obvious how to get this to work for a command-line tool, esp. for someone like me that knows little about networking.
My (currently failing) approach is here:
Something is going wrong, I think, because the redirect url is https and not http, which breaks the hacky ways of getting tokens out of the actual redirect that Figshare performs. I can provide more detail about the errors I’m getting if that would clarify things.
An alternative would be for users to get their own tokens and work with the tool that way.
FigShare has added this ability for users to generate tokens without declaring an application.
The RFigshare package allows for this option–see in that fs_auth.R code I linked above.
I think it’s good to provide this option, but I think only having this option would limit the number of users, because it adds friction–requiring users to know what tokens are, how to get them, how to provide them to the tool–so I would prefer something like the web app authorization workflow.
Again, any feedback welcome.
Sorry for the overly detailed bits about what’s not working, but as Leah suggested I’m hoping to bring other (more experienced) devs into the conversation and get some feedback on that too.