Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

records: allow JSON API to return specific fields #860

Open
GraemeWatt opened this issue Mar 6, 2025 · 1 comment
Open

records: allow JSON API to return specific fields #860

GraemeWatt opened this issue Mar 6, 2025 · 1 comment

Comments

@GraemeWatt
Copy link
Member

GraemeWatt commented Mar 6, 2025

Feedback from @agbuckley:

Do you think it could be possible to add something like Inspire's JSON API has, where extra URL params can be used to limit the set of returned fields? They're both Invenio-based, right, so maybe this is even a shared feature? We'll want to do things like retrieve a JSON summarising all the analysis:* metadata over all records in HD, so neither we nor you actually want to be returning the full-detail version!

I don't think either HEPData or INSPIRE-HEP make much use of Invenio for their JSON APIs, but perhaps @michamos could comment on the technical implementation in the INSPIRE REST API. In the meantime, HEPData search results are limited to a maximum size of 100 per page, so it is probably not a practical issue to return all fields and leave the user to extract the relevant ones. The keyword argument of the hepdata-cli tool can alternatively be used to filter the JSON results.

@michamos
Copy link

michamos commented Mar 6, 2025

We use Invenio but this behavior is not available out of the box. We add support for fields to our search factory through https://github.com/inspirehep/inspirehep/blob/6e691efe721c7804d5a89f00f7ebfc424b2dab1f/backend/inspirehep/search/factories/search.py#L70.

It's not very complex, we basically to the following:

  1. check the content type to see if we allow fields filtering (only enabled for JSON as it can give strange results with other formats such as BibTeX)
  2. get the fields request argument
  3. pass it on to OpenSearch for _source filtering which does the heavy lifting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: To do
Development

No branches or pull requests

2 participants