Merge remote-tracking branch 'IQSS/develop' into QDR-solr_and_libs_updates

qqmyers · qqmyers · commit 105dd1b7f8f4 · 2025-02-18T13:56:28.000-05:00
diff --git a/doc/release-notes/11256-news-on-support-for-external-vocabulary.md b/doc/release-notes/11256-news-on-support-for-external-vocabulary.md
@@ -0,0 +1,4 @@
+### News on Support for External Vocabulary Services
+
+It is now possible to fill Dataverse keywords metadata using an OntoPortal service.
+The code has been shared on [GDCC GitHub Repository](https://github.com/gdcc/dataverse-external-vocab-support#scripts-in-production).
diff --git a/doc/release-notes/6467-optimize-permission-lookups-for-a-user.md b/doc/release-notes/6467-optimize-permission-lookups-for-a-user.md
@@ -0,0 +1,9 @@
+The following API have been added:
+
+/api/users/{identifier}/allowedCollections/{permission}
+
+This API lists the dataverses/collections that the user has access to via the permission passed.
+By passing "any" as the permission the list will return all dataverse/collections that the user can access regardless of which permission is used.
+This API can be executed only by the User requesting their own list of accessible collections or by an Administrator.
+Valid Permissions are: AddDataverse, AddDataset, ViewUnpublishedDataverse, ViewUnpublishedDataset, DownloadFile, EditDataverse, EditDataset, ManageDataversePermissions,
+ManageDatasetPermissions, ManageFilePermissions, PublishDataverse, PublishDataset, DeleteDataverse, DeleteDatasetDraft, and "any" as a wildcard option.
diff --git a/doc/release-notes/guides-toc-depth.md b/doc/release-notes/guides-toc-depth.md
@@ -0,0 +1,3 @@
+### Guides Table of Contents Depth
+
+On the home page for each guide (User Guide, etc.) there was an overwhelming amount of information in the form of a deeply nested tabled of contents. The depth of the table of contents has been reduced to two levels, making the home page for each guide more readable. See #11166.
diff --git a/doc/sphinx-guides/source/admin/index.rst b/doc/sphinx-guides/source/admin/index.rst
@@ -11,6 +11,7 @@ This guide documents the functionality only available to superusers (such as "da
 **Contents:**
 
 .. toctree::
+   :maxdepth: 2
 
    dashboard
    external-tools
diff --git a/doc/sphinx-guides/source/api/index.rst b/doc/sphinx-guides/source/api/index.rst
@@ -9,6 +9,7 @@ API Guide
 **Contents:**
 
 .. toctree::
+   :maxdepth: 2
 
    intro
    getting-started
@@ -24,4 +25,4 @@ API Guide
    linkeddatanotification
    apps
    faq
-   changelog
+   changelog
diff --git a/doc/sphinx-guides/source/api/native-api.rst b/doc/sphinx-guides/source/api/native-api.rst
@@ -6501,6 +6501,27 @@ Example: List permissions a user (based on API Token used) has on a dataset whos
 
   curl -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/admin/permissions/:persistentId?persistentId=$PERSISTENT_IDENTIFIER"
 
+List Dataverse collections a user can act on based on their permissions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+List Dataverse collections a user can act on based on a particular permission ::
+
+    GET http://$SERVER/api/users/$identifier/allowedCollections/$permission
+
+.. note:: This API can only be called by an Administrator or by a User requesting their own list of accessible collections.
+
+The ``$identifier`` is the username of the requested user.
+The ``$permission`` is the permission (tied to the roles) that gives the user access to the collection.
+Passing ``$permission`` as 'any' will return the collection as long as the user has any access/permission on the collection
+
+.. code-block:: bash
+
+  export SERVER_URL=https://demo.dataverse.org
+  export $USERNAME=jsmith
+  export PERMISSION=PublishDataverse
+
+  curl -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/users/$USERNAME/allowedCollections/$PERMISSION"
+
 Show Role Assignee
 ~~~~~~~~~~~~~~~~~~
 
diff --git a/doc/sphinx-guides/source/container/index.rst b/doc/sphinx-guides/source/container/index.rst
@@ -4,6 +4,7 @@ Container Guide
 **Contents:**
 
 .. toctree::
+  :maxdepth: 2
 
   intro
   running/index
diff --git a/doc/sphinx-guides/source/contributor/index.md b/doc/sphinx-guides/source/contributor/index.md
@@ -4,7 +4,7 @@ Thank you for your interest in contributing to Dataverse!  We are open to contri
 
 ```{contents} Contents:
 :local:
-:depth: 3
+:depth: 2
 ```
 
 ## Ideas and Feature Requests
diff --git a/doc/sphinx-guides/source/developers/index.rst b/doc/sphinx-guides/source/developers/index.rst
@@ -9,6 +9,7 @@ Developer Guide
 **Contents:**
 
 .. toctree::
+   :maxdepth: 2
 
    intro
    dev-environment
diff --git a/doc/sphinx-guides/source/installation/index.rst b/doc/sphinx-guides/source/installation/index.rst
@@ -9,6 +9,7 @@ Installation Guide
 **Contents:**
 
 .. toctree::
+   :maxdepth: 2
 
    intro
    prep
diff --git a/doc/sphinx-guides/source/qa/index.md b/doc/sphinx-guides/source/qa/index.md
@@ -1,6 +1,8 @@
 # QA Guide
 
 ```{toctree}
+:caption: "Contents:"
+:maxdepth: 2
 overview.md
 testing-approach.md
 testing-infrastructure.md
diff --git a/doc/sphinx-guides/source/style/index.rst b/doc/sphinx-guides/source/style/index.rst
@@ -11,6 +11,7 @@ This style guide is meant to help developers implement clear and appropriate UI
 **Contents:**
 
 .. toctree::
+   :maxdepth: 2
 
    foundations
    patterns
diff --git a/doc/sphinx-guides/source/user/index.rst b/doc/sphinx-guides/source/user/index.rst
@@ -9,6 +9,7 @@ User Guide
 **Contents:**
 
 .. toctree::
+   :maxdepth: 2
 
    account
    find-use-data
diff --git a/pom.xml b/pom.xml
@@ -492,7 +492,7 @@
         <dependency>
             <groupId>com.nimbusds</groupId>
             <artifactId>oauth2-oidc-sdk</artifactId>
-            <version>10.13.2</version>
+            <version>11.22.1</version>
         </dependency>
         <!-- Caching library, current main use case is for OIDC authentication -->
         <dependency>
diff --git a/src/main/java/edu/harvard/iq/dataverse/PermissionServiceBean.java b/src/main/java/edu/harvard/iq/dataverse/PermissionServiceBean.java
@@ -2,13 +2,14 @@
 
 import edu.harvard.iq.dataverse.authorization.AuthenticationServiceBean;
 import edu.harvard.iq.dataverse.authorization.DataverseRole;
+import edu.harvard.iq.dataverse.authorization.groups.impl.ipaddress.ip.IPv4Address;
+import edu.harvard.iq.dataverse.authorization.groups.impl.ipaddress.ip.IPv6Address;
+import edu.harvard.iq.dataverse.authorization.groups.impl.ipaddress.ip.IpAddress;
 import edu.harvard.iq.dataverse.authorization.providers.builtin.BuiltinUserServiceBean;
 import edu.harvard.iq.dataverse.authorization.users.GuestUser;
 import edu.harvard.iq.dataverse.authorization.Permission;
 import edu.harvard.iq.dataverse.authorization.RoleAssignee;
-import edu.harvard.iq.dataverse.authorization.groups.Group;
 import edu.harvard.iq.dataverse.authorization.groups.GroupServiceBean;
-import edu.harvard.iq.dataverse.authorization.groups.GroupUtil;
 import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser;
 import edu.harvard.iq.dataverse.authorization.users.User;
 import edu.harvard.iq.dataverse.engine.command.Command;
@@ -37,7 +38,6 @@
 import java.util.Collections;
 import java.util.HashMap;
 import java.util.LinkedList;
-import java.util.logging.Level;
 import java.util.stream.Collectors;
 import static java.util.stream.Collectors.toList;
 import jakarta.persistence.Query;
@@ -100,6 +100,70 @@ public class PermissionServiceBean {
     @Inject
     DatasetVersionFilesServiceBean datasetVersionFilesServiceBean;
 
+    private static final String LIST_ALL_DATAVERSES_USER_HAS_PERMISSION = """
+            WITH grouplist AS (
+                  SELECT explicitgroup_authenticateduser.explicitgroup_id as id FROM explicitgroup_authenticateduser
+                  WHERE explicitgroup_authenticateduser.containedauthenticatedusers_id = @USERID
+                )
+                        
+                SELECT * FROM DATAVERSE WHERE id IN (
+                  SELECT definitionpoint_id
+                  FROM roleassignment
+                  WHERE roleassignment.assigneeidentifier IN (
+                    SELECT CONCAT('&explicit/', explicitgroup.groupalias) as assignee
+                    FROM explicitgroup
+                    WHERE explicitgroup.id IN (
+                      (
+                      SELECT explicitgroup.id id
+                      FROM explicitgroup
+                      WHERE EXISTS (SELECT id FROM grouplist WHERE id = explicitgroup.id)
+                      ) UNION (
+                      SELECT explicitgroup_explicitgroup.containedexplicitgroups_id id
+                      FROM explicitgroup_explicitgroup
+                      WHERE EXISTS (SELECT id FROM grouplist WHERE id = explicitgroup_explicitgroup.explicitgroup_id)
+                      AND EXISTS (SELECT id FROM dataverserole
+                        WHERE dataverserole.id = roleassignment.role_id AND (dataverserole.permissionbits & @PERMISSIONBIT !=0))
+                      )
+                    )
+                  ) UNION (
+                    SELECT definitionpoint_id
+                    FROM roleassignment
+                    WHERE roleassignment.assigneeidentifier = (
+                      SELECT CONCAT('@', authenticateduser.useridentifier)
+                      FROM authenticateduser
+                      WHERE authenticateduser.id = @USERID)
+                        AND EXISTS (SELECT id FROM dataverserole
+                        WHERE dataverserole.id = roleassignment.role_id AND (dataverserole.permissionbits & @PERMISSIONBIT !=0))
+                  ) UNION (
+                     SELECT definitionpoint_id
+                     FROM roleassignment
+                     WHERE roleassignment.assigneeidentifier = ':authenticated-users'
+                       AND EXISTS (SELECT id FROM dataverserole
+                       WHERE dataverserole.id = roleassignment.role_id AND (dataverserole.permissionbits & @PERMISSIONBIT !=0))
+                  ) UNION (
+                     SELECT definitionpoint_id
+                     FROM roleassignment
+                     WHERE roleassignment.assigneeidentifier IN (
+                       SELECT CONCAT('&shib/', persistedglobalgroup.persistedgroupalias) as assignee
+                       FROM persistedglobalgroup
+                       WHERE dtype = 'ShibGroup'
+                       AND EXISTS (SELECT id FROM dataverserole WHERE dataverserole.id = roleassignment.role_id AND (dataverserole.permissionbits & @PERMISSIONBIT !=0))
+                     )
+                  ) UNION (
+                     SELECT definitionpoint_id
+                     FROM roleassignment
+                     WHERE roleassignment.assigneeidentifier IN (
+                       SELECT CONCAT('&ip/', persistedglobalgroup.persistedgroupalias) as assignee
+                       FROM persistedglobalgroup
+                       LEFT OUTER JOIN ipv4range ON persistedglobalgroup.id = ipv4range.owner_id
+                       LEFT OUTER JOIN ipv6range ON persistedglobalgroup.id = ipv6range.owner_id
+                       WHERE dtype = 'IpGroup'
+                       AND EXISTS (SELECT id FROM dataverserole WHERE dataverserole.id = roleassignment.role_id AND (dataverserole.permissionbits & @PERMISSIONBIT !=0))
+                       AND @IPRANGESQL
+                     )
+                  )
+                )
+            """;
     /**
      * A request-level permission query (e.g includes IP ras).
      */
@@ -553,36 +617,6 @@ public RequestPermissionQuery request(DataverseRequest req) {
         return new RequestPermissionQuery(null, req);
     }
 
-    /**
-     * Go from (User, Permission) to a list of Dataverse objects that the user
- has the permission on.
-     *
-     * @param user
-     * @param permission
-     * @return The list of dataverses {@code user} has permission
- {@code permission} on.
-     */
-    public List<Dataverse> getDataversesUserHasPermissionOn(AuthenticatedUser user, Permission permission) {
-        Set<Group> groups = groupService.groupsFor(user);
-        String identifiers = GroupUtil.getAllIdentifiersForUser(user, groups);
-        /**
-         * @todo Are there any strings in identifiers that would break this SQL
-         * query?
-         */
-        String query = "SELECT id FROM dvobject WHERE dtype = 'Dataverse' and id in (select definitionpoint_id from roleassignment where assigneeidentifier in (" + identifiers + "));";
-        logger.log(Level.FINE, "query: {0}", query);
-        Query nativeQuery = em.createNativeQuery(query);
-        List<Integer> dataverseIdsToCheck = nativeQuery.getResultList();
-        List<Dataverse> dataversesUserHasPermissionOn = new LinkedList<>();
-        for (int dvIdAsInt : dataverseIdsToCheck) {
-            Dataverse dataverse = dataverseService.find(Long.valueOf(dvIdAsInt));
-            if (userOn(user, dataverse).has(permission)) {
-                dataversesUserHasPermissionOn.add(dataverse);
-            }
-        }
-        return dataversesUserHasPermissionOn;
-    }
-
     public List<AuthenticatedUser> getUsersWithPermissionOn(Permission permission, DvObject dvo) {
         List<AuthenticatedUser> usersHasPermissionOn = new LinkedList<>();
         Set<RoleAssignment> ras = roleService.rolesAssignments(dvo);
@@ -888,4 +922,46 @@ private boolean hasUnrestrictedReleasedFiles(DatasetVersion targetDatasetVersion
         Long result = em.createQuery(criteriaQuery).getSingleResult();
         return result > 0;
     }
+
+    public List<Dataverse> findPermittedCollections(DataverseRequest request, AuthenticatedUser user, Permission permission) {
+        return findPermittedCollections(request, user, 1 << permission.ordinal());
+    }
+    public List<Dataverse> findPermittedCollections(DataverseRequest request, AuthenticatedUser user, int permissionBit) {
+        if (user != null) {
+            // IP Group - Only check IP if a User is calling for themself
+            String ipRangeSQL = "FALSE";
+            if (request != null
+                    && request.getAuthenticatedUser() != null
+                    && request.getSourceAddress() != null
+                    && request.getAuthenticatedUser().getUserIdentifier().equalsIgnoreCase(user.getUserIdentifier())) {
+                IpAddress ip = request.getSourceAddress();
+                if (ip instanceof IPv4Address) {
+                    IPv4Address ipv4 = (IPv4Address) ip;
+                    ipRangeSQL = ipv4.toBigInteger() + " BETWEEN ipv4range.bottomaslong AND ipv4range.topaslong";
+                } else if (ip instanceof IPv6Address) {
+                    IPv6Address ipv6 = (IPv6Address) ip;
+                    long[] vals = ipv6.toLongArray();
+                    if (vals.length == 4) {
+                        ipRangeSQL = """
+                                (@0 BETWEEN ipv6range.bottoma AND ipv6range.topa
+                                AND @1 BETWEEN ipv6range.bottomb AND ipv6range.topb
+                                AND @2 BETWEEN ipv6range.bottomc AND ipv6range.topc
+                                AND @3 BETWEEN ipv6range.bottomd AND ipv6range.topd)
+                                """;
+                        for (int i = 0; i < vals.length; i++) {
+                            ipRangeSQL = ipRangeSQL.replace("@" + i, String.valueOf(vals[i]));
+                        }
+                    }
+                }
+            }
+
+            String sqlCode = LIST_ALL_DATAVERSES_USER_HAS_PERMISSION
+                    .replace("@USERID", String.valueOf(user.getId()))
+                    .replace("@PERMISSIONBIT", String.valueOf(permissionBit))
+                    .replace("@IPRANGESQL", ipRangeSQL);
+            return em.createNativeQuery(sqlCode, Dataverse.class).getResultList();
+        }
+        return null;
+    }
 }
+
diff --git a/src/main/java/edu/harvard/iq/dataverse/api/Users.java b/src/main/java/edu/harvard/iq/dataverse/api/Users.java
@@ -270,6 +270,29 @@ public Response getTracesElement(@Context ContainerRequestContext crc, @Context
         }
     }
 
+    @GET
+    @AuthRequired
+    @Path("{identifier}/allowedCollections/{permission}")
+    @Produces("application/json")
+    public Response getUserPermittedCollections(@Context ContainerRequestContext crc, @Context Request req, @PathParam("identifier") String identifier, @PathParam("permission") String permission) {
+        AuthenticatedUser authenticatedUser = null;
+        try {
+            authenticatedUser = getRequestAuthenticatedUserOrDie(crc);
+            if (!authenticatedUser.getUserIdentifier().equalsIgnoreCase(identifier) && !authenticatedUser.isSuperuser()) {
+                return error(Response.Status.FORBIDDEN, "This API call can be used by Users getting there own permitted collections or by superusers.");
+            }
+        } catch (WrappedResponse ex) {
+            return error(Response.Status.UNAUTHORIZED, "Authentication is required.");
+        }
+        try {
+            AuthenticatedUser userToQuery = authSvc.getAuthenticatedUser(identifier);
+            JsonObjectBuilder jsonObj = execCommand(new GetUserPermittedCollectionsCommand(createDataverseRequest(getRequestUser(crc)), userToQuery, permission));
+            return ok(jsonObj);
+        } catch (WrappedResponse ex) {
+            return ex.getResponse();
+        }
+    }
+
     @POST
     @Path("register")
     public Response registerOIDCUser(String body) {
diff --git a/src/main/java/edu/harvard/iq/dataverse/api/datadeposit/SWORDv2ServiceDocumentServlet.java b/src/main/java/edu/harvard/iq/dataverse/api/datadeposit/SWORDv2ServiceDocumentServlet.java
@@ -1,6 +1,8 @@
 package edu.harvard.iq.dataverse.api.datadeposit;
 
 import java.io.IOException;
+
+import edu.harvard.iq.dataverse.engine.command.DataverseRequest;
 import jakarta.inject.Inject;
 import jakarta.servlet.ServletException;
 import jakarta.servlet.http.HttpServletRequest;
@@ -29,6 +31,7 @@ public void init() throws ServletException {
 
     @Override
     protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
+        serviceDocumentManagerImpl.setIpAddress((new DataverseRequest(null, req)).getSourceAddress());
         this.api.get(req, resp);
     }
 
diff --git a/src/main/java/edu/harvard/iq/dataverse/api/datadeposit/ServiceDocumentManagerImpl.java b/src/main/java/edu/harvard/iq/dataverse/api/datadeposit/ServiceDocumentManagerImpl.java
diff --git a/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/GetUserPermittedCollectionsCommand.java b/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/GetUserPermittedCollectionsCommand.java
diff --git a/src/test/java/edu/harvard/iq/dataverse/api/IpGroupsIT.java b/src/test/java/edu/harvard/iq/dataverse/api/IpGroupsIT.java
diff --git a/src/test/java/edu/harvard/iq/dataverse/api/UsersIT.java b/src/test/java/edu/harvard/iq/dataverse/api/UsersIT.java
diff --git a/src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java b/src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,3 @@`
	`1`	`+### Guides Table of Contents Depth`
	`2`	`+`
	`3`	`+On the home page for each guide (User Guide, etc.) there was an overwhelming amount of information in the form of a deeply nested tabled of contents. The depth of the table of contents has been reduced to two levels, making the home page for each guide more readable. See #11166.`