Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support renaming files on Transfer Manager Upload #2638

Open
sydney-munro opened this issue Jul 15, 2024 · 1 comment
Open

Support renaming files on Transfer Manager Upload #2638

sydney-munro opened this issue Jul 15, 2024 · 1 comment
Labels
api: storage Issues related to the googleapis/java-storage API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@sydney-munro
Copy link
Collaborator

sydney-munro commented Jul 15, 2024

No description provided.

@product-auto-label product-auto-label bot added the api: storage Issues related to the googleapis/java-storage API. label Jul 15, 2024
@BenWhitehead BenWhitehead added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label Jul 16, 2024
@chalmagr
Copy link

Having this same issue... adding some more details

Environment details

OS type and version: MacOS 14.6.1 / Windows 11 (issue happens on both)
Java version: Temurin-11.0.19+7 (build 11.0.19+7)
Version(s): 2.42.0

Steps to reproduce

  1. Implement the shared code in the documentation to upload multiple files (https://cloud.google.com/storage/docs/uploading-objects#storage-upload-object-java)
  2. Run the code trying to upload one directory (e.g. /Users/me/source/dir) to some location in a bucket: e.g. gs://my-bucket-name/destination (directory contents are file1.txt, file2.txt) (run the code with arguments

Code example

package org.example;

import com.google.cloud.storage.transfermanager.ParallelUploadConfig;
import com.google.cloud.storage.transfermanager.TransferManager;
import com.google.cloud.storage.transfermanager.TransferManagerConfig;
import com.google.cloud.storage.transfermanager.UploadResult;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Stream;

public class Main {
    public static void main(String[] args) throws IOException {
        uploadDirectoryContents(args[0], Paths.get(args[1]), args[2]);
    }

    private static void uploadDirectoryContents(String bucketName, Path sourceDirectory, String remotePath) throws IOException {
        TransferManager transferManager = TransferManagerConfig.newBuilder().build().getService();
        ParallelUploadConfig parallelUploadConfig = ParallelUploadConfig.newBuilder()
                .setBucketName(bucketName)
                .setPrefix(remotePath)
                .build();

        List<Path> filePaths = new ArrayList<>();
        try (Stream<Path> pathStream = Files.walk(sourceDirectory)) {
            pathStream.filter(Files::isRegularFile).forEach(filePaths::add);
        }
        List<UploadResult> results = transferManager.uploadFiles(filePaths, parallelUploadConfig).getUploadResults();

        for (UploadResult result : results) {
            System.out.println(
                    "Upload for "
                            + result.getInput().getName()
                            + " completed with status "
                            + result.getStatus());
        }
    }
}

Run the above example using arguments
my-bucket-name /Users/me/source/dir destination

I would expect that the bucket contains the following files now

gs://my-bucket-name/destination/file1.txt
gs://my-bucket-name/destination/file2.txt

However, the bucket contains the following (for a MacOS scenario)

gs://my-bucket-name/destination/Users/me/source/dir/file1.txt
gs://my-bucket-name/destination/Users/me/source/dir/file2.txt

For Windows, the /Users/me/source/dir/file1.txt gets changed to C:\Users\me\source\dir\file1.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/java-storage API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

3 participants