uploader.finish() cause exception

[softboy99]

io.tus.java.client.ProtocolException: response contains different Upload-Offset value (1024) than expected (51778)

public static void upload(Context context, Bitmap bitmap, FileUploadInfo fileUploadInfo) throws IOException, ProtocolException {
        // Create a new TusClient instance
        TusClient client = new TusClient();
        Map<String,String> headers = new HashMap<>();
        headers.put("Authorization",OkHttpUtils.accessToken==null?"":"Bearer " + OkHttpUtils.accessToken.getToken());
        headers.put("uploadType","企业");
        headers.put("entityName",fileUploadInfo.getEntityName());
        headers.put("fieldName",fileUploadInfo.getFieldName());
        headers.put("keyName",fileUploadInfo.getKeyName());
        headers.put("keyValue",fileUploadInfo.getKeyValue());
        headers.put("fileName",fileUploadInfo.getKeyValue());
        headers.put("deviceNo","andersdevice");
        headers.put("operId", String.valueOf(App.currentUser.getId()));
        client.setHeaders(headers);
        // Configure tus HTTP endpoint. This URL will be used for creating new uploads
        // using the Creation extension
        client.setUploadCreationURL(new URL(API_BASE_URL+"upload"));

        // Enable resumable uploads by storing the upload URL in the preferences
        // and preserve them after app restarts
        SharedPreferences pref = PreferenceManager.getDefaultSharedPreferences(context);
        client.enableResuming(new TusPreferencesURLStore(pref));

        // Open a file using which we will then create a TusUpload. If you do not have
        // a File object, you can manually construct a TusUpload using an InputStream.
        // See the documentation for more information.

        final TusUpload upload = new TusUpload();
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        bitmap.compress(Bitmap.CompressFormat.JPEG, 100, baos);
        byte[] bArr = baos.toByteArray();
        InputStream inputStream = new ByteArrayInputStream(bArr);
        upload.setInputStream(inputStream);
        upload.setFingerprint(fileUploadInfo.getKeyValue());
        upload.setSize(bArr.length);
        System.out.println("Starting upload...");

        // We wrap our uploading code in the TusExecutor class which will automatically catch
        // exceptions and issue retries with small delays between them and take fully
        // advantage of tus' resumability to offer more reliability.
        // This step is optional but highly recommended.
        final TusExecutor executor = new TusExecutor() {
            @Override
            protected void makeAttempt() throws ProtocolException, IOException {
                // First try to resume an upload. If that's not possible we will create a new
                // upload and get a TusUploader in return. This class is responsible for opening
                // a connection to the remote server and doing the uploading.
                TusUploader uploader = client.resumeOrCreateUpload(upload);
                // Upload the file in chunks of 1KB sizes.
                uploader.setChunkSize(1024);
                // Upload the file as long as data is available. Once the
                // file has been fully uploaded the method will return -1
                do {
                    // Calculate the progress using the total size of the uploading file and
                    // the current offset.
                    long totalBytes = upload.getSize();
                    long bytesUploaded = uploader.getOffset();
                    double progress = (double) bytesUploaded / totalBytes * 100;

                    System.out.printf("Upload at %06.2f%%.\n", progress);
                } while(uploader.uploadChunk() > -1);

                // Allow the HTTP connection to be closed and cleaned up
                uploader.finish();

                System.out.println("Upload finished.");
                System.out.format("Upload available at: %s", uploader.getUploadURL().toString());
            }
        };
        Thread t = new Thread(()-> {
            try {
                executor.makeAttempts();
            } catch (ProtocolException e) {
                e.printStackTrace();
            } catch (IOException e) {
                e.printStackTrace();
            }
        });
        t.start();
    }

[Acconut]

What tus server are you using?

[softboy99]

tus-java-serverhttps://github.com/tomdesair/tus-java-server

[Acconut]

Ok, does this error also occur when you upload to https://master.tus.io/files/ (notice the trailing slash)? It is our public reference server, so we can see whether the server or client is causing the problems here.

[sdhull]

FWIW (in case someone else stumbles on this and finds it helpful)—we were running into something similar while trying to use this client for a custom jmeter script.

A little more context:
We're using a custom tusd server, though it's just some light customizations on the tusd go module which we leverage as a dependency. We are chunking uploads at 5MB. We're only using uppy.js in production (not the java client). We used the sample from the readme for jmeter and just changed a few params, added some metadata but basically used that reference sample. The script works fine for uploads <~7MB, however when we tried larger uploads we got failures, most notably a 413 status code back but didn't see any errors logged from tusd.

After much confusion, we finally realized the 413 was coming from the k8s nginx ingress controller, which was complaining:

a client request body is buffered to a temporary file
client intended to send too large chunked body

After even more confusion I found the setRequestPayloadSize method, which I had a hard time reasoning about what the difference would be between this and the setChunkSize but my colleague explained it has to do with streaming uploads.

Anyway tl;dr if you call setChunkSize and setRequestPayloadSize with the same value, it fixes the issue by switching from this streaming uploads to issuing multiple PATCH requests

[Acconut]

After even more confusion I found the setRequestPayloadSize method, which I had a hard time reasoning about what the difference would be between this and the setChunkSize but my colleague explained it has to do with streaming uploads.

Thank you for the feedback! We know that these two methods are very confusing, especially since the term chunk size has a different meaning for different tus clients. We have plan to rework the internals of tus-java-client in the near future and setChunkSize will likely be removed in that process because it is no longer necessary then.

[Acconut]

We plan to overall the API in the next major release and make such configuration with payload and chunk sizes easier: #78