Skip to content

Write Skip Count Not Updated in Chunk Processing #4514

Open
@thilotanner

Description

@thilotanner

Bug description
It seems, the write skip count is not incremented / updated if items are removed from chunks during write (in case an item cannot be successfully written)

Environment
Spring Batch 5.0.3 / Spring Boot 3.1.5 / Java 17

Steps to reproduce

public class MemberWriter implements ItemStreamWriter<String> {

    @Override
    public void write(Chunk<? extends String> memberChunk) {
        Chunk<? extends String>.ChunkIterator iterator = memberChunk.iterator();
        ...
        iterator.remove(new RuntimeException());
    }

Expected behavior
According to documentation of class Chunk, the writeSkipCount in the corresponding StepExecution object should be incremented for every item removed from ChunkIterator:
Encapsulation of a list of items to be processed and possibly a list of failed items to be skipped. To mark an item as skipped clients should iterate over the chunk using the iterator() method, and if there is a failure call Chunk.ChunkIterator.remove() on the iterator. The skipped items are then available through the chunk.

Workaround
It's possible to circumvent the issue by creating a combined SkipListener / StepExecutionListener:

public class SkipTestListener implements SkipListener<String, String>, StepExecutionListener {
    private StepExecution stepExecution;

    @Override
    public void beforeStep(StepExecution stepExecution) {
        this.stepExecution = stepExecution;
    }

    @Override
    public void onSkipInWrite(String item, Throwable t) {
        stepExecution.setWriteSkipCount(stepExecution.getWriteSkipCount() + 1);
        log.warn("Skipped item: {}", item);
    }
}

and register it accordingly in the step:

        SkipTestListener testListener = new SkipTestListener();

        new StepBuilder("process-step", jobRepository)
                .<String, String>chunk(chunkSize, transactionManager)
                .reader(reader)
                .processor(processor)
                .writer(writer)
                .faultTolerant()
                .listener((StepExecutionListener) testListener)
                .listener((SkipListener<? super String, ? super String>) testListener)
                .build();

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions