ArgumentOutOfRangeException from Update on a tracked entity after a complex collection whose elements have a nested complex collection gets shorter

[icecactus]

Bug description

Calling Update on a tracked entity throws ArgumentOutOfRangeException when:

1. the entity has a complex collection whose element type has a complex collection of its own (Order.Lines, where each Line has Parts; the inner lists can be empty),
2. the outer collection now has fewer items than when the entity started being tracked, and
3. DbSet.Update / UpdateRange / DbContext.Update is called on the entity.

The same change saved through DetectChanges works: the entity becomes Modified. It also works when the outer list grows, and when the list's items have no nested collection.

We hit it through a generic "update many" repository helper. It calls UpdateRange on whatever it is given, without checking whether the entities are already tracked:

public async Task UpdateManyAsync(IEnumerable<TEntity> entities, bool autoSave = false)
{
    var dbContext = await GetDbContextAsync();
    dbContext.Set<TEntity>().UpdateRange(entities.ToArray());
    if (autoSave)
    {
        await dbContext.SaveChangesAsync();
    }
}

The entities had been loaded tracked earlier in the same context, and a list on them was replaced with a shorter one before the call.

This looks close to #37585 and #37724, which were fixed in 10.0.6, but it still reproduces on 10.0.12.

Your code

No database is needed. The connection is never opened.

using Microsoft.EntityFrameworkCore;

// Throws: Lines items have a nested complex collection; Lines gets shorter; then Update.
Run("Lines (items have a nested collection) shorter, then Update", context =>
{
    var order = new Order { Id = 1, Lines = [new Line { Name = "a" }, new Line { Name = "b" }] };
    context.Attach(order);
    order.Lines = [new Line { Name = "a" }];
    context.Update(order);
});

// Works: the same change, detected without Update.
Run("Lines shorter, DetectChanges only", context =>
{
    var order = new Order { Id = 2, Lines = [new Line { Name = "a" }, new Line { Name = "b" }] };
    context.Attach(order);
    order.Lines = [new Line { Name = "a" }];
    context.ChangeTracker.DetectChanges();
});

// Works: Tags items have no nested collection; Tags gets shorter; then Update.
Run("Tags (flat items) shorter, then Update", context =>
{
    var order = new Order { Id = 3, Tags = [new Tag { Name = "a" }, new Tag { Name = "b" }] };
    context.Attach(order);
    order.Tags = [new Tag { Name = "a" }];
    context.Update(order);
});

// Works: Lines gets longer, then Update.
Run("Lines longer, then Update", context =>
{
    var order = new Order { Id = 4, Lines = [new Line { Name = "a" }] };
    context.Attach(order);
    order.Lines = [new Line { Name = "a" }, new Line { Name = "b" }];
    context.Update(order);
});

static void Run(string name, Action<AppContext> body)
{
    using var context = new AppContext();
    try { body(context); Console.WriteLine($"OK     {name}"); }
    catch (Exception e) { Console.WriteLine($"THROWS {name}: {e.GetType().Name}: {e.Message}"); }
}

public class AppContext : DbContext
{
    protected override void OnConfiguring(DbContextOptionsBuilder options)
        => options.UseSqlServer("Server=unused;Database=unused"); // never opened

    protected override void OnModelCreating(ModelBuilder modelBuilder)
        => modelBuilder.Entity<Order>(order =>
        {
            order.ComplexCollection(o => o.Tags, tags => tags.ToJson());
            order.ComplexCollection(o => o.Lines, lines =>
            {
                lines.ToJson();
                lines.ComplexCollection(l => l.Parts);
            });
        });
}

public class Order
{
    public int Id { get; set; }
    public List<Tag> Tags { get; set; } = [];
    public List<Line> Lines { get; set; } = [];
}
public class Tag { public string Name { get; set; } = ""; }
public class Line { public string Name { get; set; } = ""; public List<Part> Parts { get; set; } = []; }
public class Part { public string Code { get; set; } = ""; }

Output:

THROWS Lines (items have a nested collection) shorter, then Update: ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index')
OK     Lines shorter, DetectChanges only
OK     Tags (flat items) shorter, then Update
OK     Lines longer, then Update

Stack traces

System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index')
   at lambda_method44(Closure, Order, IReadOnlyList`1)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.ReadPropertyValue(IPropertyBase propertyBase)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalComplexEntry.ReadPropertyValue(IPropertyBase propertyBase)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.get_Item(IPropertyBase propertyBase)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.InternalComplexCollectionEntry.GetCollection(Boolean original)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.InternalComplexCollectionEntry.GetOrCreateEntries(Boolean original, EntityState defaultState)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.<>c.<GetFlattenedComplexEntries>b__83_0(InternalComplexCollectionEntry c)
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.MoveNext()
   at System.Linq.Enumerable.IEnumerableWhereIterator`1.MoveNext()
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.SetPropertyModified(IComplexProperty property, Boolean isModified, Boolean recurse)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.SetEntityState(EntityState oldState, EntityState newState, Boolean acceptChanges, Boolean modifyProperties)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalComplexEntry.SetEntityState(EntityState oldState, EntityState newState, Boolean acceptChanges, Boolean modifyProperties)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.SetEntityState(EntityState entityState, Boolean acceptChanges, Boolean modifyProperties, Nullable`1 forceStateWhenUnknownKey, Nullable`1 fallbackState)
   ...
   at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.UpdateRange(TEntity[] entities)

It looks like SetPropertyModified(..., recurse: true) reads the original values of each item's nested collection by the item's original ordinal, against the current (shorter) outer list.

Verbose output

No response

EF Core version

10.0.12 (also 10.0.9)

Database provider

Microsoft.EntityFrameworkCore.SqlServer (also Npgsql.EntityFrameworkCore.PostgreSQL 10.0.1 / 10.0.3)

Target framework

.NET 10.0 (SDK 10.0.401)

Operating system

Windows 11 (10.0.26200)

IDE

No response