Combination of many=True and a dotted source doesn't allow a default

[stephenfin]

Checklist

• I have verified that that issue exists against the master branch of Django REST framework.
• I have searched for similar issues in both open and closed tickets and cannot find a duplicate.
• This is not a usage question. (Those should be directed to the discussion group instead.)
• This cannot be dealt with as a third party library. (We prefer new functionality to be in the form of third party libraries where possible.)
• I have reduced the issue to the simplest possible case.
• I have included a failing test as a pull request. (If you are unable to do so we can still accept the issue.)

Steps to reproduce

Consider a model with a many-many relationship to itself via a through model.

class FooModel(models.Model):
    text = models.CharField(max_length=100)
    bar = models.ForeignKey(
        'BarModel', null=True, blank=True, on_delete=models.SET_NULL,
        related_name='foos', related_query_name='foo')


class BarModel(models.Model):
    pass

We can attempt to serialize this using something like the following.

        class _FooSerializer(serializers.ModelSerializer):
            class Meta:
                model = FooModel
                fields = ('id', 'text')

        class FooSerializer(serializers.ModelSerializer):
            other_foos = _FooSerializer(source='bar.foos', many=True)

            class Meta:
                model = FooModel
                fields = ('id', 'other_foos')

(NOTE: I haven't tested this yet. My case is a lot more complicated so I'll try reduce that is this reproducer isn't valid)

Expected behavior

This is intended to flatten the default output from:

{
    "id": 1,
    "bar": {
        "foos": [
            {
                "id": 2,
                "text": "abc"
            }
        ]
    }
}

to

{
    "id": 1,
    "bar": [
        {
            "id": 2,
            "text": "abc"
        }
    ]
}

Actual behavior

We get an exception:

Got AttributeError when attempting to get a value for field `related` on serializer `PatchDetailSerializer`.
The serializer field might be named incorrectly and not match any attribute or key on the `Patch` instance.
Original exception text was: 'NoneType' object has no attribute 'patches'.

As discussed in #5489, you need to have a default value if you wish to use dotted notation with a nullable value.

class FooSerializer(serializers.ModelSerializer):
    bar = _FooSerializer(source='bar.foos', default=None)

    class Meta:
        fields = ('id', 'bar')

This fixes things for empty case. However, we also need to specify many=True to correct the not-empty case otherwise we get the following error:

'RelatedManager' object has no attribute 'pk'

If we do that, we now get:

'NoneType' object has no attribute 'patches'

for the empty case 😞

[stephenfin]

This has been moved to a question, but it's actually a bug IMO. I've a pull request opened to fix the issue too. Have I missed something?

[auvipy]

maybe this will move to a possible bug issue

[carltongibson]

That PR is #7574.

@stephenfin Issue is that it's super complex, with a convoluted and difficult history around here.

Both the report and the test cases on the PR are asking an awful lot of a reviewer to be able to be sure that the issue is correct. So that's time, which no-one's yet found. I'd try fleshing it out, or waiting until someone can allocated the needed bandwidth.

All issues that aren't clear are being moved to discussions as part of #7740.

[stephenfin]

Ah, I thought I'd provided enough context and a reproducer to work on this. What information would you like from me to assist matters?

[lovelydinosaur]

Now resolved in #7574. Thanks for your time on this @stephenfin.

[stephenfin]

This was addressed in #7574