N-Quads serializer ignores default graph

[edmondchuc]

The following script can be run as-is:

from rdflib import Dataset

data = """
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

{
	_:b0 <http://www.w3.org/ns/prov#generatedAtTime> "2012-04-09"^^xsd:date .
}

_:b0 {
	<http://greggkellogg.net/foaf#me> a <http://xmlns.com/foaf/0.1/Person> ;
		<http://xmlns.com/foaf/0.1/knows> "http://manu.sporny.org/about#manu" ;
		<http://xmlns.com/foaf/0.1/name> "Gregg Kellogg" .

	<http://manu.sporny.org/about#manu> a <http://xmlns.com/foaf/0.1/Person> ;
		<http://xmlns.com/foaf/0.1/knows> "http://greggkellogg.net/foaf#me" ;
		<http://xmlns.com/foaf/0.1/name> "Manu Sporny" .
}


"""

g = Dataset()
g.parse(data=data, format="trig")

g.print(format="nquads")

Output:

_:nde95dc418226482f9fb7b0242109b9a3b1 <http://www.w3.org/ns/prov#generatedAtTime> "2012-04-09"^^<http://www.w3.org/2001/XMLSchema#date> _:Neae5d6b422ed4d1d872dd9674af22f8f .
<http://greggkellogg.net/foaf#me> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> _:nde95dc418226482f9fb7b0242109b9a3b1 .
<http://manu.sporny.org/about#manu> <http://xmlns.com/foaf/0.1/name> "Manu Sporny" _:nde95dc418226482f9fb7b0242109b9a3b1 .
<http://manu.sporny.org/about#manu> <http://xmlns.com/foaf/0.1/knows> "http://greggkellogg.net/foaf#me" _:nde95dc418226482f9fb7b0242109b9a3b1 .
<http://greggkellogg.net/foaf#me> <http://xmlns.com/foaf/0.1/knows> "http://manu.sporny.org/about#manu" _:nde95dc418226482f9fb7b0242109b9a3b1 .
<http://manu.sporny.org/about#manu> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> _:nde95dc418226482f9fb7b0242109b9a3b1 .
<http://greggkellogg.net/foaf#me> <http://xmlns.com/foaf/0.1/name> "Gregg Kellogg" _:nde95dc418226482f9fb7b0242109b9a3b1 .

---

Issue

I would have expected the first statement of the output to omit the graph label as it is a statement in the default graph.

_:nde95dc418226482f9fb7b0242109b9a3b1 <http://www.w3.org/ns/prov#generatedAtTime> "2012-04-09"^^<http://www.w3.org/2001/XMLSchema#date> _:Neae5d6b422ed4d1d872dd9674af22f8f .

See https://www.w3.org/TR/n-quads/#simple-triples for reference.

[edmondchuc]

Hmm, it's somewhat related to #1804.

[aucampia]

May be related to this also:

• https://github.com/RDFLib/rdflib/blob/6f2c11cd2c549d6410f9a1c948ab3a8dbf77ca00/test/variants/rdf11trig_eg2.trig
• https://github.com/RDFLib/rdflib/blob/6f2c11cd2c549d6410f9a1c948ab3a8dbf77ca00/test/variants/rdf11trig_eg2.nq

• rdflib/test/test_graph/test_variants.py

  Lines 144 to 166 in 6f2c11c

  	("variants/rdf11trig_eg2"): pytest.mark.xfail(
  	reason="""
  	This fails randomly, passing less than 10% of the time, and always failing
  	with comparing hext against trig. Not clear why, it may be a big with hext
  	parsing.
  	AssertionError: checking rdf11trig_eg2.hext against rdf11trig_eg2.trig
  	in both:
  	(rdflib.term.BNode('cb0'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/mbox'), rdflib.term.URIRef('mailto:bob@oldcorp.example.org'))
  	(rdflib.term.BNode('cb0'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/name'), rdflib.term.Literal('Bob'))
  	(rdflib.term.URIRef('http://example.org/bob'), rdflib.term.URIRef('http://purl.org/dc/terms/publisher'), rdflib.term.Literal('Bob'))
  	(rdflib.term.URIRef('http://example.org/alice'), rdflib.term.URIRef('http://purl.org/dc/terms/publisher'), rdflib.term.Literal('Alice'))
  	only in first:
  	(rdflib.term.BNode('cb0'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/knows'), rdflib.term.BNode('cbb5eb12b5dcf688537b0298cce144c6dd68cf047530d0b4a455a8f31f314244fd'))
  	(rdflib.term.BNode('cbb5eb12b5dcf688537b0298cce144c6dd68cf047530d0b4a455a8f31f314244fd'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/mbox'), rdflib.term.URIRef('mailto:alice@work.example.org'))
  	(rdflib.term.BNode('cbb5eb12b5dcf688537b0298cce144c6dd68cf047530d0b4a455a8f31f314244fd'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/name'), rdflib.term.Literal('Alice'))
  	only in second:
  	(rdflib.term.BNode('cb0'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/knows'), rdflib.term.BNode('cbcd41774964510991c01701d8430149bc373e1f23734d9c938c81a40b1429aa33'))
  	(rdflib.term.BNode('cbcd41774964510991c01701d8430149bc373e1f23734d9c938c81a40b1429aa33'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/mbox'), rdflib.term.URIRef('mailto:alice@work.example.org'))
  	(rdflib.term.BNode('cbcd41774964510991c01701d8430149bc373e1f23734d9c938c81a40b1429aa33'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/name'), rdflib.term.Literal('Alice'))
  	""",
  	raises=AssertionError,
  	),

• rdflib/test/test_roundtrip.py

  Lines 145 to 166 in 6f2c11c

  	("trig", "rdf11trig_eg2.trig"): pytest.mark.xfail(
  	reason="""
  	Something is going wrong here with blank node serialization. In the second
  	graph below bob knows someone who does not exist, while in first he knows
  	someone that does exist and has the name Alice.
  	AssertionError: in both:
  	(rdflib.term.BNode('cbb5eb12b5dcf688537b0298cce144c6dd68cf047530d0b4a455a8f31f314244fd'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/mbox'), rdflib.term.URIRef('mailto:alice@work.example.org'))
  	(rdflib.term.BNode('cbb5eb12b5dcf688537b0298cce144c6dd68cf047530d0b4a455a8f31f314244fd'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/name'), rdflib.term.Literal('Alice'))
  	(rdflib.term.URIRef('http://example.org/alice'), rdflib.term.URIRef('http://purl.org/dc/terms/publisher'), rdflib.term.Literal('Alice'))
  	(rdflib.term.URIRef('http://example.org/bob'), rdflib.term.URIRef('http://purl.org/dc/terms/publisher'), rdflib.term.Literal('Bob'))
  	only in first:
  	(rdflib.term.BNode('cb0'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/knows'), rdflib.term.BNode('cbb5eb12b5dcf688537b0298cce144c6dd68cf047530d0b4a455a8f31f314244fd'))
  	(rdflib.term.BNode('cb0'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/mbox'), rdflib.term.URIRef('mailto:bob@oldcorp.example.org'))
  	(rdflib.term.BNode('cb0'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/name'), rdflib.term.Literal('Bob'))
  	only in second:
  	(rdflib.term.BNode('cb7be1d0397a49ddd4ae8aa96acc7b6135903c5f3fa5e47bf619c0e4b438aafcc1'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/knows'), rdflib.term.BNode('cb0'))
  	(rdflib.term.BNode('cb7be1d0397a49ddd4ae8aa96acc7b6135903c5f3fa5e47bf619c0e4b438aafcc1'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/mbox'), rdflib.term.URIRef('mailto:bob@oldcorp.example.org'))
  	(rdflib.term.BNode('cb7be1d0397a49ddd4ae8aa96acc7b6135903c5f3fa5e47bf619c0e4b438aafcc1'), rdflib.term.URIRef('http://xmlns.com/foaf/0.1/name'), rdflib.term.Literal('Bob'))
  	""",
  	raises=AssertionError,
  	),

EDIT: Actually on second thought no, maybe not.

[edmondchuc]

I guess this is a more general issue with how rdflib serializes context-aware stores. Changing the output format to trig results in the same issue, thus breaking round-tripping.

@prefix foaf: <http://xmlns.com/foaf/0.1/> .
@prefix ns1: <urn:x-rdflib:> .
@prefix prov: <http://www.w3.org/ns/prov#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

_:nd65e1d4bf7a34d92a06e4e619a245037b1 {
    <http://greggkellogg.net/foaf#me> a foaf:Person ;
        foaf:knows "http://manu.sporny.org/about#manu" ;
        foaf:name "Gregg Kellogg" .

    <http://manu.sporny.org/about#manu> a foaf:Person ;
        foaf:knows "http://greggkellogg.net/foaf#me" ;
        foaf:name "Manu Sporny" .
}

_:N427df78321f84718beec24f5f0c7e26c {
    [] prov:generatedAtTime "2012-04-09"^^xsd:date .
}

[edmondchuc]

I noticed this issue while I was working on implementing a more efficient integration of pyld as a parser into rdflib core #1836.

My implementation sets the graph name to rdflib.graph.DATASET_DEFAULT_GRAPH_ID when the statement is from the default graph and correctly serializes the dataset.

I noticed the output in the format nquads and trig were different. Those parsers and serializers fail round-trips by ignoring the default graph and incorrectly setting it to a blank node.

An easy fix (I think) is to set the context to rdflib.graph.DATASET_DEFAULT_GRAPH_ID in the failing serializers.

[edmondchuc]

To add further to this, it may be that those other serializers are adding statements from the default graph as None which results in adding those statements to a graph labelled with a blank node. I need to confirm this.

For example:

from rdflib.graph import DATASET_DEFAULT_GRAPH_ID

# Instead of this
store.add((s, p, o), None)

# Do this
store.add((s, p, o), DATASET_DEFAULT_GRAPH_ID)

[edmondchuc]

An easy fix (I think) is to set the context to rdflib.graph.DATASET_DEFAULT_GRAPH_ID in the failing serializers.

Oops, I take that back. This only works correctly for trig.

The nquads serializer just need to omit the graph label when it sees rdflib.graph.DATASET_DEFAULT_GRAPH_ID.

Currently it serializes something like:

_:b0 <http://www.w3.org/ns/prov#generatedAtTime> "2012-04-09"^^<http://www.w3.org/2001/XMLSchema#date> <urn:x-rdflib:default> .
<http://manu.sporny.org/about#manu> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> _:b0 .
<http://greggkellogg.net/foaf#me> <http://xmlns.com/foaf/0.1/knows> "http://manu.sporny.org/about#manu"^^<http://www.w3.org/2001/XMLSchema#string> _:b0 .
<http://manu.sporny.org/about#manu> <http://xmlns.com/foaf/0.1/knows> "http://greggkellogg.net/foaf#me"^^<http://www.w3.org/2001/XMLSchema#string> _:b0 .
<http://greggkellogg.net/foaf#me> <http://xmlns.com/foaf/0.1/name> "Gregg Kellogg"^^<http://www.w3.org/2001/XMLSchema#string> _:b0 .
<http://greggkellogg.net/foaf#me> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> _:b0 .
<http://manu.sporny.org/about#manu> <http://xmlns.com/foaf/0.1/name> "Manu Sporny"^^<http://www.w3.org/2001/XMLSchema#string> _:b0 .

Notice the <urn:x-rdflib:default>.

The nquads serializer just need to omit the graph label when it sees rdflib.graph.DATASET_DEFAULT_GRAPH_ID.

You're not wrong, I've been addressing this in the dataset re-work, changes to nquads serializer here

[aucampia]

The nquads serializer just need to omit the graph label when it sees rdflib.graph.DATASET_DEFAULT_GRAPH_ID.

You're not wrong, I've been addressing this in the dataset re-work, changes to nquads serializer here

Can't we get this in without breaking changes?

Can't we get this in without breaking changes?

Yes. In this instance, correcting the serialization doesn't cause any breaking changes.

[edmondchuc]

Thanks for your work @gjhiggins. I've copied your code from the nquads serializer out into a separate PR. I hope you don't mind. I need this patch to get the JSON-LD 1.1 tests to pass.

Can I ask why + [DATASET_DEFAULT_GRAPH_ID] is required?

rdflib/rdflib/plugins/serializers/nquads.py

Lines 38 to 41 in 4fba0ff

	for context in list(self.store.contexts()) + [DATASET_DEFAULT_GRAPH_ID]:
	graph = self.store.graph(context)
	for triple in graph:
	stream.write(self._nq_row(triple, context).encode(encoding, "replace"))

I had to remove it because the serialize method was outputting double the statements.

Thanks for your work @gjhiggins. I've copied your code from the nquads serializer out into a separate PR. I hope you don't mind. I need this patch to get the JSON-LD 1.1 tests to pass.

That's cool, I don't mind at all, whatever works for you.

Can I ask why + [DATASET_DEFAULT_GRAPH_ID] is required?

It's a consequence of switching over to Dataset. ConjunctiveGraph.contexts() returns all graphs, including the default graph and Dataset.graphs() doesn't include the (nameless) default graph.

[sdasda7777]

Hi, I just noticed when I take the multigraph example from JSON-LD standard and convert it to N-Quads, main graph is suddenly referenced by a blank label, instead of no label. My code is:

from rdflib.graph import Dataset

data = """{
"@context": [
"http://schema.org/",
{"@base": "http://example.com/"}
],
"@graph": [{
"@id": "people/alice",
"gender": [
{"@value": "weiblich", "@language": "de"},
{"@value": "female",   "@language": "en"}
],
"knows": {"@id": "people/bob"},
"name": "Alice"
}, {
"@id": "graphs/1",
"@graph": {
"@id": "people/alice",
"parent": {
"@id": "people/bob",
"name": "Bob"
}
}
}, {
"@id": "graphs/2",
"@graph": {
"@id": "people/bob",
"sibling": {
"name": "Mary",
"sibling": {"@id": "people/bob"}
}
}
}]
}"""

ds = Dataset()
ds.parse(data=data, format="json-ld")
print(ds.serialize(format="nquads").strip())

The result looks like this for me:

<http://example.com/people/bob> <http://schema.org/name> "Bob" <http://example.com/graphs/1> .
<http://example.com/people/alice> <http://schema.org/parent> <http://example.com/people/bob> <http://example.com/graphs/1> .
<http://example.com/people/alice> <http://schema.org/gender> "female"@en _:N6535627397b54eb2b076091aaccf8a98 .
<http://example.com/people/alice> <http://schema.org/name> "Alice" _:N6535627397b54eb2b076091aaccf8a98 .
<http://example.com/people/alice> <http://schema.org/gender> "weiblich"@de _:N6535627397b54eb2b076091aaccf8a98 .
<http://example.com/people/alice> <http://schema.org/knows> <http://example.com/people/bob> _:N6535627397b54eb2b076091aaccf8a98 .
<http://example.com/people/bob> <http://schema.org/sibling> _:Na4b162b6579f4d0a9aa68d2d0f65572c <http://example.com/graphs/2> .
_:Na4b162b6579f4d0a9aa68d2d0f65572c <http://schema.org/name> "Mary" <http://example.com/graphs/2> .
_:Na4b162b6579f4d0a9aa68d2d0f65572c <http://schema.org/sibling> <http://example.com/people/bob> <http://example.com/graphs/2> .

However on the JSON-LD playground, the output for N-Quads conversion looks like this instead:

<http://example.com/people/alice> <http://schema.org/gender> "female"@en .
<http://example.com/people/alice> <http://schema.org/gender> "weiblich"@de .
<http://example.com/people/alice> <http://schema.org/knows> <http://example.com/people/bob> .
<http://example.com/people/alice> <http://schema.org/name> "Alice" .
<http://example.com/people/alice> <http://schema.org/parent> <http://example.com/people/bob> <http://example.com/graphs/1> .
<http://example.com/people/bob> <http://schema.org/name> "Bob" <http://example.com/graphs/1> .
<http://example.com/people/bob> <http://schema.org/sibling> _:b0 <http://example.com/graphs/2> .
_:b0 <http://schema.org/name> "Mary" <http://example.com/graphs/2> .
_:b0 <http://schema.org/sibling> <http://example.com/people/bob> <http://example.com/graphs/2> .

Is this issue likely to be solved soon?