SHACL validation report with violations resulting in sh:conforms true

[costas80]

See the original discussion of this issue here.

It seems that runtime errors may result in violations being added to the SHACL validation report, while still resulting in an overall sh:conforms true. To illustrate with an example, considering this shape ...

@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix foobar: <https://foo.bar/> .

foobar:FooShape a sh:NodeShape;
	sh:targetNode foobar:FooShape;
	sh:sparql [
		sh:select """
			SELECT $this WHERE {
				$this https://foo.bar/predicate ?value.
			}""";
	];
	sh:severity sh:Violation.

... where the IRIREF is invalid, this throws an exception while carrying out the validation. The exception results in a violation being recorded, however the overall sh:conforms is still set to true:

PREFIX dash:    <http://datashapes.org/dash#>
PREFIX foobar:  <https://foo.bar/>
PREFIX graphql: <http://datashapes.org/graphql#>
PREFIX ns0:     <http://www.w3.org/ns/locn#>
PREFIX ns1:     <http://itb.ec.europa.eu/sample/po#>
PREFIX owl:     <http://www.w3.org/2002/07/owl#>
PREFIX rdf:     <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs:    <http://www.w3.org/2000/01/rdf-schema#>
PREFIX sh:      <http://www.w3.org/ns/shacl#>
PREFIX swa:     <http://topbraid.org/swa#>
PREFIX tosh:    <http://topbraid.org/tosh#>
PREFIX xsd:     <http://www.w3.org/2001/XMLSchema#>

[ rdf:type     sh:ValidationReport;
  sh:conforms  true;
  sh:result    [ rdf:type                      dash:FailureResult;
                 sh:focusNode                  foobar:FooShape;
                 sh:resultMessage              "Failed to create validator: Invalid SPARQL constraint (Line 3, column 11: Unresolved prefixed name: https:):\n\r\n\t\t\tSELECT $this WHERE {\r\n\t\t\t\t$this https://foo.bar/predicate ?value.\r\n\t\t\t}";
                 sh:resultSeverity             sh:Violation;
                 sh:sourceConstraintComponent  sh:SPARQLConstraintComponent;
                 sh:sourceShape                foobar:FooShape
               ]
] .

Whether or not a runtime error due to a bad shape should be reported as a violation is a secondary point (in my view it should be reported). The key issue is that this results in a seemingly inconsistent report. I would expect that any report including violations should have sh:conforms false.

What is your opinion on this?

[osi]

we just found this issue as well. i think saying the graph conforms is misleading.

[costas80]

@osi, in our use case (our SHACL validator web app which wraps this library), we implemented report post-processing logic to enforce sh:conforms false if the report contains any violations. Hopefully this will get implemented directly in the library though.

[osi]

that’s what i was thinking of as well, post-processing the ***@***.*** - http://fotap.org/~osiOn May 19, 2025, at 11:32 PM, Costas Simatos ***@***.***> wrote: costas80 left a comment (TopQuadrant/shacl#188) @osi, in our use case (our SHACL validator web app which wraps this library), we implemented report post-processing logic to enforce sh:conforms false if the report contains any violations. Hopefully this will get implemented directly in the library though. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[afs]

The report entry is of type dash:FailureResult
The code (ValidationEngine.updateConforms) looks for entries of type sh:ValidationResult.

(the java class ResourceValidationReport isn't used - that checks for sh:result properties with no other conditions)

[osi]

correct Andy. we then use Jena (still on v4) to load this report, because it sees the conforms flag set to true, it does not check any of the entries. semantically, the SHACL engine seems wrong. whether the graph conforms is ambiguous because it was unable to check all of the ***@***.*** - http://fotap.org/~osiOn May 20, 2025, at 6:34 AM, Andy Seaborne ***@***.***> wrote: afs left a comment (TopQuadrant/shacl#188) The report entry is of type dash:FailureResult The code (ValidationEngine.updateConforms) looks for entries of type sh:ValidationResult. (the java class ResourceValidationReport isn't used - that checks for sh:result properties with no other conditions) —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>