Results and the output of Computations

[Legion2]

Let's discuss the results and output of Computation returned by the websocket-api and external services (e.g. Blob storage).

Each computation produces different results and output artifacts. They mostly depend on the type of environment of the Computation, but also the purpose of the Computation (programming task, simulation or visualisation). There are many types of artifacts that are created by a Computation, for example files, logs, stdout, stderr, file hierarchy, upload to external services, visual output(only when there is a display). And there are even more ways to visualize these types of artifact to the user.

To support all the different types of artifacts and visualisations from above and even those not listed above, we have to define an extensible, generalizable, but also customizable format which does not prevent use cases we currently can't think of while allowing to handle all artifacts in an uniform manner.

To enable this I propose we create an abstraction of "artifacts" which are the outputs of a Computation. I will call this thing Artifact for now. Each artifact have an id, metadata, type and additional type dependent attributes. Maybe the Artifact also have a visualization type attribute, but maybe this is part of the type dependent attributes or can be derive from the Artifact type or is part of the metadata.

Examples:
A calculation created a plot and saved it as an image.
This image is an Artifact of type file and has the type dependent attribute mimetype which indicates that the file is an image. The fronted can then display this artifact as an image. The file itself is base64 encoded and also a type dependent attribute content.

Second example, the computation is a simulation and the result is a huge file containing the detailed results. This could be realized as Artifact of type file but this would result in a huge json object. Instead the Artifact is of type s3-file with the type dependent attribute url which contains an signed url to the temporary simulation results file.

Because of this abstraction, all artifacts/results can be handled in an uniform way in the ViPLab backend and api. If needed the different types of Artifacts are interpreted by the fronted.

[pascalseeland]

I'm totally fine with creating an abstraction called Artifact to capture the different file outputs. In the current state of the documentation we have 3 different parts. Notifications, Output and Files.
I would like keep the Output as an extra element, which only contains stdout and stderr. I would even define them as mandatory even if both elements are empty.
Notifications are parsed elements from stdout and stderr to be shown in the UI. These could nicely fit into an Artifact of type Notifcation, which has the json-object as its content.
Files itself do fit to your proposal.

[Legion2]

Ok, stdout and stderr are defined as mandatory. Everything else is optional and defined as artifact.

The name "artifact" was only used by me to describe the concept, we can also use another name if you have a better idea.

[Legion2]

"artifacts" should also have an id so we can identify them if there are multiple of the same type.

[pascalseeland]

I did update the example in VirtualProgrammingLab/documentation#9 accordingly.