Skip to content

Commit 94dac7b

Browse files
rerpharerpha
committed
first draft - add diagram
1 parent 634b43d commit 94dac7b

File tree

5 files changed

+39
-11
lines changed

5 files changed

+39
-11
lines changed

‎doc/specific_iocs/dae/Datastreaming.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,9 @@ In general this works by producing both neutron events and histograms, sample en
1515

1616
All data is serialised into [Flatbuffers](https://flatbuffers.dev/) blobs using [these schemas](https://github.com/ess-dmsc/streaming-data-types) - we have a tool called [saluki](https://github.com/ISISComputingGroup/saluki) which can deserialise these and make them human-readable after they've been put into Kafka.
1717

18-
Overall architecture is still being decided, but this is an initial idea of how it could look.
18+
Overall architecture is still being decided, but this is an initial idea of how it could look:
1919

20-
<TODO>
20+
![](ISISDSLayout.drawio.svg)
2121

2222

2323
{#kafkacluster}

‎doc/specific_iocs/dae/ISISDSLayout.drawio.svg‎

Lines changed: 4 additions & 0 deletions
Loading

‎doc/specific_iocs/dae/ISISDSLayout.xml‎

Lines changed: 26 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
<mxfile host="app.diagrams.net" agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36" version="29.1.1">
22
<diagram id="Hcih1Z2I40CavoK4oXOS" name="Page-1">
3-
<mxGraphModel dx="2037" dy="1076" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
3+
<mxGraphModel dx="2194" dy="1158" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
44
<root>
55
<mxCell id="0" />
66
<mxCell id="1" parent="0" />
@@ -35,7 +35,7 @@
3535
<mxPoint y="31" as="offset" />
3636
</mxGeometry>
3737
</mxCell>
38-
<mxCell id="4" edge="1" parent="1" source="6" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=1;exitDx=0;exitDy=0;startArrow=none;startFill=0;">
38+
<mxCell id="4" edge="1" parent="1" source="6" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=1;exitDx=0;exitDy=0;startArrow=none;startFill=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" target="28">
3939
<mxGeometry relative="1" as="geometry">
4040
<mxPoint x="1010" y="1488" as="targetPoint" />
4141
</mxGeometry>
@@ -72,10 +72,15 @@
7272
<mxPoint as="offset" />
7373
</mxGeometry>
7474
</mxCell>
75+
<mxCell id="MyJvBkN_DfxryQa6yyBt-33" edge="1" parent="1" source="28" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;startArrow=classic;startFill=1;endArrow=none;endFill=0;">
76+
<mxGeometry relative="1" as="geometry">
77+
<mxPoint x="1000.0000000000005" y="1440" as="targetPoint" />
78+
</mxGeometry>
79+
</mxCell>
7580
<mxCell id="28" parent="1" style="rounded=1;whiteSpace=wrap;html=1;" value="forwarder" vertex="1">
76-
<mxGeometry height="60" width="120" x="950" y="1490" as="geometry" />
81+
<mxGeometry height="60" width="120" x="940" y="1350" as="geometry" />
7782
</mxCell>
78-
<mxCell id="7KtQmuTptNfZlMXJsWDl-53" parent="1" style="rounded=0;whiteSpace=wrap;html=1;" value="Blockserver" vertex="1">
83+
<mxCell id="7KtQmuTptNfZlMXJsWDl-53" parent="1" style="rounded=0;whiteSpace=wrap;html=1;dashed=1;" value="Blockserver" vertex="1">
7984
<mxGeometry height="60" width="120" x="90" y="1360" as="geometry" />
8085
</mxCell>
8186
<mxCell id="7KtQmuTptNfZlMXJsWDl-55" edge="1" parent="1" source="7KtQmuTptNfZlMXJsWDl-54" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" target="7KtQmuTptNfZlMXJsWDl-53">
@@ -117,7 +122,7 @@
117122
<mxCell id="9" parent="7KtQmuTptNfZlMXJsWDl-72" style="rounded=0;whiteSpace=wrap;html=1;movable=1;resizable=1;rotatable=1;deletable=1;editable=1;locked=0;connectable=1;" value="FPGA streaming boards" vertex="1">
118123
<mxGeometry height="60" width="120" x="20" y="60" as="geometry" />
119124
</mxCell>
120-
<mxCell id="7KtQmuTptNfZlMXJsWDl-33" parent="7KtQmuTptNfZlMXJsWDl-72" style="rounded=0;whiteSpace=wrap;html=1;" value="(borzoi)~&lt;br&gt;process that&lt;br&gt;A) bridges FPGA UDP config and EPICS and&lt;br&gt;B) provides an interface very similar to ISISDAE&lt;div&gt;&lt;br&gt;&lt;/div&gt;" vertex="1">
125+
<mxCell id="7KtQmuTptNfZlMXJsWDl-33" parent="7KtQmuTptNfZlMXJsWDl-72" style="rounded=0;whiteSpace=wrap;html=1;dashed=1;" value="(borzoi)~&lt;br&gt;process that&lt;br&gt;A) bridges FPGA UDP config and EPICS and&lt;br&gt;B) provides an interface very similar to ISISDAE&lt;div&gt;&lt;br&gt;&lt;/div&gt;" vertex="1">
121126
<mxGeometry height="130" width="120" x="20" y="170" as="geometry" />
122127
</mxCell>
123128
<mxCell id="7KtQmuTptNfZlMXJsWDl-47" parent="7KtQmuTptNfZlMXJsWDl-72" style="swimlane;" value="For new instruments" vertex="1">
@@ -138,13 +143,13 @@
138143
<mxCell id="7KtQmuTptNfZlMXJsWDl-75" connectable="0" parent="1" style="group" value="" vertex="1">
139144
<mxGeometry height="380" width="170" x="450" y="670" as="geometry" />
140145
</mxCell>
141-
<mxCell id="7KtQmuTptNfZlMXJsWDl-36" parent="7KtQmuTptNfZlMXJsWDl-75" style="rounded=0;whiteSpace=wrap;html=1;" value="ISISDAE IOC" vertex="1">
146+
<mxCell id="7KtQmuTptNfZlMXJsWDl-36" parent="7KtQmuTptNfZlMXJsWDl-75" style="rounded=0;whiteSpace=wrap;html=1;dashed=1;" value="ISISDAE IOC" vertex="1">
142147
<mxGeometry height="100" width="120" x="20" y="250" as="geometry" />
143148
</mxCell>
144149
<mxCell id="7KtQmuTptNfZlMXJsWDl-38" edge="1" parent="7KtQmuTptNfZlMXJsWDl-75" source="7KtQmuTptNfZlMXJsWDl-37" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;startArrow=classic;startFill=1;" target="7KtQmuTptNfZlMXJsWDl-36">
145150
<mxGeometry relative="1" as="geometry" />
146151
</mxCell>
147-
<mxCell id="7KtQmuTptNfZlMXJsWDl-43" connectable="0" parent="7KtQmuTptNfZlMXJsWDl-38" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" value="asyn calls?" vertex="1">
152+
<mxCell id="7KtQmuTptNfZlMXJsWDl-43" connectable="0" parent="7KtQmuTptNfZlMXJsWDl-38" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" value="DCOM" vertex="1">
148153
<mxGeometry relative="1" x="0.075" as="geometry">
149154
<mxPoint as="offset" />
150155
</mxGeometry>
@@ -182,6 +187,20 @@
182187
<mxPoint as="offset" />
183188
</mxGeometry>
184189
</mxCell>
190+
<mxCell id="MyJvBkN_DfxryQa6yyBt-31" edge="1" parent="1" source="MyJvBkN_DfxryQa6yyBt-30" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=1;entryY=0;entryDx=0;entryDy=0;startArrow=classic;startFill=1;endArrow=none;endFill=0;" target="6">
191+
<mxGeometry relative="1" as="geometry" />
192+
</mxCell>
193+
<mxCell id="MyJvBkN_DfxryQa6yyBt-32" connectable="0" parent="MyJvBkN_DfxryQa6yyBt-31" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" value="ev44/hs00" vertex="1">
194+
<mxGeometry relative="1" x="-0.3585" y="4" as="geometry">
195+
<mxPoint as="offset" />
196+
</mxGeometry>
197+
</mxCell>
198+
<mxCell id="MyJvBkN_DfxryQa6yyBt-30" parent="1" style="rounded=0;whiteSpace=wrap;html=1;dashed=1;" value="(azawakh)&lt;div&gt;consumer soft ioc that provides areadetector live view from kafka stream/spectra plots over epics&lt;/div&gt;" vertex="1">
199+
<mxGeometry height="100" width="200" x="680" y="670" as="geometry" />
200+
</mxCell>
201+
<mxCell id="MyJvBkN_DfxryQa6yyBt-34" parent="1" style="rounded=0;whiteSpace=wrap;html=1;dashed=1;" value="IOCs&amp;nbsp; - blocks and archived vals, maybe runlog" vertex="1">
202+
<mxGeometry height="60" width="120" x="940" y="1440" as="geometry" />
203+
</mxCell>
185204
</root>
186205
</mxGraphModel>
187206
</diagram>

‎doc/specific_iocs/dae/datastreaming/Datastreaming--neutron-events-histograms.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,13 @@
11
{#dseventshistos}
22
# Data streaming: Neutron events and histograms
33

4-
54
## For DAE2/DAE3 instruments
65
The ICP (communicated to via the ISISDAE IOC) is responsible for communicating with the DAE2/DAE3 in terms of setting configuration, as well as streaming events and histograms from both.
76

87

9-
108
## For new instruments using FPGA-based acquisition electronics
9+
`borzoi` is responsible for communicating with the electronics and sending run starts/stops. It will have a similar interface to `ISISDAE` so we can drop-in replace it in the GUI.(?)
10+
1111

12+
## Live view, spectra plots etc.
13+
These will be provided by a soft IOC (`azawakh`) which effectively consumes from event and histogram topics (and possibly run starts?) which will serve areaDetector and other PVs.

‎doc/specific_iocs/dae/datastreaming/Datastreaming-run-starts-stops.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,5 @@
11
{#dsrunstartstops}
22
# Data streaming: run starts/stops
3+
4+
Run starts and stops will be dealt with by `borzoi` and the flatbuffers will be constructed in this process. It may need to be hooked onto by `ISISDAE` for older instruments using DAE2/DAE3 and the ISISICP.
5+

0 commit comments

Comments
 (0)