Skip to content

Add features to make iPython console work in cluster #210

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from
Draft

Add features to make iPython console work in cluster #210

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
e0c4801
add control of environment variables to the helm chart
gilesknap Sep 12, 2025
6dded26
add in runtime folder and HOME
gilesknap Sep 12, 2025
cf733c9
add in the runtime and autosave mounts
gilesknap Sep 12, 2025
5a6fd94
fix env and workingDir for main container
gilesknap Sep 12, 2025
f4b0a48
Apply spelling suggestions from coPilot code review
gilesknap Sep 12, 2025
1752a8c
genericize chart, removing references to EPICS, IOC and epics-containers
gilesknap Sep 19, 2025
6464a7c
fix opi folder mounts
gilesknap Sep 19, 2025
a1991c4
remove ioc specifics from pvc.yaml
gilesknap Sep 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Charts/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ These features will be accessed via `ec`. See https://github.com/epics-container

This script will:

- Inspect the values of `editable` and `autostart` in the `values.yaml` file of the specified IOC (TODO: at present it uses the p47-services source code to do so but this should be determined from the cluster in future).
- Inspect the values of `editable` and `autostart` in the `values.yaml` file of the specified service (TODO: at present it uses the p47-services source code to do so but this should be determined from the cluster in future).
- Port forward the debugpy port (5678) from the pod to localhost.
- If editable is true, it will mount the PVC locally using pv-mounter and open VSCode to the /workspaces/xxx folder.
- If autostart is false, it will exec into the container and launch debugpy to run the main program.
Expand Down
9 changes: 9 additions & 0 deletions Charts/fastcs/extra.values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,12 @@ extraContainers:
- command
- arg1
- arg2

global:
env:
- name: example
value: example

env:
- name: example
value: example
6 changes: 4 additions & 2 deletions Charts/fastcs/templates/pvcs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# PVC for debugging volume
{{ if .Values.editable }}
{{- $location := default .Values.global.location .Values.location | required "ERROR - You must supply location or global.location" -}}
{{- $ioc_group := default .Values.global.ioc_group .Values.ioc_group | required "ERROR - You must supply ioc_group or global.ioc_group" -}}
{{- $location := default .Values.global.location .Values.location | default "" -}}
{{- $ioc_group := default .Values.global.ioc_group .Values.ioc_group | default "" -}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could set is_ioc to true here if ioc_group is not empty, so that it doesn't have to be a value? And so that we can't have ioc_group=something with is_ioc=false, which would probably do bad things.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds sensible to me.


kind: PersistentVolumeClaim
apiVersion: v1
Expand All @@ -10,7 +10,9 @@ metadata:
labels:
app: {{ .Release.Name }}
location: {{ $location }}
with: {{ $ioc_group }}
ioc_group: {{ $ioc_group }}
{{- end }}
spec:
accessModes:
- ReadWriteMany
Expand Down
4 changes: 3 additions & 1 deletion Charts/fastcs/templates/service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,10 @@ metadata:
{{- include "fastcs.labels" . | nindent 4 }}
app: {{ .Release.Name }}
location: {{ .Values.global.location }}
{{- with .Values.global.ioc_group }}
ioc_group: {{ .Values.global.ioc_group }}
is_ioc: "true"
{{- end }}
is_ioc: {{ .Values.is_ioc | quote }}
spec:
selector:
app: {{ .Release.Name }}
Expand Down
73 changes: 57 additions & 16 deletions Charts/fastcs/templates/statefulset.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,24 +4,27 @@ Default the derivable substitution values.
This keeps the length of the values.txt file for each individual IOC
to a minimum
*/ -}}
{{- $location := default .Values.global.location .Values.location | required "ERROR - You must supply location or global.location" -}}
{{- $ioc_group := default .Values.global.ioc_group .Values.ioc_group | required "ERROR - You must supply ioc_group or global.ioc_group" -}}
{{- $location := default .Values.global.location .Values.location | default "" -}}
# for epics-containers services repos, ioc_group is set in services/values.yaml
{{- $ioc_group := default .Values.global.ioc_group .Values.ioc_group | default "" -}}
# for epics-containers services repos, opisClaim is named after the ioc_group in services/values.yaml
# for non-epics-containers services repos, opisClaim should be set in the service values.yaml
{{- $opisClaim := default (print $ioc_group "-opi-claim") .Values.opisClaim -}}
{{- $runtimeClaim := default (print $ioc_group "-runtime-claim") .Values.runtimeClaim -}}
{{- $autosaveClaim := default (print $ioc_group "-autosave-claim") .Values.autosaveClaim -}}
{{- $image := .Values.image | required "ERROR - You must supply image." -}}

{{- $enabled := eq .Values.global.enabled false | ternary false true -}}
{{- $enabled := eq .Values.global.enabled false | ternary false true }}

apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ .Release.Name }}
labels:
location: {{ $location }}
location: {{ $location | quote }}
{{- with $ioc_group }}
ioc_group: {{ $ioc_group }}
{{- end }}
enabled: {{ $enabled | quote }}
is_ioc: "true"
is_ioc: {{ .Values.is_ioc | quote }}
{{- include "fastcs.labels" . | nindent 4 }}
spec:
replicas: {{ $enabled | ternary 1 0 }}
Expand All @@ -35,10 +38,12 @@ spec:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
location: {{ $location }}
location: {{ $location | quote }}
{{- with $ioc_group }}
ioc_group: {{ $ioc_group }}
{{- end }}
enabled: {{ $enabled | quote }}
is_ioc: "true"
is_ioc: {{ .Values.is_ioc | quote }}
{{- include "fastcs.labels" . | nindent 8 }}
{{- with .Values.podLabels }}
{{- toYaml . | nindent 8 }}
Expand Down Expand Up @@ -78,6 +83,19 @@ spec:
- name: {{ .name }}
image: {{ .image }}
imagePullPolicy: {{ $.Values.image.pullPolicy }}
# a writable place to have cwd
workingDir: /tmp
env:
- name: HOME
value: /tmp
- name: TERM
value: xterm-256color
{{- with $.Values.env }}
{{- toYaml . | nindent 12}}
{{- end }}
{{- with $.Values.global.env }}
{{- toYaml . | nindent 12}}
{{- end }}
{{- with $.Values.securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
Expand All @@ -90,11 +108,13 @@ spec:
{{- with $.Values.volumeMounts }}
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with $.Values.opisMountPoint }}
- name: opis-volume
mountPath: /epics/opi
subPath: "{{ $.Release.Name }}"
mountPath: {{ . }}
subPath: {{ $.Release.Name }}
{{- end }}
- name: config-volume
mountPath: {{ $.Values.iocConfig }}
mountPath: {{ $.Values.configFolder }}
{{- if $.Values.editable }}
- name: {{ $.Release.Name }}-develop
mountPath: /dest
Expand All @@ -103,6 +123,19 @@ spec:
- name: {{ .Chart.Name }}
image: '{{ .Values.image.repository }}{{ ternary "-debug" "" .Values.editable }}:{{ .Values.image.tag }}'
{{- with .Values.securityContext }}
# a writable place to have cwd
workingDir: /tmp
env:
- name: HOME
value: /tmp
- name: TERM
value: xterm-256color
{{- with $.Values.env }}
{{- toYaml . | nindent 12}}
{{- end }}
{{- with $.Values.global.env }}
{{- toYaml . | nindent 12}}
{{- end }}
Comment on lines +133 to +138
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the precedence here if a variable appears in both? env should override global.env

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point, I think in fact the schema will fail for having the same key twice in a dictionary.

This means some exciting dictionary mergining in GoLang templates! blurg!

securityContext:
{{- toYaml . | nindent 12 }}
{{- end }}
Expand Down Expand Up @@ -138,11 +171,13 @@ spec:
{{- with .Values.volumeMounts }}
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with $.Values.opisMountPoint }}
- name: opis-volume
mountPath: /epics/opi
subPath: "{{ .Release.Name }}"
mountPath: {{ . }}
subPath: {{ $.Release.Name }}
{{- end }}
- name: config-volume
mountPath: {{ .Values.iocConfig }}
mountPath: {{ .Values.configFolder }}
{{- if .Values.editable }}
- name: {{ .Release.Name }}-develop
mountPath: /workspaces
Expand All @@ -155,9 +190,15 @@ spec:
{{- with .Values.volumes }}
{{- toYaml . | nindent 8 }}
{{- end }}
# if a mount point has been specified, claim a PVC to mount
# the PVC name will be ixx-epics-opis for IOCs (created in $opisClaim above)
# the PVC name must be specified in values.yaml for non-IOC services
{{- if .Values.opisMountPoint | default false }}
- name: opis-volume
persistentVolumeClaim:
claimName: {{ $opisClaim }}
# use the supplied claim name or default to one based on ioc_group
claimName: {{ $.Values.opisClaimName | default $opisClaim }}
{{- end }}
- name: config-volume
configMap:
name: {{ .Release.Name }}-config
Expand Down
50 changes: 47 additions & 3 deletions Charts/fastcs/values.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@
"description": "command to run for the production container",
"type": "array"
},
"configFolder": {
"description": "path to the location of config folder",
"type": "string"
},
"debugCommand": {
"description": "command to run for the debugging (non- autostart) container",
"type": "array",
Expand All @@ -45,6 +49,21 @@
"description": "editable and autostart are used for debugging and development in-cluster",
"type": "boolean"
},
"env": {
"description": "Add environment to individual services",
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"value": {
"type": "string"
}
}
}
},
"extraContainers": {
"description": "Add arbitrary containers to the pod",
"type": "array",
Expand All @@ -71,6 +90,23 @@
"global": {
"description": "shared values for all services",
"type": "object",
"properties": {
"env": {
"description": "Add environment to all services",
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"value": {
"type": "string"
}
}
}
}
},
"additionalProperties": true
},
"hostNetwork": {
Expand Down Expand Up @@ -99,9 +135,9 @@
"type": "string"
}
},
"iocConfig": {
"description": "path to the location of config folder (defaults to be the same as C++ IOCs)",
"type": "string"
"is_ioc": {
"description": "set to true of IOCs in services repos",
"type": "boolean"
},
"livenessProbe": {
"$ref": "https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.33.3/container.json#/properties/livenessProbe",
Expand All @@ -111,6 +147,14 @@
"$ref": "https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.33.3/podspec.json#/properties/nodeSelector",
"type": "object"
},
"opisClaim": {
"description": "name of the PVC to use to store opi files, leave blank for IOCs - will then default to ixx-epics-opis",
"type": "string"
},
"opisMountPoint": {
"description": "where to mount the opis volume inside the container",
"type": "string"
},
"podAnnotations": {
"description": "Add annotations to the pod",
"type": "object"
Expand Down
20 changes: 17 additions & 3 deletions Charts/fastcs/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,12 @@

# @schema description: shared values for all services
# @schema additionalProperties: true
global: {}
global:
# @schema description: Add environment to all services
env: []

# @schema description: Add environment to individual services
env: []

# @schema description: container image URI
image:
Expand Down Expand Up @@ -62,8 +67,17 @@ editable: false
# true: entrypoint is command above
autostart: true

# @schema description: path to the location of config folder (defaults to be the same as C++ IOCs)
iocConfig: /epics/ioc/config
# @schema description: path to the location of config folder
configFolder: /config

# @schema description: name of the PVC to use to store opi files, leave blank for IOCs - will then default to ixx-epics-opis
opisClaim: ""

# @schema description: where to mount the opis volume inside the container
opisMountPoint: ""

# @schema description: set to true of IOCs in services repos
is_ioc: false

# @schema description: service port for Channel Access
ca_server_port: 5064
Expand Down
Loading