Load user Lisp configuration

Author: luciusmagn

autolith.asd

@@ -47,6 +47,7 @@
                              (:file "overlays")
                              (:file "durable-mutations")
                              (:file "image-commits")
+                             (:file "user-init")
                              (:file "generations")
                              (:file "self-status")
                              (:file "self-discard")
@@ -111,6 +112,7 @@
                              (:file "release-script-tests")
                              (:file "release-server-tests")
                              (:file "application-tests")
+                             (:file "user-init-tests")
                              (:file "tests"))))
   :perform (asdf:test-op (operation component)
              (declare (ignore operation component))

docs/autolith-minimal-technical-spec.org

@@ -76,6 +76,13 @@ The active SBCL image contains:
 - Self-inspection and self-modification facilities.
 - Loaded Lisp dependencies.
 
+After the tracked system and selected private image commit load, Autolith loads
+=${XDG_CONFIG_HOME:-~/.config}/autolith/init.lisp= when it exists. This
+user-authored Common Lisp is the final executable configuration layer and
+therefore takes precedence over built-in and privately committed definitions.
+It executes with the user's full privileges, is loaded again when reconnecting
+a retained generation, and is never included in the pristine recovery image.
+
 This process is expected to evolve while it is running.
 
 Setup may install a preloaded baseline active image containing compiled Autolith
@@ -402,7 +409,7 @@ with a distinguished recovery status.
 
 * 8. Persistence
 
-Autolith persists eight kinds of state:
+Autolith persists nine kinds of state:
 
 - Source in Git.
 - Conversations as S-expression files.
@@ -412,6 +419,7 @@ Autolith persists eight kinds of state:
 - A private Git history retaining complete image-commit artifacts.
 - Working generations as saved Lisp cores and reconstruction scripts.
 - Worker images as immutable SBCL cores with manifests and durable notes.
+- User-authored executable configuration under the XDG config root.
 
 ** 8.1 Conversation Files
 
@@ -493,7 +501,20 @@ Every executed tool result records nonnegative CPU and real durations. Terminal
 presentation shows both durations. Calls rejected before execution and repaired
 unknown-outcome records omit timing rather than inventing a duration.
 
-** 8.2 Mutation Journal
+** 8.2 User Configuration
+
+The optional =${XDG_CONFIG_HOME:-~/.config}/autolith/init.lisp= file is the
+explicit entry point for user Lisp. Autolith loads it in the =AUTOLITH= package
+after the tracked system and selected private image commit, before the first
+provider request. A retained generation reconnect loads the current file again
+instead of relying on a copy captured in its core.
+
+The file is trusted executable code with the same user privileges as Autolith.
+A load failure is a structured startup error identifying the file and original
+condition. The file is separate from legacy mutation overlays and private image
+commits, so mutation rollback never discards user policy.
+
+** 8.3 Mutation Journal
 
 The mutation journal is append-only and records enough information to
 understand recent changes after a crash.
@@ -512,7 +533,7 @@ A journal entry should identify:
 The journal is for recovery and diagnosis. It does not replace private
 image commits or their complete replay scripts.
 
-** 8.3 Persistent Memories
+** 8.4 Persistent Memories
 
 Persistent memories are stored in one readable =memories.sexp= file beneath the
 data root. The file begins with a versioned header and continues as an
@@ -540,7 +561,7 @@ a short excerpt for selection; exact content remains available through
 =memory.search= and =memory.read=. Memory corruption degrades this catalog to a
 bounded unavailable notice instead of preventing unrelated model work.
 
-** 8.4 Workspace Agendas
+** 8.5 Workspace Agendas
 
 Workspace agendas are stored in one readable =agendas.sexp= file beneath the
 data root. The complete versioned state is published atomically with private

docs/guide.org

@@ -303,6 +303,19 @@ failures. =/later= lists the queue and =/later cancel ID= removes one entry.
 
 Use a double slash to send a user message that begins with a literal slash.
 
+* User Lisp configuration
+
+Autolith loads the optional
+=${XDG_CONFIG_HOME:-~/.config}/autolith/init.lisp= after its tracked system and
+selected private image commit, before it creates the provider request path. The
+file is read in the =AUTOLITH= package and is loaded again when reconnecting a
+retained generation, so it remains the final user-controlled configuration
+layer. It is ordinary Common Lisp running with the user's full privileges.
+
+This entry point is distinct from =overlays/*.lisp=, which exist only to migrate
+old self-modifications. User configuration is never built into the pristine
+recovery image.
+
 Conversations are append-only readable forms under
 =${XDG_DATA_HOME:-~/.local/share}/autolith/conversations/=. A truncated final form is
 ignored after a crash; malformed complete records are rejected. If a crash

src/application.lisp

@@ -206,6 +206,7 @@
     (configuration-ensure-directories preferred-configuration)
     (durable-mutations-load preferred-configuration)
     (let* ((overlay-failures (image-state-load preferred-configuration))
+           (user-init-pathname (user-init-load preferred-configuration))
            (reasoning-traces-p
              (preferences-reasoning-traces-p preferred-configuration))
            (permission-state (permissions-load preferred-configuration))
@@ -240,6 +241,7 @@
                                        :ui ui
                                        :permission-state permission-state
                                        :reasoning-traces-p reasoning-traces-p)))
+      (declare (ignore user-init-pathname))
       (setf (application-overlay-failures application) overlay-failures)
       (application--load-goal application)
       application)))
@@ -265,29 +267,34 @@
             :model (configuration-model previous)
             :reasoning-effort (configuration-reasoning-effort previous)
             :immutable-p effective-immutable-p))
+         (prepared-configuration
+           (progn
+             (configuration-ensure-directories retained-configuration)
+             (user-init-load retained-configuration)
+             retained-configuration))
          (reasoning-traces-p
-           (preferences-reasoning-traces-p retained-configuration))
-         (permission-state (permissions-load retained-configuration))
+           (preferences-reasoning-traces-p prepared-configuration))
+         (permission-state (permissions-load prepared-configuration))
          (recovery-conversation-id
            (let ((value (uiop:getenv "AUTOLITH_RECOVERY_CONVERSATION_ID")))
              (and (non-empty-string-p value) value)))
          (overlay-failures nil)
          (conversation
            (cond
              (conversation-id
-              (conversation-load-by-id retained-configuration conversation-id))
+              (conversation-load-by-id prepared-configuration conversation-id))
              (recovery-conversation-id
-              (conversation-load-by-id retained-configuration
+              (conversation-load-by-id prepared-configuration
                                        recovery-conversation-id))
              ((conversation-persisted-p retained-conversation)
               (conversation-load-by-id
-               retained-configuration
+               prepared-configuration
                (conversation-identifier retained-conversation)))
              (t
-              (conversation-create retained-configuration))))
+              (conversation-create prepared-configuration))))
          (configuration
            (application--configuration-for-conversation
-            retained-configuration
+            prepared-configuration
             conversation))
          (provider (provider-create
                     configuration

src/conditions.lisp

@@ -16,6 +16,19 @@
   ()
   (:documentation "A failure caused by invalid or unavailable configuration."))
 
+(define-condition user-init-error (configuration-error)
+  ((pathname
+    :initarg :pathname
+    :reader user-init-error-pathname
+    :type pathname
+    :documentation "The user initialization file that could not be loaded.")
+   (cause
+    :initarg :cause
+    :reader user-init-error-cause
+    :type serious-condition
+    :documentation "The underlying condition signaled while loading the file."))
+  (:documentation "Loading the user's executable Autolith configuration failed."))
+
 (define-condition working-directory-error (configuration-error)
   ((requested-path
     :initarg :requested-path

src/configuration.lisp

@@ -83,6 +83,11 @@
     :reader configuration-cache-root
     :type pathname
     :documentation "The root for replaceable caches and temporary artifacts.")
+   (config-root
+    :initarg :config-root
+    :reader configuration-config-root
+    :type pathname
+    :documentation "The root for user-authored executable configuration.")
    (codex-auth-path
     :initarg :codex-auth-path
     :reader configuration-codex-auth-path
@@ -189,6 +194,9 @@
          (cache-home (environment-directory
                       "XDG_CACHE_HOME"
                       (merge-pathnames ".cache/" home)))
+         (config-home (environment-directory
+                       "XDG_CONFIG_HOME"
+                       (merge-pathnames ".config/" home)))
          (codex-home (environment-directory
                       "CODEX_HOME"
                       (merge-pathnames ".codex/" home)))
@@ -220,6 +228,7 @@
                    :data-root (merge-pathnames "autolith/" data-home)
                    :state-root (merge-pathnames "autolith/" state-home)
                    :cache-root (merge-pathnames "autolith/" cache-home)
+                   :config-root (merge-pathnames "autolith/" config-home)
                    :codex-auth-path (merge-pathnames "auth.json" codex-home)
                    :model selected-model
                    :reasoning-effort selected-effort
@@ -253,6 +262,7 @@ Selecting a different model recomputes the context window for that model."
                  :data-root (configuration-data-root configuration)
                  :state-root (configuration-state-root configuration)
                  :cache-root (configuration-cache-root configuration)
+                 :config-root (configuration-config-root configuration)
                  :codex-auth-path (configuration-codex-auth-path configuration)
                  :model (or model (configuration-model configuration))
                  :reasoning-effort (or reasoning-effort
@@ -352,10 +362,11 @@ Selecting a different model recomputes the context window for that model."
 
 (-> configuration-ensure-directories (configuration) configuration)
 (defun configuration-ensure-directories (configuration)
-  "Create CONFIGURATION's private data, state, and cache directories."
+  "Create CONFIGURATION's private data, state, cache, and config directories."
   (dolist (directory (list (configuration-data-root configuration)
                            (configuration-state-root configuration)
-                           (configuration-cache-root configuration)))
+                           (configuration-cache-root configuration)
+                           (configuration-config-root configuration)))
     (ensure-directories-exist directory))
   configuration)
 
@@ -364,6 +375,11 @@ Selecting a different model recomputes the context window for that model."
   "Return the directory containing append-only conversation files."
   (merge-pathnames "conversations/" (configuration-data-root configuration)))
 
+(-> configuration-user-init-path (configuration) pathname)
+(defun configuration-user-init-path (configuration)
+  "Return the user-authored Lisp initialization pathname."
+  (merge-pathnames "init.lisp" (configuration-config-root configuration)))
+
 (-> configuration-memory-path (configuration) pathname)
 (defun configuration-memory-path (configuration)
   "Return the append-only persistent memory pathname."

src/user-init.lisp

@@ -0,0 +1,26 @@
+(in-package #:autolith)
+
+;;;; -- User Initialization --
+
+(defvar *user-init-loading-p* nil
+  "True only while Autolith loads the user's executable configuration.")
+
+(-> user-init-load (configuration) (option pathname))
+(defun user-init-load (configuration)
+  "Load CONFIGURATION's user initialization file and return its pathname.
+
+The file is read in the AUTOLITH package after tracked and privately committed
+definitions have loaded. It executes with the user's full privileges."
+  (let ((pathname (configuration-user-init-path configuration)))
+    (when (uiop:file-exists-p pathname)
+      (handler-case
+          (let ((*package* (find-package '#:autolith))
+                (*user-init-loading-p* t))
+            (load pathname :verbose nil :print nil)
+            pathname)
+        (serious-condition (cause)
+          (error 'user-init-error
+                 :message (format nil "Could not load user initialization at ~A: ~A"
+                                  pathname cause)
+                 :pathname pathname
+                 :cause cause))))))

tests/application-tests.lisp

@@ -1566,6 +1566,7 @@
             :data-root (configuration-data-root base)
             :state-root (configuration-state-root base)
             :cache-root (configuration-cache-root base)
+            :config-root (configuration-config-root base)
             :codex-auth-path (configuration-codex-auth-path base)
             :model (configuration-model base)
             :reasoning-effort (configuration-reasoning-effort base)

tests/memory-tests.lisp

@@ -13,6 +13,7 @@
                  :data-root (configuration-data-root configuration)
                  :state-root (configuration-state-root configuration)
                  :cache-root (configuration-cache-root configuration)
+                 :config-root (configuration-config-root configuration)
                  :codex-auth-path (configuration-codex-auth-path configuration)
                  :model (configuration-model configuration)
                  :reasoning-effort

tests/search-tool-tests.lisp

@@ -12,6 +12,7 @@
                    :data-root (configuration-data-root base)
                    :state-root (configuration-state-root base)
                    :cache-root (configuration-cache-root base)
+                   :config-root (configuration-config-root base)
                    :codex-auth-path (configuration-codex-auth-path base)
                    :model (configuration-model base)
                    :reasoning-effort (configuration-reasoning-effort base)