split, fix and cleanup predef and runBefore handling (#176)

Prior to 0.1.92 we concatenated and executed all predef code. That was
very simple but had quite
a few downsides, including incorrect line number reporting and some edge
cases when handling complex
compilation units, e.g. ordering of imports etc. 

#157 changed this
behaviour to instead compile
the given files and add the results to the classpath. At the same time
we unintendedly broke
the ability to execute code on startup of the repl - that's reintroduced
as a new parameter
 `--runBefore` now.

Author: mpollmeier

README.md

@@ -20,22 +20,22 @@ srp
 Prerequisite: jdk11+
 
 ## TOC
-<!-- markdown-toc --maxdepth 3 README.md|tail -n +4 -->
 - [Benefits over / comparison with](#benefits-over--comparison-with)
   * [Regular Scala REPL](#regular-scala-repl)
   * [Ammonite](#ammonite)
   * [scala-cli](#scala-cli)
 - [Prerequisite for all of the below: run `sbt stage` or download the latest release](#prerequisite-for-all-of-the-below-run-sbt-stage-or-download-the-latest-release)
 - [REPL](#repl)
   * [run with defaults](#run-with-defaults)
-  * [customize prompt, greeting and exit code](#customize-prompt-greeting-and-exit-code)
-  * [execute some predef code](#execute-some-predef-code)
+  * [execute code at the start with `--runBefore`](#execute-code-at-the-start-with---runbefore)
+  * [`--predef`: code that is compiled but not executed](#--predef-code-that-is-compiled-but-not-executed)
   * [Operators: Redirect to file, pipe to external command](#operators-redirect-to-file-pipe-to-external-command)
   * [Add dependencies with maven coordinates](#add-dependencies-with-maven-coordinates)
   * [Importing additional script files interactively](#importing-additional-script-files-interactively)
   * [Adding classpath entries](#adding-classpath-entries)
   * [Rendering of output](#rendering-of-output)
   * [Exiting the REPL](#exiting-the-repl)
+  * [customize prompt, greeting and exit code](#customize-prompt-greeting-and-exit-code)
   * [Looking up the current terminal width](#looking-up-the-current-terminal-width)
 - [Scripting](#scripting)
   * [Simple "Hello world" script](#simple-hello-world-script)
@@ -49,7 +49,6 @@ Prerequisite: jdk11+
   * [Attach a debugger (remote jvm debug)](#attach-a-debugger-remote-jvm-debug)
 - [Server mode](#server-mode)
 - [Embed into your own project](#embed-into-your-own-project)
-- [Global predef file: `~/.scala-repl-pp.sc`](#global-predef-file-scala-repl-ppsc)
 - [Verbose mode](#verbose-mode)
 - [Inherited classpath](#inherited-classpath)
 - [Parameters cheat sheet: the most important ones](#parameters-cheat-sheet-the-most-important-ones)
@@ -62,7 +61,8 @@ Prerequisite: jdk11+
   * [How can I get a new binary (bootstrapped) release?](#how-can-i-get-a-new-binary-bootstrapped-release)
   * [Updating the Scala version](#updating-the-scala-version)
   * [Updating the shaded libraries](#updating-the-shaded-libraries)
-- [Fineprint](#fineprint)  
+- [Fineprint](#fineprint)
+
 
 ## Benefits over / comparison with
 
@@ -100,10 +100,28 @@ scala-cli wraps and invokes the regular Scala REPL (by default; or optionally Am
 ./srp
 ```
 
-### customize prompt, greeting and exit code
-./srp --prompt myprompt --greeting 'hey there!' --onExitCode 'println("see ya!")'
+### execute code at the start with `--runBefore`
+```
+./srp --runBefore 'import Byte.MaxValue'
+
+scala> MaxValue
+val res0: Int = 127
+```
 
-### execute some predef code
+You can specify this parameter multiple times, the given statements will be executed in the given order.
+
+If you want to execute some code _every single time_ you start a session, just write it to `~/.scala-repl-pp.sc`
+```
+echo 'import Short.MaxValue' > ~/.scala-repl-pp.sc
+
+./srp
+
+scala> MaxValue
+val res0: Int = 32767
+```
+
+### `--predef`: add source files to the classpath
+Additional source files that are compiled added to the classpath, but unlike `runBefore` not executed straight away can be provided via `--predef`. 
 ```
 echo 'def foo = 42' > foo.sc
 
@@ -112,6 +130,12 @@ scala> foo
 val res0: Int = 42
 ```
 
+You can specify this parameter multiple times (`--predef one.sc --predef two.sc`).
+
+Why not use `runBefore` instead? For simple examples like the one above, you can do so. If it gets more complicated and you have multiple files referencing each others, `predef` allows you to treat it as one compilation unit, which isn't possible with `runBefore`. And as you add more code it's get's easier to manage in files rather than command line arguments. 
+
+Note that predef files may not contain toplevel statements like `println("foo")` - instead, these either belong into your main script (if you're executing one) and/or can be passed to the repl via `runBefore`.
+
 ### Operators: Redirect to file, pipe to external command
 Inspired by unix shell redirection and pipe operators (`>`, `>>` and `|`) you can redirect output into files with `#>` (overrides existing file) and `#>>` (create or append to file), and use `#|` to pipe the output to a command, such as `less`:
 ```scala
@@ -274,6 +298,15 @@ $
 ```
 Context: we'd prefer to cancel the long-running operation, but that's not so easy on the JVM.
 
+### customize prompt, greeting and exit code
+```
+./srp --prompt myprompt --greeting 'hey there!' --onExitCode 'println("see ya!")'
+
+hey there!
+myprompt> :exit
+see ya!
+```
+
 ### Looking up the current terminal width
 In case you want to adjust your output rendering to the available terminal size, you can look it up:
 
@@ -487,26 +520,6 @@ stringcalc> add(One, Two)
 val res0: stringcalc.Number = Number(3)
 ```
 
-## Global predef file: `~/.scala-repl-pp.sc`
-Code that should be available across all srp sessions can be written into your local `~/.scala-repl-pp.sc`. 
-
-```
-echo 'def bar = 90' > ~/.scala-repl-pp.sc
-echo 'def baz = 91' > script1.sc
-echo 'def bam = 92' > script2.sc
-
-./srp --predef script1.sc --predef script2.sc
-
-scala> bar
-val res0: Int = 90
-
-scala> baz
-val res1: Int = 91
-
-scala> bam
-val res2: Int = 92
-```
-
 ## Verbose mode
 If verbose mode is enabled, you'll get additional information about classpaths and complete scripts etc. 
 To enable it, you can either pass `--verbose` or set the environment variable `SCALA_REPL_PP_VERBOSE=true`.

build.sbt

@@ -26,7 +26,7 @@ lazy val core = project.in(file("core"))
     executableScriptName := "srp",
     libraryDependencies ++= Seq(
       "org.scala-lang" %% "scala3-compiler" % scalaVersion.value,
-      "org.slf4j"       % "slf4j-simple" % "2.0.13" % Optional,
+      "org.slf4j"       % "slf4j-simple"    % "2.0.16" % Optional,
     ),
     assemblyJarName := "srp.jar", // TODO remove the '.jar' suffix - when doing so, it doesn't work any longer
   )

core/src/main/scala/replpp/Config.scala

@@ -8,7 +8,8 @@ import replpp.shaded.scopt.OParserBuilder
 import java.nio.file.Path
 
 case class Config(
-  predefFiles: Seq[Path] = Nil,
+  predefFiles: Seq[Path] = Nil, // these files will be precompiled and added to the classpath
+  runBefore: Seq[String] = Nil, // these statements will be interpreted on startup
   nocolors: Boolean = false,
   verbose: Boolean = false,
   classpathConfig: Config.ForClasspath = Config.ForClasspath(),
@@ -41,6 +42,10 @@ case class Config(
       add("--predef", predefFile.toString)
     }
 
+    runBefore.foreach { runBefore =>
+      add("--runBefore", runBefore)
+    }
+
     if (nocolors) add("--nocolors")
     if (verbose) add("--verbose")
 
@@ -103,6 +108,7 @@ object Config {
     OParser.sequence(
       programName("scala-repl-pp"),
       opts.predef((x, c) => c.copy(predefFiles = c.predefFiles :+ x)),
+      opts.runBefore((x, c) => c.copy(runBefore = c.runBefore :+ x)),
       opts.nocolors((_, c) => c.copy(nocolors = true)),
       opts.verbose((_, c) => c.copy(verbose = true)),
       opts.classpathEntry((x, c) => c.copy(classpathConfig = c.classpathConfig.copy(additionalClasspathEntries = c.classpathConfig.additionalClasspathEntries :+ x))),
@@ -142,7 +148,16 @@ object Config {
         .unbounded()
         .optional()
         .action(action)
-        .text("import additional script files on startup - may be passed multiple times")
+        .text("given source files will be compiled and added to classpath - this may be passed multiple times")
+    }
+
+    def runBefore[C](using builder: OParserBuilder[C])(action: Action[String, C]) = {
+      builder.opt[String]("runBefore")
+        .valueName("'val foo = 42'")
+        .unbounded()
+        .optional()
+        .action(action)
+        .text("given code will be executed on startup - this may be passed multiple times")
     }
 
     def nocolors[C](using builder: OParserBuilder[C])(action: Action[Unit, C]) = {

core/src/main/scala/replpp/InteractiveShell.scala

@@ -2,6 +2,7 @@ package replpp
 
 import dotty.tools.repl.State
 
+import java.lang.System.lineSeparator
 import scala.util.control.NoStackTrace
 
 object InteractiveShell {
@@ -21,22 +22,22 @@ object InteractiveShell {
     )
 
     val initialState: State = replDriver.initialState
-    val predefCode = DefaultPredef
+    val runBeforeLines = (DefaultRunBeforeLines ++ globalRunBeforeLines ++ config.runBefore).mkString(lineSeparator)
     val state: State = {
       if (verboseEnabled(config)) {
         println(s"compiler arguments: ${compilerArgs.mkString(",")}")
-        println(predefCode)
-        replDriver.run(predefCode)(using initialState)
+        println(runBeforeLines)
+        replDriver.run(runBeforeLines)(using initialState)
       } else {
-        replDriver.runQuietly(predefCode)(using initialState)
+        replDriver.runQuietly(runBeforeLines)(using initialState)
       }
     }
 
-    if (predefCode.nonEmpty && state.objectIndex != 1) {
+    if (runBeforeLines.nonEmpty && state.objectIndex != 1) {
       throw new AssertionError(s"compilation error for predef code - error should have been reported above ^") with NoStackTrace
     }
 
     replDriver.runUntilQuit(using state)()
   }
 
-}
+}

core/src/main/scala/replpp/UsingDirectives.scala

@@ -25,22 +25,6 @@ object UsingDirectives {
   def findImportedFiles(lines: IterableOnce[String], rootPath: Path): Seq[Path] =
     scanFor(FileDirective, lines).iterator.map(resolveFile(rootPath, _)).toSeq
 
-  def findImportedFilesRecursively(lines: IterableOnce[String], rootPath: Path): Seq[Path] = {
-    val results = Seq.newBuilder[Path]
-    val visited = mutable.Set.empty[Path]
-
-    findImportedFiles(lines, rootPath).foreach { file =>
-      results += file
-      visited += file
-
-      val recursiveFiles = findImportedFilesRecursively(file, visited.toSet)
-      results ++= recursiveFiles
-      visited ++= recursiveFiles
-    }
-
-    results.result().distinct
-  }
-
   def findDeclaredDependencies(lines: IterableOnce[String]): Seq[String] =
     scanFor(LibDirective, lines)
 

core/src/main/scala/replpp/package.scala

@@ -1,6 +1,5 @@
 import replpp.util.{ClasspathHelper, SimpleDriver, linesFromFile}
 
-import java.lang.System.lineSeparator
 import java.nio.file.{Files, Path, Paths}
 import scala.collection.mutable
 
@@ -9,22 +8,21 @@ package object replpp {
   val VerboseEnvVar    = "SCALA_REPL_PP_VERBOSE"
   lazy val pwd: Path = Paths.get(".").toAbsolutePath
   lazy val home: Path = Paths.get(System.getProperty("user.home"))
-  lazy val globalPredefFile = home.resolve(".scala-repl-pp.sc")
-  lazy val globalPredefFileMaybe = Option(globalPredefFile).filter(Files.exists(_))
+  lazy val globalRunBeforeFile: Path = home.resolve(".scala-repl-pp.sc")
+  lazy val globalRunBeforeFileMaybe: Option[Path] = Option(globalRunBeforeFile).filter(Files.exists(_))
+  lazy val globalRunBeforeLines: Seq[String] = globalRunBeforeFileMaybe.map(linesFromFile).getOrElse(Seq.empty)
 
-  private[replpp] def DefaultPredefLines(using colors: Colors) = {
+  private[replpp] def DefaultRunBeforeLines(using colors: Colors) = {
     val colorsImport = colors match {
       case Colors.BlackWhite => "replpp.Colors.BlackWhite"
       case Colors.Default => "replpp.Colors.Default"
     }
     Seq(
       "import replpp.Operators.*",
-      s"given replpp.Colors = $colorsImport"
+      s"given replpp.Colors = $colorsImport",
     )
   }
 
-  private[replpp] def DefaultPredef(using Colors) = DefaultPredefLines.mkString(lineSeparator)
-
   /** verbose mode can either be enabled via the config, or the environment variable `SCALA_REPL_PP_VERBOSE=true` */
   def verboseEnabled(config: Config): Boolean = {
     config.verbose ||
@@ -44,14 +42,13 @@ package object replpp {
   /** recursively find all relevant source files from main script, global predef file, 
     * provided predef files, other scripts that were imported with `using file` directive */
   def allSourceFiles(config: Config): Seq[Path] =
-    (allPredefFiles(config) ++ config.scriptFile).distinct.sorted
+    (allPredefFiles(config) ++ config.scriptFile ++ globalRunBeforeFileMaybe).distinct.sorted
 
   def allPredefFiles(config: Config): Seq[Path] = {
     val allPredefFiles  = mutable.Set.empty[Path]
     allPredefFiles ++= config.predefFiles
-    globalPredefFileMaybe.foreach(allPredefFiles.addOne)
 
-    // the directly resolved predef files might reference additional files via `using` directive
+    // the directly referenced predef files might reference additional files via `using` directive
     val predefFilesDirect = allPredefFiles.toSet
     predefFilesDirect.foreach { predefFile =>
       val importedFiles = UsingDirectives.findImportedFilesRecursively(predefFile, visited = allPredefFiles.toSet)
@@ -64,19 +61,18 @@ package object replpp {
       allPredefFiles ++= importedFiles
     }
 
-    allPredefFiles.toSeq.sorted
+    allPredefFiles.toSeq.filter(Files.exists(_)).sorted
   }
 
   def allSourceLines(config: Config): Seq[String] =
-    allSourceFiles(config).flatMap(linesFromFile)
+    allSourceFiles(config).flatMap(linesFromFile) ++ config.runBefore
 
   /** precompile given predef files (if any) and update Config to include the results in the classpath */
   def precompilePredefFiles(config: Config): Config = {
-    val allPredefFiles = (config.predefFiles :+ globalPredefFile).filter(Files.exists(_))
-    if (allPredefFiles.nonEmpty) {
+    if (config.predefFiles.nonEmpty) {
       val predefClassfilesDir = new SimpleDriver().compileAndGetOutputDir(
         replpp.compilerArgs(config),
-        inputFiles = allPredefFiles,
+        inputFiles = allPredefFiles(config),
         verbose = config.verbose
       ).get
       config.withAdditionalClasspathEntry(predefClassfilesDir)

core/src/main/scala/replpp/scripting/NonForkingScriptRunner.scala

@@ -1,6 +1,6 @@
 package replpp.scripting
 
-import replpp.{Config, allPredefFiles, allSourceFiles}
+import replpp.{Config, allPredefFiles}
 
 import java.nio.file.Files
 import scala.util.{Failure, Success}
@@ -36,6 +36,9 @@ object NonForkingScriptRunner {
       commandArgs ++ parameterArgs
     }
 
+    if (config.runBefore.nonEmpty)
+      println(s"[WARNING] ScriptingDriver does not support `runBefore` code, the given ${config.runBefore.size} statements will be ignored")
+
     val verboseEnabled = replpp.verboseEnabled(config)
     new ScriptingDriver(
       compilerArgs = replpp.compilerArgs(config) :+ "-nowarn",
@@ -52,4 +55,4 @@ object NonForkingScriptRunner {
     }
   }
 
-}
+}

core/src/main/scala/replpp/scripting/ScriptingDriver.scala

@@ -17,7 +17,7 @@ import scala.util.{Failure, Try}
   * Runs a given script on the current JVM.
   *
   * Similar to dotty.tools.scripting.ScriptingDriver, but simpler and faster.
-  * Main difference: we don't (need to) recursively look for main method entrypoints in the entire classpath,
+  * Main difference: we don't (need to) recursively look for main method entry points in the entire classpath,
   * because we have a fixed class and method name that ScriptRunner uses when it embeds the script and predef code.
   * */
 class ScriptingDriver(compilerArgs: Array[String], predefFiles: Seq[Path], scriptFile: Path, scriptArgs: Array[String], verbose: Boolean) {

core/src/test/scala/replpp/ConfigTests.scala

@@ -19,6 +19,7 @@ class ConfigTests extends AnyWordSpec with Matchers {
   "asJavaArgs (inverse of Config.parse)" in {
     val config = Config(
       predefFiles = List(Paths.get("/some/path/predefFile1"), Paths.get("/some/path/predefFile2")),
+      runBefore = List("val foo = 42", "println(foo)"),
       nocolors = true,
       verbose = true,
       classpathConfig = Config.ForClasspath(
@@ -39,6 +40,8 @@ class ConfigTests extends AnyWordSpec with Matchers {
     javaArgs shouldBe Seq(
       "--predef", Paths.get("/some/path/predefFile1").toString,
       "--predef", Paths.get("/some/path/predefFile2").toString,
+      "--runBefore", "val foo = 42",
+      "--runBefore", "println(foo)",
       "--nocolors",
       "--verbose",
       "--classpathEntry", "cp1",