Skip to content

Commit 0afa898

Browse files
JolanrensenJolanrensen
committed
Merge branch 'master' into expression-converter-kt2.1
2 parents c62a82f + 835d7e3 commit 0afa898

File tree

945 files changed

+397134
-49592
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

945 files changed

+397134
-49592
lines changed

‎.github/workflows/generated-sources-master.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,11 @@ jobs:
1313
- name: Checkout repository
1414
uses: actions/checkout@v4
1515

16-
- name: Set up JDK 11
16+
- name: Set up JDK 21
1717
uses: actions/setup-java@v4
1818
with:
1919
distribution: 'temurin'
20-
java-version: '11'
20+
java-version: '21'
2121

2222
- name: Run Gradle task
2323
run: ./gradlew :core:processKDocsMain korro

‎.github/workflows/generated-sources.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -26,11 +26,11 @@ jobs:
2626
- name: Checkout repository
2727
uses: actions/checkout@v4
2828

29-
- name: Set up JDK 11
29+
- name: Set up JDK 21
3030
uses: actions/setup-java@v4
3131
with:
3232
distribution: 'temurin'
33-
java-version: '11'
33+
java-version: '21'
3434

3535
- name: Configure Git User
3636
run: |

‎CONTRIBUTING.md

+8-3
Original file line numberDiff line numberDiff line change
@@ -85,9 +85,9 @@ so do familiarize yourself with the following guidelines.
8585

8686
## Environment requirements
8787

88-
* JDK >= 11 referred to by the `JAVA_HOME` environment variable.
88+
* JDK >= 21 referred to by the `JAVA_HOME` environment variable.
8989

90-
* Note, any version above 11 should work in theory, but JDK 11 is the only version we test with,
90+
* Note, any version above 21 should work in theory, but JDK 21 is the only version we test with,
9191
so it is the recommended version.
9292

9393
* We recommend using [IntelliJ IDEA](https://www.jetbrains.com/idea/download/) as the IDE. This
@@ -96,7 +96,8 @@ has the best support for Kotlin, compiler plugins, Gradle, and [Kotlin Notebook]
9696
* We recommend using the [Ktlint plugin](https://plugins.jetbrains.com/plugin/15057-ktlint) for [IntelliJ IDEA](https://www.jetbrains.com/idea/download/).
9797
It is able to read the `.editorconfig` file and apply the same formatting rules as [Ktlint](https://pinterest.github.io/ktlint/latest/) in the CI.
9898

99-
* Check out the [KDoc Preprocessor guide](KDOC_PREPROCESSING.md) to understand how to work with the KDoc preprocessor.
99+
* Check out the [KDoc Preprocessor guide](KDOC_PREPROCESSING.md) to understand how to work with
100+
[KoDEx](https://github.com/Jolanrensen/KoDEx).
100101

101102
## Building
102103

@@ -107,6 +108,10 @@ This library is built with Gradle.
107108
things up during development.
108109
* Make sure to pass the extra parameter `-Pkotlin.dataframe.debug=true` to enable debug mode. This flag will
109110
make sure some extra checks are run, which are important but too heavy for production.
111+
* The parameter `-PskipKodex` allows you to skip [kdoc processing](KDOC_PREPROCESSING.md),
112+
making local publishing faster: `./gradlew publishToMavenLocal -PskipKodex`.
113+
This, however, publishes the library with "broken" KDocs,
114+
so it's only meant for faster iterations during development.
110115

111116
You can import this project into IDEA, but you have to delegate the build actions
112117
to Gradle (in Preferences -> Build, Execution, Deployment -> Build Tools -> Gradle -> Runner)

‎KDOC_PREPROCESSING.md

+118-91
Large diffs are not rendered by default.

‎README.md

+12-12
Original file line numberDiff line numberDiff line change
@@ -215,21 +215,21 @@ Check it out on [**Datalore**](https://datalore.jetbrains.com/view/notebook/vq5j
215215

216216
Explore [**more examples here**](examples).
217217

218-
## Kotlin, Kotlin Jupyter, OpenAPI, Arrow and JDK versions
218+
## Kotlin, Kotlin Jupyter, Arrow, and JDK versions
219219

220220
This table shows the mapping between main library component versions and minimum supported Java versions.
221221

222-
| Kotlin DataFrame Version | Minimum Java Version | Kotlin Version | Kotlin Jupyter Version | OpenAPI version | Apache Arrow version |
223-
|--------------------------|----------------------|----------------|------------------------|-----------------|----------------------|
224-
| 0.10.0 | 8 | 1.8.20 | 0.11.0-358 | 3.0.0 | 11.0.0 |
225-
| 0.10.1 | 8 | 1.8.20 | 0.11.0-358 | 3.0.0 | 11.0.0 |
226-
| 0.11.0 | 8 | 1.8.20 | 0.11.0-358 | 3.0.0 | 11.0.0 |
227-
| 0.11.1 | 8 | 1.8.20 | 0.11.0-358 | 3.0.0 | 11.0.0 |
228-
| 0.12.0 | 8 | 1.9.0 | 0.11.0-358 | 3.0.0 | 11.0.0 |
229-
| 0.12.1 | 8 | 1.9.0 | 0.11.0-358 | 3.0.0 | 11.0.0 |
230-
| 0.13.1 | 8 | 1.9.22 | 0.12.0-139 | 3.0.0 | 15.0.0 |
231-
| 0.14.1 | 8 | 2.0.20 | 0.12.0-139 | 3.0.0 | 17.0.0 |
232-
| 0.15.0 | 8 | 2.0.20 | 0.12.0-139 | 3.0.0 | 18.1.0 |
222+
| Kotlin DataFrame Version | Minimum Java Version | Kotlin Version | Kotlin Jupyter Version | Apache Arrow version |
223+
|--------------------------|----------------------|----------------|------------------------|----------------------|
224+
| 0.10.0 | 8 | 1.8.20 | 0.11.0-358 | 11.0.0 |
225+
| 0.10.1 | 8 | 1.8.20 | 0.11.0-358 | 11.0.0 |
226+
| 0.11.0 | 8 | 1.8.20 | 0.11.0-358 | 11.0.0 |
227+
| 0.11.1 | 8 | 1.8.20 | 0.11.0-358 | 11.0.0 |
228+
| 0.12.0 | 8 | 1.9.0 | 0.11.0-358 | 11.0.0 |
229+
| 0.12.1 | 8 | 1.9.0 | 0.11.0-358 | 11.0.0 |
230+
| 0.13.1 | 8 | 1.9.22 | 0.12.0-139 | 15.0.0 |
231+
| 0.14.1 | 8 | 2.0.20 | 0.12.0-139 | 17.0.0 |
232+
| 0.15.0 | 8 | 2.0.20 | 0.12.0-139 | 18.1.0 |
233233

234234
## Code of Conduct
235235

‎RELEASE_CHECK_LIST.md

+1
Original file line numberDiff line numberDiff line change
@@ -37,5 +37,6 @@
3737
- `Level.ERROR` messages and their functions are removed.
3838
- Update regions in the file accordingly.
3939
21. Update Notebook examples, both in the project and on Datalore.
40+
- Dev notebooks may contain updated code for release Notebook examples.
4041

4142
(Activities that need to be done for **R**elease **C**andidate releases are marked as such)

‎build.gradle.kts

+65-35
Original file line numberDiff line numberDiff line change
@@ -19,12 +19,11 @@ plugins {
1919
alias(kotlin.jvm)
2020
alias(publisher)
2121
alias(serialization) apply false
22-
alias(jupyter.api) apply false
2322
alias(dokka)
2423
alias(kover)
2524
alias(ktlint)
2625
alias(korro) apply false
27-
alias(docProcessor) apply false
26+
alias(kodex) apply false
2827
alias(simpleGit) apply false
2928
alias(dependencyVersions)
3029
alias(buildconfig) apply false
@@ -50,21 +49,27 @@ configurations {
5049
}
5150

5251
dependencies {
53-
api(project(":core"))
54-
api(project(":dataframe-arrow"))
55-
api(project(":dataframe-excel"))
56-
api(project(":dataframe-openapi"))
57-
api(project(":dataframe-jdbc"))
58-
// TODO enable when it leaves the experimental phase
59-
// api(project(":dataframe-csv"))
60-
61-
kover(project(":core"))
62-
kover(project(":dataframe-arrow"))
63-
kover(project(":dataframe-excel"))
64-
kover(project(":dataframe-openapi"))
65-
kover(project(":dataframe-jdbc"))
66-
kover(project(":dataframe-csv"))
67-
kover(project(":plugins:kotlin-dataframe"))
52+
api(projects.core)
53+
54+
// expose all optional IO dependencies by default
55+
api(projects.dataframeArrow)
56+
api(projects.dataframeExcel)
57+
api(projects.dataframeJdbc)
58+
api(projects.dataframeCsv)
59+
api(projects.dataframeJson)
60+
61+
// experimental, so not included by default:
62+
// api(projects.dataframeOpenapi)
63+
64+
kover(projects.core)
65+
kover(projects.dataframeArrow)
66+
kover(projects.dataframeExcel)
67+
kover(projects.dataframeOpenapi)
68+
kover(projects.dataframeJdbc)
69+
kover(projects.dataframeCsv)
70+
kover(projects.dataframeJson)
71+
kover(projects.plugins.kotlinDataframe)
72+
kover(projects.dataframeJupyter)
6873
}
6974

7075
enum class Version : Comparable<Version> {
@@ -90,16 +95,10 @@ fun String.findVersion(): Version {
9095

9196
// these names of outdated dependencies will not show up in the table output
9297
val dependencyUpdateExclusions = listOf(
93-
// TODO Requires more work to be updated to 1.7.0+, https://github.com/Kotlin/dataframe/issues/594
94-
libs.plugins.kover.get().pluginId,
95-
// TODO 5.8.0 is not possible due to https://github.com/Kotlin/dataframe/issues/595
96-
libs.kotestAssertions.get().name,
97-
// Can't be updated to 7.4.0+ due to Java 8 compatibility
98-
libs.android.gradle.api.get().group,
9998
// Directly dependent on the Gradle version
10099
"org.gradle.kotlin.kotlin-dsl",
101-
// Can't be updated to 2.1.0+ due to Java 8 compatibility
102-
libs.plugins.simpleGit.get().pluginId,
100+
// need to revise our tests to update
101+
libs.android.gradle.api.get().group,
103102
)
104103

105104
// run `./gradlew dependencyUpdates` to check for updates
@@ -140,18 +139,49 @@ tasks.named<DependencyUpdatesTask>("dependencyUpdates").configure {
140139
}
141140
}
142141

143-
kotlin.jvmToolchain(11)
142+
kotlin {
143+
jvmToolchain(21)
144+
compilerOptions {
145+
jvmTarget = JvmTarget.JVM_1_8
146+
}
147+
}
148+
149+
// DataFrame targets Java 8 for maximum compatibility.
150+
// This is, however, not always possible thanks to external dependencies.
151+
// In those cases, we default to Java 11.
152+
val modulesUsingJava11 = with(projects) {
153+
setOf(
154+
dataframeJupyter,
155+
dataframeGeo,
156+
examples.ideaExamples.titanic,
157+
)
158+
}.map { it.path }
144159

145160
allprojects {
146-
tasks.withType<KotlinCompile> {
147-
compilerOptions {
148-
jvmTarget = JvmTarget.JVM_1_8
161+
if (path in modulesUsingJava11) {
162+
tasks.withType<KotlinCompile> {
163+
compilerOptions {
164+
jvmTarget = JvmTarget.JVM_11
165+
freeCompilerArgs.add("-Xjdk-release=11")
166+
}
167+
}
168+
tasks.withType<JavaCompile> {
169+
sourceCompatibility = JavaVersion.VERSION_11.toString()
170+
targetCompatibility = JavaVersion.VERSION_11.toString()
171+
options.release.set(11)
172+
}
173+
} else {
174+
tasks.withType<KotlinCompile> {
175+
compilerOptions {
176+
jvmTarget = JvmTarget.JVM_1_8
177+
freeCompilerArgs.add("-Xjdk-release=8")
178+
}
179+
}
180+
tasks.withType<JavaCompile> {
181+
sourceCompatibility = JavaVersion.VERSION_1_8.toString()
182+
targetCompatibility = JavaVersion.VERSION_1_8.toString()
183+
options.release.set(8)
149184
}
150-
}
151-
152-
tasks.withType<JavaCompile> {
153-
sourceCompatibility = JavaVersion.VERSION_1_8.toString()
154-
targetCompatibility = JavaVersion.VERSION_1_8.toString()
155185
}
156186

157187
// Attempts to configure ktlint for each sub-project that uses the plugin
@@ -166,7 +196,7 @@ allprojects {
166196
}
167197

168198
// set the java toolchain version to 11 for all subprojects for CI stability
169-
extensions.findByType<KotlinJvmProjectExtension>()?.jvmToolchain(11)
199+
extensions.findByType<KotlinJvmProjectExtension>()?.jvmToolchain(21)
170200

171201
// Attempts to configure buildConfig for each sub-project that uses it
172202
try {

‎core/README.md

+4-5
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,9 @@ or [:dataframe-jdbc](../dataframe-jdbc), however, this is has not happened yet f
99

1010
At the moment, these integrations are still part of the `:core` module:
1111

12-
- csv/tsv
12+
- (deprecated) csv/tsv
1313
- html
1414
- json
15-
- jupyter
1615

1716
### Korro code samples
1817

@@ -36,10 +35,10 @@ We can also generate "normal" DataFrame samples for the website. This can be don
3635
[OtherSamples class](./src/test/kotlin/org/jetbrains/kotlinx/dataframe/samples/api/OtherSamples.kt). Generated
3736
HTML files will be stored in [docs/StardustDocs/snippets/manual](../docs/StardustDocs/snippets/manual).
3837

39-
### KDoc Preprocesor
38+
### KoDEx
4039

41-
The code you're working on needs to be edited in [src](src), but the KDocs are processed by the
42-
[KDoc Preprocessor](https://github.com/Jolanrensen/docProcessorGradlePlugin) when the project is published (or the task
40+
The code you're working on needs to be edited in [src](src), but the KDocs are processed by
41+
[KoDEx](https://github.com/Jolanrensen/kodex) when the project is published (or the task
4342
is run manually). The generated sources with adjusted KDocs will be overwritten
4443
in [generated-sources](generated-sources).
4544
See the [KDoc Preprocessing Guide](../KDOC_PREPROCESSING.md) for more information.

0 commit comments

Comments
 (0)