Skip to content

Commit 8d2543b

Browse files
mina-ashammina-asham
authored
Merge branch 'master' into minaasham/upgrade-replace-avro-python3
2 parents 2bddaa7 + 9ba5afc commit 8d2543b

File tree

414 files changed

+15049
-1823
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

414 files changed

+15049
-1823
lines changed

‎.github/workflows/nightly-trigger.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,6 @@ jobs:
3333
- master
3434
- release-1.20
3535
- release-1.19
36-
- release-1.18
3736
runs-on: ubuntu-latest
3837
steps:
3938
- name: Trigger Workflow

‎docs/config.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,11 +34,11 @@ pygmentsUseClasses = true
3434
# we change the version for the complete docs when forking of a release branch
3535
# etc.
3636
# The full version string as referenced in Maven (e.g. 1.2.1)
37-
Version = "2.0-SNAPSHOT"
37+
Version = "2.1-SNAPSHOT"
3838

3939
# For stable releases, leave the bugfix version out (e.g. 1.2). For snapshot
4040
# release this should be the same as the regular version
41-
VersionTitle = "2.0-SNAPSHOT"
41+
VersionTitle = "2.1-SNAPSHOT"
4242

4343
# The branch for this version of Apache Flink
4444
Branch = "master"

‎docs/content.zh/docs/deployment/cli.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -576,7 +576,7 @@ related options. Here's an overview of all the Python related options for the ac
576576
<td>
577577
Specify the path of the python interpreter used to execute the python UDF worker
578578
(e.g.: --pyExecutable /usr/local/bin/python3).
579-
The python UDF worker depends on Python 3.8+, Apache Beam (version == 2.43.0),
579+
The python UDF worker depends on Python 3.8+, Apache Beam (version >= 2.54.0, <= 2.61.0),
580580
Pip (version >= 20.3) and SetupTools (version >= 37.0.0).
581581
Please ensure that the specified environment meets the above requirements.
582582
</td>

‎docs/content.zh/docs/dev/table/sqlClient.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -322,7 +322,7 @@ Mode "embedded" (default) submits Flink jobs from the local machine.
322322
/usr/local/bin/python3). The
323323
python UDF worker depends on
324324
Python 3.8+, Apache Beam
325-
(version == 2.43.0), Pip
325+
(version >= 2.54.0, <= 2.61.0), Pip
326326
(version >= 20.3) and SetupTools
327327
(version >= 37.0.0). Please
328328
ensure that the specified

‎docs/content.zh/docs/ops/metrics.md

Lines changed: 48 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1545,31 +1545,33 @@ Certain RocksDB native metrics are available but disabled by default, you can fi
15451545
<table class="table table-bordered">
15461546
<thead>
15471547
<tr>
1548-
<th class="text-left" style="width: 18%">Scope</th>
1549-
<th class="text-left" style="width: 26%">Metrics</th>
1550-
<th class="text-left" style="width: 48%">Description</th>
1551-
<th class="text-left" style="width: 8%">Type</th>
1548+
<th class="text-left" style="width: 15%">Scope</th>
1549+
<th class="text-left" style="width: 15%">Infix</th>
1550+
<th class="text-left" style="width: 15%">Metrics</th>
1551+
<th class="text-left" style="width: 50%">Description</th>
1552+
<th class="text-left" style="width: 5%">Type</th>
15521553
</tr>
15531554
</thead>
15541555
<tbody>
15551556
<tr>
15561557
<th rowspan="4"><strong>Task/Operator</strong></th>
1557-
<td>forst.fileCache.hit</td>
1558+
<td rowspan="4">forst.fileCache</td>
1559+
<td>hit</td>
15581560
<td>The hit count of ForSt state backend cache.</td>
15591561
<td>Counter</td>
15601562
</tr>
15611563
<tr>
1562-
<td>forst.fileCache.miss</td>
1564+
<td>miss</td>
15631565
<td>The miss count of ForSt state backend cache.</td>
15641566
<td>Counter</td>
15651567
</tr>
15661568
<tr>
1567-
<td>forst.fileCache.usedBytes</td>
1569+
<td>usedBytes</td>
15681570
<td>The bytes cached in ForSt state backend cache.</td>
15691571
<td>Gauge</td>
15701572
</tr>
15711573
<tr>
1572-
<td>forst.fileCache.remainingBytes</td>
1574+
<td>remainingBytes</td>
15731575
<td>The remaining space in the volume for the configured cache. Only available when 'state.backend.forst.cache.reserve-size' is set above 0. </td>
15741576
<td>Gauge</td>
15751577
</tr>
@@ -2280,6 +2282,44 @@ logged by `SystemResourcesMetricsInitializer` during the startup.
22802282
</tbody>
22812283
</table>
22822284

2285+
### Async State Processing
2286+
2287+
<table class="table table-bordered">
2288+
<thead>
2289+
<tr>
2290+
<th class="text-left" style="width: 15%">Scope</th>
2291+
<th class="text-left" style="width: 10%">Infix</th>
2292+
<th class="text-left" style="width: 20%">Metrics</th>
2293+
<th class="text-left" style="width: 50%">Description</th>
2294+
<th class="text-left" style="width: 5%">Type</th>
2295+
</tr>
2296+
</thead>
2297+
<tbody>
2298+
<tr>
2299+
<th rowspan="4"><strong>Operator</strong></th>
2300+
<td rowspan="4">asyncStateProcessing</td>
2301+
<td>numInFlightRecords</td>
2302+
<td>The number of in-flight records in the async execution controller's buffers.</td>
2303+
<td>Gauge</td>
2304+
</tr>
2305+
<tr>
2306+
<td>activeBufferSize</td>
2307+
<td>The number of records which are pending to be processed.</td>
2308+
<td>Gauge</td>
2309+
</tr>
2310+
<tr>
2311+
<td>blockingBufferSize</td>
2312+
<td>The number of records which are blocked by the ongoing records.</td>
2313+
<td>Gauge</td>
2314+
</tr>
2315+
<tr>
2316+
<td>numBlockingKeys</td>
2317+
<td>The number of different keys are blocked in async execution controller.</td>
2318+
<td>Gauge</td>
2319+
</tr>
2320+
</tbody>
2321+
</table>
2322+
22832323
## End-to-End latency tracking
22842324

22852325
Flink allows to track the latency of records travelling through the system. This feature is disabled by default.

‎docs/content/docs/deployment/cli.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -574,7 +574,7 @@ related options. Here's an overview of all the Python related options for the ac
574574
<td>
575575
Specify the path of the python interpreter used to execute the python UDF worker
576576
(e.g.: --pyExecutable /usr/local/bin/python3).
577-
The python UDF worker depends on Python 3.8+, Apache Beam (version == 2.43.0),
577+
The python UDF worker depends on Python 3.8+, Apache Beam (version >= 2.54.0,<= 2.61.0),
578578
Pip (version >= 20.3) and SetupTools (version >= 37.0.0).
579579
Please ensure that the specified environment meets the above requirements.
580580
</td>

‎docs/content/docs/dev/table/sqlClient.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -260,7 +260,7 @@ Mode "embedded" (default) submits Flink jobs from the local machine.
260260
/usr/local/bin/python3). The
261261
python UDF worker depends on
262262
Python 3.8+, Apache Beam
263-
(version == 2.43.0), Pip
263+
(version >= 2.54.0, <= 2.61.0), Pip
264264
(version >= 20.3) and SetupTools
265265
(version >= 37.0.0). Please
266266
ensure that the specified

‎docs/content/docs/ops/metrics.md

Lines changed: 48 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1535,31 +1535,33 @@ Certain RocksDB native metrics are available but disabled by default, you can fi
15351535
<table class="table table-bordered">
15361536
<thead>
15371537
<tr>
1538-
<th class="text-left" style="width: 18%">Scope</th>
1539-
<th class="text-left" style="width: 26%">Metrics</th>
1540-
<th class="text-left" style="width: 48%">Description</th>
1541-
<th class="text-left" style="width: 8%">Type</th>
1538+
<th class="text-left" style="width: 15%">Scope</th>
1539+
<th class="text-left" style="width: 15%">Infix</th>
1540+
<th class="text-left" style="width: 15%">Metrics</th>
1541+
<th class="text-left" style="width: 50%">Description</th>
1542+
<th class="text-left" style="width: 5%">Type</th>
15421543
</tr>
15431544
</thead>
15441545
<tbody>
15451546
<tr>
15461547
<th rowspan="4"><strong>Task/Operator</strong></th>
1547-
<td>forst.fileCache.hit</td>
1548+
<td rowspan="4">forst.fileCache</td>
1549+
<td>hit</td>
15481550
<td>The hit count of ForSt state backend cache.</td>
15491551
<td>Counter</td>
15501552
</tr>
15511553
<tr>
1552-
<td>forst.fileCache.miss</td>
1554+
<td>miss</td>
15531555
<td>The miss count of ForSt state backend cache.</td>
15541556
<td>Counter</td>
15551557
</tr>
15561558
<tr>
1557-
<td>forst.fileCache.usedBytes</td>
1559+
<td>usedBytes</td>
15581560
<td>The bytes cached in ForSt state backend cache.</td>
15591561
<td>Gauge</td>
15601562
</tr>
15611563
<tr>
1562-
<td>forst.fileCache.remainingBytes</td>
1564+
<td>remainingBytes</td>
15631565
<td>The remaining space in the volume for the configured cache. Only available when 'state.backend.forst.cache.reserve-size' is set above 0. </td>
15641566
<td>Gauge</td>
15651567
</tr>
@@ -2230,6 +2232,44 @@ Metrics below can be used to measure the effectiveness of speculative execution.
22302232
</tbody>
22312233
</table>
22322234

2235+
### Async State Processing
2236+
2237+
<table class="table table-bordered">
2238+
<thead>
2239+
<tr>
2240+
<th class="text-left" style="width: 15%">Scope</th>
2241+
<th class="text-left" style="width: 10%">Infix</th>
2242+
<th class="text-left" style="width: 20%">Metrics</th>
2243+
<th class="text-left" style="width: 50%">Description</th>
2244+
<th class="text-left" style="width: 5%">Type</th>
2245+
</tr>
2246+
</thead>
2247+
<tbody>
2248+
<tr>
2249+
<th rowspan="4"><strong>Operator</strong></th>
2250+
<td rowspan="4">asyncStateProcessing</td>
2251+
<td>numInFlightRecords</td>
2252+
<td>The number of in-flight records in the async execution controller's buffers.</td>
2253+
<td>Gauge</td>
2254+
</tr>
2255+
<tr>
2256+
<td>activeBufferSize</td>
2257+
<td>The number of records which are pending to be processed.</td>
2258+
<td>Gauge</td>
2259+
</tr>
2260+
<tr>
2261+
<td>blockingBufferSize</td>
2262+
<td>The number of records which are blocked by the ongoing records.</td>
2263+
<td>Gauge</td>
2264+
</tr>
2265+
<tr>
2266+
<td>numBlockingKeys</td>
2267+
<td>The number of different keys are blocked in async execution controller.</td>
2268+
<td>Gauge</td>
2269+
</tr>
2270+
</tbody>
2271+
</table>
2272+
22332273
## End-to-End latency tracking
22342274

22352275
Flink allows to track the latency of records travelling through the system. This feature is disabled by default.

‎docs/layouts/shortcodes/generated/python_configuration.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@
2424
<td><h5>python.executable</h5></td>
2525
<td style="word-wrap: break-word;">"python"</td>
2626
<td>String</td>
27-
<td>Specify the path of the python interpreter used to execute the python UDF worker. The python UDF worker depends on Python 3.8+, Apache Beam (version == 2.43.0), Pip (version &gt;= 20.3) and SetupTools (version &gt;= 37.0.0). Please ensure that the specified environment meets the above requirements. The option is equivalent to the command line option "-pyexec".</td>
27+
<td>Specify the path of the python interpreter used to execute the python UDF worker. The python UDF worker depends on Python 3.8+, Apache Beam (version >= 2.54.0, <= 2.61.0), Pip (version &gt;= 20.3) and SetupTools (version &gt;= 37.0.0). Please ensure that the specified environment meets the above requirements. The option is equivalent to the command line option "-pyexec".</td>
2828
</tr>
2929
<tr>
3030
<td><h5>python.execution-mode</h5></td>

‎docs/setup_hugo.sh

100644100755
Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,19 @@
1818
################################################################################
1919

2020
# setup hugo
21-
HUGO_REPO=https://github.com/gohugoio/hugo/releases/download/v0.110.0/hugo_extended_0.110.0_Linux-64bit.tar.gz
22-
HUGO_ARTIFACT=hugo_extended_0.110.0_Linux-64bit.tar.gz
21+
22+
# Detect Operating System
23+
OS="Linux"
24+
[[ "$OSTYPE" == "darwin"* ]] && OS="Mac"
25+
26+
# Setup Hugo based on OS
27+
if [ "$OS" = "Mac" ]; then
28+
HUGO_ARTIFACT="hugo_extended_0.110.0_darwin-universal.tar.gz"
29+
else
30+
HUGO_ARTIFACT="hugo_extended_0.110.0_Linux-64bit.tar.gz"
31+
fi
32+
33+
HUGO_REPO="https://github.com/gohugoio/hugo/releases/download/v0.110.0/${HUGO_ARTIFACT}"
2334
if ! curl --fail -OL $HUGO_REPO ; then
2435
echo "Failed to download Hugo binary"
2536
exit 1

0 commit comments

Comments
 (0)