Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark - delete clickhouse intro #1855

Merged
merged 19 commits into from
Jan 21, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
74 changes: 41 additions & 33 deletions check_move.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,41 +45,49 @@ def process_md_file(file_path):
with open(file_path, "r", encoding="utf-8") as f:
content = f.read()

links = link_pattern.findall(content)

for link in links:
if not urlparse(link).scheme and not os.path.isabs(link):
full_path: str = os.path.normpath(
os.path.join(os.path.dirname(file_path), link)
)
if not full_path.endswith(".md") and not full_path.endswith(".mdx"):
full_path += ".md"

for [from_path, to_path] in move_pairs:
# Skip change of suffix
from_base, from_ext = os.path.splitext(from_path)
to_base, to_ext = os.path.splitext(to_path)
if (
from_ext in [".md", ".mdx", ""] or to_ext in [".md", ".mdx", ""]
) and (from_base == to_base):
continue
# In md, the link relative path starts from the directory where the document is located, not the document
relative_to_path = os.path.relpath(
to_path, os.path.dirname(file_path)
links = link_pattern.findall(content)

new_content = content
for link in links:
if not urlparse(link).scheme and not os.path.isabs(link):
full_path: str = os.path.normpath(
os.path.join(os.path.dirname(file_path), link)
)
if not full_path.endswith(".md") and not full_path.endswith(".mdx"):
full_path += ".md"

for [from_path, to_path] in move_pairs:
# Skip change of suffix
from_base, from_ext = os.path.splitext(from_path)
to_base, to_ext = os.path.splitext(to_path)
if (
from_ext in [".md", ".mdx", ""] or to_ext in [".md", ".mdx", ""]
) and (from_base == to_base):
continue
# In md, the link relative path starts from the directory where the document is located, not the document
relative_to_path = os.path.relpath(to_path, os.path.dirname(file_path))
relative_to_path = remove_suffix(relative_to_path, ".md")
relative_to_path = remove_suffix(relative_to_path, ".mdx")

if is_same_file(full_path, from_path):
print(
f"{file_path} has a link moved by this commit: from {link} to {relative_to_path}"
)
change_detected = True
# Replace the old link with the new one
new_content = new_content.replace(
f"({link})", f"({relative_to_path})"
)
relative_to_path = remove_suffix(relative_to_path, ".md")
relative_to_path = remove_suffix(relative_to_path, ".mdx")

if is_same_file(full_path, from_path):
print(
f"{file_path} has a link moved by this commit: from {link} to {relative_to_path}"
)
change_detected = True
for deleted_path in deletes:
if is_same_file(full_path, deleted_path):
print(f"{file_path} has a link removed by this commit: {link}")
change_detected = True

for deleted_path in deletes:
if is_same_file(full_path, deleted_path):
print(f"{file_path} has a link removed by this commit: {link}")
change_detected = True
# Write the updated content back to the file
if new_content != content:
with open(file_path, "w", encoding="utf-8") as f:
f.write(new_content)


def extract_file_changes(git_show_output: List[AnyStr]):
Expand Down Expand Up @@ -130,4 +138,4 @@ def travel(root_path: str):

if change_detected:
print("Failed!")
sys.exit(1)
sys.exit(1)
2 changes: 1 addition & 1 deletion docs/admin-manual/auth/authentication/federation.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ Note: To log in using other clients, refer to the section "How Clients Use Clear

### LDAP Information Cache

To avoid frequent access to the LDAP service, Doris caches LDAP information in memory. You can configure the `ldap_user_cache_timeout_s` parameter in the `ldap.conf` file to specify the cache time for LDAP users, which defaults to 12 hours. After modifying the information in the LDAP service or modifying the corresponding role permissions in Doris, the changes may not take effect immediately due to caching. You can use the `refresh ldap` statement to refresh the cache. For details, see [REFRESH-LDAP](../../../sql-manual/sql-statements/Utility-Statements/REFRESH-LDAP.md).
To avoid frequent access to the LDAP service, Doris caches LDAP information in memory. You can configure the `ldap_user_cache_timeout_s` parameter in the `ldap.conf` file to specify the cache time for LDAP users, which defaults to 12 hours. After modifying the information in the LDAP service or modifying the corresponding role permissions in Doris, the changes may not take effect immediately due to caching. You can use the `refresh ldap` statement to refresh the cache. For details, see [REFRESH-LDAP](../../../sql-manual/sql-statements/account-management/REFRESH-LDAP).

### Limitations of LDAP Verification

Expand Down
12 changes: 6 additions & 6 deletions docs/admin-manual/auth/authentication/internal.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,13 +71,13 @@ Doris supports the following password policies to help users manage passwords be
Doris itself does not support a blacklist, only a whitelist function, but we can simulate a blacklist in some ways. Suppose a user named `user@'192.%'` is created, allowing users from 192.* to log in. If you want to prohibit users from 192.168.10.1 from logging in, you can create another user `cmy@'192.168.10.1'` and set a new password. Since 192.168.10.1 has a higher priority than 192.%, users from 192.168.10.1 will no longer be able to log in using the old password.

## Related Commands
- Create User: [CREATE USER](../../../sql-manual/sql-statements/Account-Management-Statements/CREATE-USER.md)
- View User: [SHOW ALL GRANTS](../../../sql-manual/sql-statements/Show-Statements/SHOW-GRANTS.md)
- Modify User: [ALTER USER](../../../sql-manual/sql-statements/Account-Management-Statements/ALTER-USER.md)
- Change Password: [SET PASSWORD](../../../sql-manual/sql-statements/Account-Management-Statements/SET-PASSWORD.md)
- Delete User: [DROP USER](../../../sql-manual/sql-statements/Account-Management-Statements/DROP-USER.md)
- Create User: [CREATE USER](../../../sql-manual/sql-statements/account-management/CREATE-USER)
- View User: [SHOW ALL GRANTS](../../../sql-manual/sql-statements/account-management/SHOW-GRANTS)
- Modify User: [ALTER USER](../../../sql-manual/sql-statements/account-management/ALTER-USER)
- Change Password: [SET PASSWORD](../../../sql-manual/sql-statements/account-management/SET-PASSWORD)
- Delete User: [DROP USER](../../../sql-manual/sql-statements/account-management/DROP-USER)
- Set User Attributes: [SET PROPERTY](../../../sql-manual/sql-statements/Account-Management-Statements/SET-PROPERTY.md)
- View User Attributes: [SHOW PROPERTY](../../../sql-manual/sql-statements/Show-Statements/SHOW-PROPERTY.md)
- View User Attributes: [SHOW PROPERTY](../../../sql-manual/sql-statements/account-management/SHOW-PROPERTY)

## Other Explanations
1. User Identity Priority Selection Issue During Login
Expand Down
4 changes: 2 additions & 2 deletions docs/admin-manual/auth/authorization/data.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,8 @@ With column permissions in Doris, you can achieve fine-grained access control ov
Currently, column permissions only support Select_priv.

### Related Commands
- Grant: [GRANT](../../../sql-manual/sql-statements/Account-Management-Statements/GRANT.md)
- Revoke: [REVOKE](../../../sql-manual/sql-statements/Account-Management-Statements/REVOKE.md)
- Grant: [GRANT](../../../sql-manual/sql-statements/account-management/GRANT-TO)
- Revoke: [REVOKE](../../../sql-manual/sql-statements/account-management/REVOKE-FROM.md)

### Column Permission Example

Expand Down
18 changes: 9 additions & 9 deletions docs/admin-manual/auth/authorization/internal.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,15 +108,15 @@ userN has priv1 permission through role3 and priv2 and privN permissions through

## Related Commands

- Grant/assign role: [GRANT](../../../sql-manual/sql-statements/Account-Management-Statements/GRANT.md)
- Revoke/revoke role: [REVOKE](../../../sql-manual/sql-statements/Account-Management-Statements/REVOKE.md)
- Create role: [CREATE ROLE](../../../sql-manual/sql-statements/Account-Management-Statements/CREATE-ROLE.md)
- Delete role: [DROP ROLE](../../../sql-manual/sql-statements/Account-Management-Statements/DROP-ROLE.md)
- Modify role: [ALTER ROLE](../../../sql-manual/sql-statements/Account-Management-Statements/ALTER-ROLE.md)
- View current user permissions and roles: [SHOW GRANTS](../../../sql-manual/sql-statements/Show-Statements/SHOW-GRANTS.md)
- View all user permissions and roles: [SHOW ALL GRANTS](../../../sql-manual/sql-statements/Show-Statements/SHOW-GRANTS.md)
- View created roles: [SHOW ROLES](../../../sql-manual/sql-statements/Show-Statements/SHOW-ROLES.md)
- View supported permissions: [SHOW PRIVILEGES](../../../sql-manual/sql-statements/Show-Statements/SHOW-PRIVILEGES.md)
- Grant/assign role: [GRANT](../../../sql-manual/sql-statements/account-management/GRANT-TO)
- Revoke/revoke role: [REVOKE](../../../sql-manual/sql-statements/account-management/REVOKE-FROM.md)
- Create role: [CREATE ROLE](../../../sql-manual/sql-statements/account-management/CREATE-ROLE)
- Delete role: [DROP ROLE](../../../sql-manual/sql-statements/account-management/DROP-ROLE)
- Modify role: [ALTER ROLE](../../../sql-manual/sql-statements/account-management/ALTER-ROLE)
- View current user permissions and roles: [SHOW GRANTS](../../../sql-manual/sql-statements/account-management/SHOW-GRANTS)
- View all user permissions and roles: [SHOW ALL GRANTS](../../../sql-manual/sql-statements/account-management/SHOW-GRANTS)
- View created roles: [SHOW ROLES](../../../sql-manual/sql-statements/account-management/SHOW-ROLES)
- View supported permissions: [SHOW PRIVILEGES](../../../sql-manual/sql-statements/account-management/SHOW-PRIVILEGES)

## Best Practices

Expand Down
2 changes: 1 addition & 1 deletion docs/admin-manual/config/be-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ There are two ways to view the configuration items of BE:

2. View by command

You can view the configuration items of the BE in the MySQL client with the following command,Concrete language law reference [SHOW-CONFIG](../../sql-manual/sql-statements/Database-Administration-Statements/SHOW-CONFIG.md):
You can view the configuration items of the BE in the MySQL client with the following command,Concrete language law reference [SHOW-CONFIG](../../sql-manual/sql-statements/cluster-management/instance-management/SHOW-FRONTEND-CONFIG):

`SHOW BACKEND CONFIG;`

Expand Down
2 changes: 1 addition & 1 deletion docs/admin-manual/config/fe-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ There are two ways to view the configuration items of FE:

2. View by command

After the FE is started, you can view the configuration items of the FE in the MySQL client with the following command,Concrete language law reference [SHOW-CONFIG](../../sql-manual/sql-statements/Database-Administration-Statements/SHOW-CONFIG.md):
After the FE is started, you can view the configuration items of the FE in the MySQL client with the following command,Concrete language law reference [SHOW-CONFIG](../../sql-manual/sql-statements/cluster-management/instance-management/SHOW-FRONTEND-CONFIG):

`SHOW FRONTEND CONFIG;`

Expand Down
6 changes: 3 additions & 3 deletions docs/admin-manual/data-admin/backup-restore/backup.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ For concepts related to backup, please refer to [Backup and Restore](./overview.
suites/backup_restore/test_create_and_drop_repository.groovy
-->

Use the appropriate statement to create a Repository based on your storage choice. For detailed usage, please refer to [Create Repository](../../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md). When backing up using the same path for the Repository across different clusters, ensure to use different labels to avoid conflicts that may cause data confusion.
Use the appropriate statement to create a Repository based on your storage choice. For detailed usage, please refer to [Create Repository](../../../sql-manual/sql-statements/data-modification/backup-and-restore/CREATE-REPOSITORY). When backing up using the same path for the Repository across different clusters, ensure to use different labels to avoid conflicts that may cause data confusion.

### Option 1: Create Repository on S3

Expand Down Expand Up @@ -162,7 +162,7 @@ PROPERTIES

## Step 2. Backup

Refer to the following statements to back up databases, tables, or partitions. For detailed usage, please refer to [Backup](../../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md).
Refer to the following statements to back up databases, tables, or partitions. For detailed usage, please refer to [Backup](../../../sql-manual/sql-statements/data-modification/backup-and-restore/BACKUP).

It is recommended to use meaningful label names, such as those containing the databases and tables included in the backup.

Expand Down Expand Up @@ -262,4 +262,4 @@ mysql> SHOW SNAPSHOT ON example_repo;

## Step 5. Cancel Backup (if necessary)

You can use `CANCEL BACKUP FROM db_name;` to cancel a backup task in a database. For more specific usage, refer to [Cancel Backup](../../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CANCEL-BACKUP.md).
You can use `CANCEL BACKUP FROM db_name;` to cancel a backup task in a database. For more specific usage, refer to [Cancel Backup](../../../sql-manual/sql-statements/data-modification/backup-and-restore/CANCEL-BACKUP).
4 changes: 2 additions & 2 deletions docs/admin-manual/data-admin/recyclebin.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ You can query the Recycle Bin with the following command:
SHOW CATALOG RECYCLE BIN [WHERE NAME [= "name" | LIKE "name_matcher"]];
```

For more detailed syntax and best practices, please refer to the [SHOW-CATALOG-RECYCLE-BIN](../../sql-manual/sql-statements/Show-Statements/SHOW-CATALOG-RECYCLE-BIN.md) command manual, You can also type `help SHOW CATALOG RECYCLE BIN` on the MySql client command line for more help.
For more detailed syntax and best practices, please refer to the [SHOW-CATALOG-RECYCLE-BIN](../../sql-manual/sql-statements/recycle/SHOW-CATALOG-RECYCLE-BIN) command manual, You can also type `help SHOW CATALOG RECYCLE BIN` on the MySql client command line for more help.


### Start Data Recovery
Expand All @@ -63,4 +63,4 @@ RECOVER TABLE example_db.example_tbl;
RECOVER PARTITION p1 FROM example_tbl;
```

For more detailed syntax and best practices used by RECOVER, please refer to the [RECOVER](../../sql-manual/sql-statements/Database-Administration-Statements/RECOVER.md) command manual, You can also type `HELP RECOVER` on the MySql client command line for more help.
For more detailed syntax and best practices used by RECOVER, please refer to the [RECOVER](../../sql-manual/sql-statements/recycle/RECOVER) command manual, You can also type `HELP RECOVER` on the MySql client command line for more help.
8 changes: 4 additions & 4 deletions docs/admin-manual/small-file-mgr.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ File management has three main commands: `CREATE FILE`, `SHOW FILE` and `DROP FI

### CREATE FILE

This statement is used to create and upload a file to the Doris cluster. For details, see [CREATE FILE](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-FILE.md).
This statement is used to create and upload a file to the Doris cluster. For details, see [CREATE FILE](../sql-manual/sql-statements/security/CREATE-FILE).

Examples:

Expand Down Expand Up @@ -75,7 +75,7 @@ Examples:

### SHOW FILE

This statement can view the files that have been created successfully. For details, see [SHOW FILE](../sql-manual/sql-statements/Data-Definition-Statements/Drop/DROP-FILE.md).
This statement can view the files that have been created successfully. For details, see [SHOW FILE](../sql-manual/sql-statements/security/DROP-FILE).

Examples:

Expand All @@ -87,7 +87,7 @@ Examples:

### DROP FILE

This statement can view and delete an already created file. For specific operations, see [DROP FILE](../sql-manual/sql-statements/Data-Definition-Statements/Drop/DROP-FILE.md).
This statement can view and delete an already created file. For specific operations, see [DROP FILE](../sql-manual/sql-statements/security/DROP-FILE).

Examples:

Expand Down Expand Up @@ -129,4 +129,4 @@ Because the file meta-information and content are stored in FE memory. So by def

## More Help

For more detailed syntax and best practices used by the file manager, see [CREATE FILE](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-FILE.md), [DROP FILE](../sql-manual/sql-statements/Data-Definition-Statements/Drop/DROP-FILE.md) and [SHOW FILE](../sql-manual/sql-statements/Show-Statements/SHOW-FILE.md) command manual, you can also enter `HELP CREATE FILE`, `HELP DROP FILE` and `HELP SHOW FILE` in the MySql client command line to get more help information.
For more detailed syntax and best practices used by the file manager, see [CREATE FILE](../sql-manual/sql-statements/security/CREATE-FILE), [DROP FILE](../sql-manual/sql-statements/security/DROP-FILE) and [SHOW FILE](../sql-manual/sql-statements/security/SHOW-FILE) command manual, you can also enter `HELP CREATE FILE`, `HELP DROP FILE` and `HELP SHOW FILE` in the MySql client command line to get more help information.
10 changes: 5 additions & 5 deletions docs/admin-manual/workload-management/workload-group.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ mysql [information_schema]>create workload group if not exists g1
Query OK, 0 rows affected (0.03 sec)

```
You can refer to [CREATE-WORKLOAD-GROUP](../../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-WORKLOAD-GROUP)。
You can refer to [CREATE-WORKLOAD-GROUP](../../sql-manual/sql-statements/cluster-management/compute-management/CREATE-WORKLOAD-GROUP)。

The CPU limit configured at this point is a soft limit. Since version 2.1, Doris will automatically create a group named normal, which cannot be deleted.

Expand Down Expand Up @@ -213,7 +213,7 @@ If the g1 Workload Group is not visible, you can use the ADMIN account to execut
"GRANT USAGE_PRIV ON WORKLOAD GROUP 'g1' TO 'user_1'@'%';"
```
This statement means granting the user_1 the permission to use the Workload Group named g1.
More details can be found in [grant](../../sql-manual/sql-statements/Account-Management-Statements/GRANT)。
More details can be found in [grant](../../sql-manual/sql-statements/account-management/GRANT-TO)。

**Two ways to bind Workload Group to user**
1. By setting the user property, you can bind the user to a default Workload Group. The default is normal. It's important to note that the value here cannot be left empty; otherwise, the statement will fail.
Expand All @@ -234,7 +234,7 @@ When both methods are used to specify a Workload Group for the user, the session
```
show workload groups;
```
More details can be found in [SHOW-WORKLOAD-GROUPS](../../sql-manual/sql-statements/Show-Statements/SHOW-WORKLOAD-GROUPS)
More details can be found in [SHOW-WORKLOAD-GROUPS](../../sql-manual/sql-statements/cluster-management/compute-management/SHOW-WORKLOAD-GROUPS)

2. You can view the Workload Group through the system table:
```
Expand Down Expand Up @@ -262,15 +262,15 @@ mysql [information_schema]>select cpu_share from information_schema.workload_gro

```

More details can be found in [ALTER-WORKLOAD-GROUP](../../sql-manual/sql-statements/Data-Definition-Statements/Alter/ALTER-WORKLOAD-GROUP)
More details can be found in [ALTER-WORKLOAD-GROUP](../../sql-manual/sql-statements/cluster-management/compute-management/ALTER-WORKLOAD-GROUP)

## Drop Workload Group
```
mysql [information_schema]>drop workload group g1;
Query OK, 0 rows affected (0.01 sec)
```

More details can be found in[DROP-WORKLOAD-GROUP](../../sql-manual/sql-statements/Data-Definition-Statements/Drop/DROP-WORKLOAD-GROUP)
More details can be found in[DROP-WORKLOAD-GROUP](../../sql-manual/sql-statements/cluster-management/compute-management/DROP-WORKLOAD-GROUP)

## Explanation of Switching Between CPU Soft and Hard Limit Modes
Currently, Doris does not support running both CPU soft and hard limits simultaneously. At any given time, a Doris cluster can only operate in either CPU soft limit or CPU hard limit mode.
Expand Down
Loading
Loading