Skip to content

Optimize: Modify cls log module #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 40 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,32 +36,34 @@ Detailed usage examples can be found in [CLS examples](./examples/)。
1. Import repository

In the project's WORKSPACE file, import the cpp-cls-logging repository and its dependencies:
```
load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")

git_repository(
name = "trpc_cpp",
remote = "https://github.com/trpc-group/trpc-cpp.git",
branch = "main",
)

load("@trpc_cpp//trpc:workspace.bzl", "trpc_workspace")
trpc_workspace()

git_repository(
name = "cpp-cls-logging",
remote = "https://github.com/trpc-ecosystem/cpp-logging-cls.git",
branch = "main",
)

load("@cpp-cls-logging//trpc:workspace.bzl", "logging_cls_workspace")
logging_cls_workspace()
```

```bzl
load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")

git_repository(
name = "trpc_cpp",
remote = "https://github.com/trpc-group/trpc-cpp.git",
branch = "main",
)

load("@trpc_cpp//trpc:workspace.bzl", "trpc_workspace")
trpc_workspace()

git_repository(
name = "cpp-cls-logging",
remote = "https://github.com/trpc-ecosystem/cpp-logging-cls.git",
branch = "main",
)

load("@cpp-cls-logging//trpc:workspace.bzl", "logging_cls_workspace")
logging_cls_workspace()
```

2. Import plugin

In the target where CLS is needed, import the "@cpp-cls-logging//trpc/logging/cls:cls_log_api" dependency. For example:
```

```bzl
cc_binary(
name = "helloworld_server",
srcs = ["helloworld_server.cc"],
Expand All @@ -79,6 +81,7 @@ Not supported yet.
## Plugin Configuration

To use the Etcd plugin, you must add the plugin configuration to the framework configuration file:

```yaml
plugins:
log:
Expand Down Expand Up @@ -110,20 +113,20 @@ The following is a detailed explanation of these configuration items:

The CLS plugin provides a plugin registration interface ::trpc::cls::Init(), and users need to call this interface before starting the framework.

For server scenarios, users need to call the TrpcApp::RegisterPlugins function in the service startup:
1. For server scenarios, users need to call the TrpcApp::RegisterPlugins function in the service startup:

```cpp
#include "trpc/logging/cls/cls_log_api.h"
```cpp
#include "trpc/logging/cls/cls_log_api.h"

class HelloworldServer : public ::trpc::TrpcApp {
public:
...
int RegisterPlugins() override {
::trpc::logging::cls::Init();
return 0;
}
};
```
class HelloworldServer : public ::trpc::TrpcApp {
public:
...
int RegisterPlugins() override {
::trpc::logging::cls::Init();
return 0;
}
};
```

2. For pure client scenarios, you need to call it after starting the framework configuration initialization and before starting other framework modules:

Expand All @@ -149,6 +152,7 @@ For server scenarios, users need to call the TrpcApp::RegisterPlugins function i
* Before using the CLS plugin, you need to ensure that the CLS service is correctly activated and configured. In addition, you also need to know how to create log topics and configure log delivery rules on the Tencent Cloud console. The following is a brief introduction to these two aspects:

### Activate and configure CLS service

1. Log in to the Tencent Cloud Console [TencentCloud](https://cloud.tencent.com/login?s_url=https%3A%2F%2Fconsole.cloud.tencent.com%2F) and find the Cloud Log Service[CLS](https://cloud.tencent.com/product/cls) in the product list.

2. In the CLS console, create a new log set.
Expand All @@ -158,13 +162,15 @@ For server scenarios, users need to call the TrpcApp::RegisterPlugins function i
4. In the newly created log topic, configure log delivery rules. You can set the delivery target (e.g., COS, CDN, etc.), delivery frequency, and delivery file format according to your needs.

### Get the Topic ID of the log topic

1. Log in to the Tencent Cloud Console [TencentCloud](https://cloud.tencent.com/login?s_url=https%3A%2F%2Fconsole.cloud.tencent.com%2F) and find the Cloud Log Service[CLS](https://cloud.tencent.com/product/cls) in the product list.

2. Enter the page of the log topic you just created.

3. On the basic information page of the log topic, find the Topic ID of the log topic. Add this Topic ID to the plugin configuration to send log data to that topic.

### Special Tips

1. To ensure that log data can be correctly sent to the CLS platform, please make sure your Tencent Cloud account has sufficient permissions. When using the CLS plugin, you need to configure the SecretId and SecretKey in the yaml file. These two parameters can be created and obtained in the API key management page of the Tencent Cloud Console[TencentCloud](https://cloud.tencent.com/login?s_url=https%3A%2F%2Fconsole.cloud.tencent.com%2F).

2. Please note that sending log data to the CLS platform may incur certain costs. For specific cost standards, please refer to the billing instructions in the Tencent Cloud official document [TencentCloud](https://buy.cloud.tencent.com/pricing).
Expand Down
38 changes: 22 additions & 16 deletions README.zh_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,9 @@ ClsSink 类是一个自定义的 spdlog 日志接收器,负责将日志发送

1. 引入仓库

在项目的WORKSPACE文件中,引入cpp-cls-logging仓库及其依赖:
```
在项目的WORKSPACE文件中,引入 `cpp-cls-logging` 仓库及其依赖:

```bzl
load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")

git_repository(
Expand All @@ -60,8 +61,9 @@ ClsSink 类是一个自定义的 spdlog 日志接收器,负责将日志发送

2. 引入插件

在需要用到CLS的目标中引入“@cpp-cls-logging//trpc/logging/cls:cls_log_api”依赖。例如:
```
在需要用到 CLS 的目标中引入“@cpp-cls-logging//trpc/logging/cls:cls_log_api”依赖。例如:

```bzl
cc_binary(
name = "helloworld_server",
srcs = ["helloworld_server.cc"],
Expand All @@ -79,6 +81,7 @@ ClsSink 类是一个自定义的 spdlog 日志接收器,负责将日志发送
## 插件配置

要使用Etcd插件,必须在框架配置文件中加上插件配置:

```yaml
plugins:
log:
Expand Down Expand Up @@ -110,20 +113,20 @@ plugins:

CLS插件提供了插件注册的接口 ::trpc::cls::Init(),用户需要在框架启动前调用该接口进行初始化。

对于服务端场景,用户需要在服务启动的 TrpcApp::RegisterPlugins 函数中调用:
1. 对于服务端场景,用户需要在服务启动的 TrpcApp::RegisterPlugins 函数中调用:

```cpp
#include "trpc/logging/cls/cls_log_api.h"
```cpp
#include "trpc/logging/cls/cls_log_api.h"

class HelloworldServer : public ::trpc::TrpcApp {
public:
...
int RegisterPlugins() override {
::trpc::cls::Init();
return 0;
}
};
```
class HelloworldServer : public ::trpc::TrpcApp {
public:
...
int RegisterPlugins() override {
::trpc::cls::Init();
return 0;
}
};
```

2. 对于纯客户端场景,需要在启动框架配置初始化后,框架其他模块启动前调用:

Expand All @@ -149,6 +152,7 @@ CLS插件提供了插件注册的接口 ::trpc::cls::Init(),用户需要在框
* 在使用CLS插件之前,您需要确保已正确开通并配置了CLS服务。此外,您还需要了解如何在腾讯云控制台上创建日志主题和配置日志投递规则。以下是有关这两个方面的简要说明:

### 开通并配置CLS服务

1. 登录腾讯云控制台 [TencentCloud](https://cloud.tencent.com/login?s_url=https%3A%2F%2Fconsole.cloud.tencent.com%2F) 在产品列表中找到云日志服务[CLS](https://cloud.tencent.com/product/cls)。

2. 在CLS控制台中,创建一个新的日志集。
Expand All @@ -158,13 +162,15 @@ CLS插件提供了插件注册的接口 ::trpc::cls::Init(),用户需要在框
4. 在新创建的日志主题中,配置日志投递规则。您可以根据需要设置投递的目标(例如COS、CDN等)、投递频率、投递文件格式等参数。

### 获取日志主题的Topic ID

1. 登录腾讯云控制台 [TencentCloud](https://cloud.tencent.com/login?s_url=https%3A%2F%2Fconsole.cloud.tencent.com%2F) 在产品列表中找到云日志服务[CLS](https://cloud.tencent.com/product/cls)。

2. 进入刚刚创建的日志主题页面。

3. 在日志主题的基本信息页面,找到该日志主题的Topic ID。将此Topic ID添加到插件配置中,以便将日志数据发送到该主题。

### 特别提示

1. 为了确保日志数据能够正确地发送到CLS平台,请确保您的腾讯云账户具有足够的权限。在使用CLS插件时,您需要在配置SecretId和SecretKey在yaml文件中,这两个参数可以在腾讯云控制台[TencentCloud](https://cloud.tencent.com/login?s_url=https%3A%2F%2Fconsole.cloud.tencent.com%2F)的API密钥管理页面中创建和获取。

2. 请注意,发送到CLS平台的日志数据可能会产生一定的费用。具体的费用标准,请参考腾讯云官方文档[TencentCloud](https://buy.cloud.tencent.com/pricing)中的计费说明。
Expand Down
4 changes: 2 additions & 2 deletions examples/client/trpc_cpp_fiber.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ plugins:
log:
default:
- name: default
min_level: 2
min_level: 2 # 0-trace, 1-debug, 2-info, 3-warn, 4-error, 5-critical
format: "[%Y-%m-%d %H:%M:%S.%e] [thread %t] [%l] [%@] %v"
mode: 2
mode: 2 # 1-Synchronous, 2-Asynchronous, 3-Extreme-Speed
sinks:
stdout: # sink of the local console
eol: true # Whether each output is automatically wrapped. Default line feed
Expand Down
4 changes: 2 additions & 2 deletions examples/server/trpc_cpp_fiber.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@ plugins:
log:
default:
- name: default
min_level: 2
min_level: 2 # 0-trace, 1-debug, 2-info, 3-warn, 4-error, 5-critical
format: "[%Y-%m-%d %H:%M:%S.%e] [thread %t] [%l] [%@] %v"
mode: 2
mode: 2 # 1-Synchronous, 2-Asynchronous, 3-Extreme-Speed
sinks:
stdout: # sink of the local console
eol: true # Whether each output is automatically wrapped. Default line feed
Expand Down
14 changes: 7 additions & 7 deletions trpc/logging/cls/cls_log.cc
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,15 @@ namespace trpc::cls {

std::unordered_map<std::string, uint32_t> logger_plungin_id_map;

void SetLoggerPlunginId(const std::string& logger_name, uint32_t plugin_id) {
void SetLoggerPluginId(const std::string& logger_name, uint32_t plugin_id) {
logger_plungin_id_map[logger_name] = plugin_id;
}

uint32_t GetPlunginIdFromLogger(const std::string& logger_name) { return logger_plungin_id_map[logger_name]; }
uint32_t GetPluginIdFromLogger(const std::string& logger_name) { return logger_plungin_id_map[logger_name]; }

std::unordered_map<std::string, uint32_t>& GetLoggerPlunginIdMap() { return logger_plungin_id_map; }
std::unordered_map<std::string, uint32_t>& GetLoggerPluginIdMap() { return logger_plungin_id_map; }

void ClsLog::initSDK() {
void ClsLog::InitSDK() {
// Init cls-sdk client.
sdk_config_.set_maxsendworkercount(1);
sdk_config_.set_endpoint(endpoint_);
Expand All @@ -49,7 +49,7 @@ void ClsLog::initSDK() {
cls_sink_->Start();
}

bool ClsLog::initSpdLogger() {
bool ClsLog::InitSpdLogger() {
auto formatter =
std::make_unique<spdlog::pattern_formatter>(spdlog::pattern_time_type::local, spdlog::details::os::default_eol);
cls_sink_->set_formatter(std::move(formatter));
Expand Down Expand Up @@ -81,13 +81,13 @@ int ClsLog::Init() noexcept {
cls_sink_ = std::make_shared<ClsSinkMt>();

// Initialize spdlog logger configuration
if (!initSpdLogger()) {
if (!InitSpdLogger()) {
TRPC_FMT_ERROR("Init spdlog logger fail!");
return -1;
}

// Initialize the cls-sdk configuration
initSDK();
InitSDK();

return 0;
}
Expand Down
26 changes: 13 additions & 13 deletions trpc/logging/cls/cls_log.h
Original file line number Diff line number Diff line change
Expand Up @@ -36,16 +36,16 @@ namespace trpc::cls {
/// @brief Set the plugin ID for a given logger name.
/// @param logger_name The name of the logger.
/// @param plugin_id The ID of the plugin associated with the logger.
void SetLoggerPlunginId(const std::string& logger_name, uint32_t plugin_id);
void SetLoggerPluginId(const std::string& logger_name, uint32_t plugin_id);

/// @brief Get the plugin ID associated with a given logger name.
/// @param logger_name The name of the logger.
/// @return The plugin ID associated with the logger name.
uint32_t GetPlunginIdFromLogger(const std::string& logger_name);
uint32_t GetPluginIdFromLogger(const std::string& logger_name);

/// @brief Get the map of logger names to plugin IDs.
/// @return A reference to the unordered_map containing logger names as keys and plugin IDs as values.
std::unordered_map<std::string, uint32_t>& GetLoggerPlunginIdMap();
std::unordered_map<std::string, uint32_t>& GetLoggerPluginIdMap();

/// @brief Adds key-value pairs to the context's filter data for each logger plugin.
/// @tparam T The type of the context object.
Expand All @@ -63,14 +63,14 @@ std::unordered_map<std::string, uint32_t>& GetLoggerPlunginIdMap();
/// @endcode
template <typename T, typename... Args>
void WithFields(T& context, Args&&... key_value_pairs) {
auto plugin_id_map = GetLoggerPlunginIdMap();
auto plugin_id_map = GetLoggerPluginIdMap();
for (const auto& [logger_name, plugin_id] : plugin_id_map) {
auto* data_map = context->template GetFilterData<std::unordered_map<std::string, std::string>>(
GetPlunginIdFromLogger(logger_name.c_str()));
GetPluginIdFromLogger(logger_name.c_str()));
if (!data_map) {
std::unordered_map<std::string, std::string> new_data_map;
(new_data_map.emplace(std::forward<Args>(key_value_pairs)), ...);
context->SetFilterData(GetPlunginIdFromLogger(logger_name.c_str()), std::move(new_data_map));
context->SetFilterData(GetPluginIdFromLogger(logger_name.c_str()), std::move(new_data_map));
} else {
(data_map->emplace(std::forward<Args>(key_value_pairs)), ...);
}
Expand All @@ -79,11 +79,11 @@ void WithFields(T& context, Args&&... key_value_pairs) {

class ClsLog : public Logging {
public:
std::string Name() const { return "cls-log"; }
std::string Name() const override { return "cls-log"; }

std::string LoggerName() const { return logger_name_; }
const std::string& LoggerName() const override { return logger_name_; }

std::string Version() const { return "1.0.0"; }
std::string Version() const { return "1.0.0"; }

explicit ClsLog(const ClsLogSinkConfig& sink_config)
: logger_name_(sink_config.logger_name),
Expand All @@ -92,7 +92,7 @@ class ClsLog : public Logging {
endpoint_(sink_config.endpoint),
secret_id_(sink_config.secret_id),
secret_key_(sink_config.secret_key) {
SetLoggerPlunginId(logger_name_, this->GetPluginID());
SetLoggerPluginId(logger_name_, this->GetPluginID());
Init();
}

Expand Down Expand Up @@ -154,10 +154,10 @@ class ClsLog : public Logging {

private:
// Initialize the CLS SDK
void initSDK();
void InitSDK();

// Initialize the spdlog logger instance
bool initSpdLogger();
bool InitSpdLogger();

private:
// logger configuration
Expand All @@ -181,4 +181,4 @@ class ClsLog : public Logging {

using ClsLogPtr = RefPtr<ClsLog>;

} // namespace trpc::cls
} // namespace trpc::cls
2 changes: 1 addition & 1 deletion trpc/logging/cls/cls_log_api.cc
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ bool Init() {
return -1;
}
for (const auto& config : config_list) {
TrpcPlugin::GetInstance()->RegisterLogging(MakeRefCounted<ClsLog>(std::move(config)));
TrpcPlugin::GetInstance()->RegisterLogging(MakeRefCounted<ClsLog>(config));
}

return true;
Expand Down
18 changes: 18 additions & 0 deletions trpc/logging/cls/cls_log_test.cc
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,24 @@ class ClsLogTest : public ::testing::Test {
return;
}

ASSERT_TRUE(config_list.size() == 1);

auto& cls_conf = config_list[0];
cls_conf.Display();
ASSERT_STREQ(cls_conf.logger_name.c_str(), "default");
ASSERT_TRUE(cls_conf.mode == 2);
ASSERT_TRUE(cls_conf.logs_per_request == 100);
ASSERT_TRUE(cls_conf.request_timeout == 50);
ASSERT_TRUE(cls_conf.connect_timeout == 5);

ASSERT_STREQ(cls_conf.topic_id.c_str(), "0***********e");
ASSERT_STREQ(cls_conf.cls_region.c_str(), "a***********u");
ASSERT_STREQ(cls_conf.cls_domain.c_str(), "tencentyun");
ASSERT_STREQ(cls_conf.secret_id.c_str(), "A***********t");
ASSERT_STREQ(cls_conf.secret_key.c_str(), "J***********x");
ASSERT_STREQ(cls_conf.key_message.c_str(), "msg");
ASSERT_STREQ(cls_conf.endpoint.c_str(), "a***********u.cls.tencentyun.com");

cls_default_ = MakeRefCounted<ClsLog>(config_list[0]);
default_log_->RegisterRawSink(cls_default_);
}
Expand Down
Loading