Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: remove deprecated rotate file #2056

Merged
merged 1 commit into from
Dec 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
163 changes: 0 additions & 163 deletions docs/audit-log.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,166 +13,3 @@ auditLog:
console:
enabled: false
```

### Logging to a Rotating File

#### Enabling Rotating File Logging

To enable audit logging to a rotating file, set the following in your configuration (this feature is disabled by default):

```yaml
auditLog:
rotateFile:
enabled: true
```

With this configuration, the default behavior is:

- Rotate logs at midnight (local system timezone)
- Log file format: redhat-developer-hub-audit-%DATE%.log
- Log files stored in /var/log/redhat-developer-hub/audit
- No automatic log file deletion
- No gzip compression of archived logs
- No file size limit

#### Customizing Log File Location and Name

To change the directory where log files are stored, specify a custom path (an absolute path is recommended): By default, the audit logs are written in the `/var/log/redhat-developer-hub/audit` directory.

```yaml
auditLog:
rotateFile:
logFileDirPath: /custom-path
```

---

**NOTE**

The specified directory will be created automatically if it does not exist.

---

By default, the audit log files will be in the following format: `redhat-developer-hub-audit-%DATE%.log` where `%DATE%` is the format specified in [`auditLog.rotateFile.dateFormat`](#configuring-file-retention-policy).

To customize the log file name format, use:

```yaml
auditLog:
rotateFile:
logFileName: custom-audit-log-%DATE%.log
```

#### Configuring File Rotation Frequency

The default file rotation occurs daily at 00:00 local time. You can adjust the rotation frequency with the following configurations:

```yaml
auditLog:
rotateFile:
frequency: '12h' # Default: `custom`
dateFormat: 'YYYY-MM-DD' # Default: `YYYY-MM-DD`
utc: false # Default: `false`
maxSize: 100m # Default: undefined
```

`frequency` options include:

- `daily`: Rotate daily at 00:00 local time
- `Xm`: Rotate every X minutes (where X is a number between 0 and 59)
- `Xh`: Rotate every X hours (where X is a number between 0 and 23)
- `test`: Rotate every 1 minute
- `custom`: Use `dateFormat` to set the rotation frequency (default if frequency is not specified)

---

**NOTE**
If `frequency` is set to `Xh`, `Xm` or `test`, the `dateFormat` setting must be configured in a format that includes the specified time component. Otherwise, the rotation will not work as expected.

For example, `dateFormat: 'YYYY-MM-DD-HH'` for hourly rotation. `dateFormat: 'YYYY-MM-DD-HH-mm'` for minute rotation.

---

Examples:

```yaml
auditLog:
rotateFile:
# If you want to rotate the file every 17 minutes
dateFormat: 'YYYY-MM-DD-HH-mm'
frequency: '17m'
```

The `dateFormat` setting configures both the %DATE% in logFileName and the file rotation frequency if frequency is set to `custom`. The default format is `YYYY-MM-DD`, meaning daily rotation. Supported values are based on [Moment.js formats](https://momentjs.com/docs/#/displaying/format/).

If `frequency` is set to `custom`, then rotations will take place when the date string, represented in the specified `dateFormat`, changes.

Examples:

```yaml
auditLog:
rotateFile:
# If you want rotations to occur every week for some reason and at the start of each month. Example `%DATE$` = '2025-Jul-Week 30'
dateFormat: 'YYYY-MMM-[Week] ww'
```

```yaml
auditLog:
rotateFile:
# If you want to rotate the file at noon and midnight
dateFormat: 'YYYY-MM-DD-A'
```

To use UTC time for `dateFormat` instead of local time:

```yaml
auditLog:
rotateFile:
utc: true # Default: False
```

To set a maximum log file size before rotation (which would add a count suffix to the filename upon reaching the size limit): Ex: `redhat-developer-hub-audit-2024-07-22.log.3`.

To configure `maxSize`, provide a number followed by one of `k`, `m`, or `g` to specify the file size in kilobytes, megabytes, or gigabytes. No `maxSize` is configured by default.

```yaml
auditLog:
rotateFile:
maxSize: 100m # Sets a max file size limit of 100MB for audit log
```

#### Configuring File Retention Policy

By default, log files are not deleted or archived. You can configure the maximum number of files to keep:

```yaml
auditLog:
rotateFile:
maxFilesOrDays: 14 # Deletes the oldest log when there are more than 14 log files
```

Or, configure the maximum number of days to retain logs by appending:

```yaml
auditLog:
rotateFile:
maxFilesOrDays: 5d # Deletes logs older than 5 days
```

---

**NOTE**

If log deletion is enabled, a `.<sha256 hash>-audit.json` will be generated in the directory where the logs are to track generated logs. Any log file not contained in it will not be subject to automatic deletion.

Currently, a new `.<sha256 hash>-audit.json` file is generated every time the backend is started. This means old audit logs will no longer be tracked/deleted with the exception of any log files reused by the current backend.

---

To archive and compress rotated logs using gzip:

```yaml
auditLog:
rotateFile:
zippedArchive: true # Default: false
```
3 changes: 1 addition & 2 deletions packages/backend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,7 @@
"app": "*",
"global-agent": "3.0.0",
"undici": "6.19.8",
"winston": "3.14.2",
"winston-daily-rotate-file": "5.0.0"
"winston": "3.14.2"
},
"devDependencies": {
"@backstage/cli": "0.28.2",
Expand Down
6 changes: 1 addition & 5 deletions packages/backend/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,7 @@ backend.add(
logger: config => {
const auditLogConfig = config?.getOptionalConfig('auditLog');
return {
transports: [
...transports.log,
...transports.auditLog(auditLogConfig),
...transports.auditLogFile(auditLogConfig),
],
transports: [...transports.log, ...transports.auditLog(auditLogConfig)],
};
},
}),
Expand Down
24 changes: 0 additions & 24 deletions packages/backend/src/logger/customLogger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@ import type { Config } from '@backstage/config';

import * as winston from 'winston';

import 'winston-daily-rotate-file';

const defaultFormat = winston.format.combine(
winston.format.timestamp({
format: 'YYYY-MM-DD HH:mm:ss',
Expand Down Expand Up @@ -51,28 +49,6 @@ export const transports = {
}),
];
},
auditLogFile: (config?: Config) => {
if (!config?.getOptionalBoolean('rotateFile.enabled')) {
return [];
}
return [
new winston.transports.DailyRotateFile({
format: auditLogWinstonFormat,
dirname:
config?.getOptionalString('rotateFile.logFileDirPath') ??
'/var/log/redhat-developer-hub/audit',
filename:
config?.getOptionalString('rotateFile.logFileName') ??
'redhat-developer-hub-audit-%DATE%.log',
datePattern: config?.getOptionalString('rotateFile.dateFormat'),
frequency: config?.getOptionalString('rotateFile.frequency'),
zippedArchive: config?.getOptionalBoolean('rotateFile.zippedArchive'),
utc: config?.getOptionalBoolean('rotateFile.utc'),
maxSize: config?.getOptionalString('rotateFile.maxSize'),
maxFiles: config?.getOptional('rotateFile.maxFilesOrDays'),
}),
];
},
};

export const createStaticLogger = ({ service }: { service: string }) => {
Expand Down
24 changes: 0 additions & 24 deletions yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -22404,7 +22404,6 @@ __metadata:
prettier: 3.4.1
undici: 6.19.8
winston: 3.14.2
winston-daily-rotate-file: 5.0.0
languageName: unknown
linkType: soft

Expand Down Expand Up @@ -28614,15 +28613,6 @@ __metadata:
languageName: node
linkType: hard

"file-stream-rotator@npm:^0.6.1":
version: 0.6.1
resolution: "file-stream-rotator@npm:0.6.1"
dependencies:
moment: ^2.29.1
checksum: ebdf6a9e7ca886a50f4dafb2284d4569cefd5bdf4e4451ead25f4d68b7f9776b2620a3d110d534edd40935d1e17f37d818e2129303201870ff89c71b19b49ac1
languageName: node
linkType: hard

"file-type@npm:^16.5.4":
version: 16.5.4
resolution: "file-type@npm:16.5.4"
Expand Down Expand Up @@ -45636,20 +45626,6 @@ __metadata:
languageName: node
linkType: hard

"winston-daily-rotate-file@npm:5.0.0":
version: 5.0.0
resolution: "winston-daily-rotate-file@npm:5.0.0"
dependencies:
file-stream-rotator: ^0.6.1
object-hash: ^3.0.0
triple-beam: ^1.4.1
winston-transport: ^4.7.0
peerDependencies:
winston: ^3
checksum: 45d0a1c1d1a178a22a6f92f4248139e0889720947d5afa657314826d6ea48e7dceae37521e2de2ed3a121993c4ae4ddcb0b510613c489a68d2eed689a304bef5
languageName: node
linkType: hard

"winston-transport@npm:^4.5.0, winston-transport@npm:^4.7.0":
version: 4.7.0
resolution: "winston-transport@npm:4.7.0"
Expand Down
Loading