Skip to content

Commit 6fcd6a6

Browse files
vasil-cfvasil-cf
authored
fix: 2.7 docs mongo (#65)
1 parent 4b60493 commit 6fcd6a6

15 files changed

+1532
-1569
lines changed

‎codefresh/README.md

Lines changed: 108 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -33,9 +33,9 @@ Helm chart for deploying [Codefresh On-Premises](https://codefresh.io/docs/docs/
3333
- [Projects pipelines limit](#projects-pipelines-limit)
3434
- [Enable session cookie](#enable-session-cookie)
3535
- [X-Frame-Options response header](#x-frame-options-response-header)
36-
- [Auto-index creation in MongoDB](#auto-index-creation-in-mongodb)
3736
- [Image digests in containers](#image-digests-in-containers)
3837
- [Configuring OIDC Provider](#configuring-oidc-provider)
38+
- [Maintaining MongoDB Indexes](#maintaining-mongodb-indexes)
3939
- [Upgrading](#upgrading)
4040
- [To 2.0.0](#to-200)
4141
- [To 2.0.12](#to-2012)
@@ -91,7 +91,7 @@ imageCredentials:
9191
password: '{ "type": "service_account", "project_id": "codefresh-enterprise", "private_key_id": ... }'
9292
```
9393
94-
- Specify `.Values.global.appUrl`, `.Values.global.firebaseUrl` and `.Values.global.firebaseSecret`
94+
- Specify `.Values.global.appUrl`, `.Values.global.firebaseUrl`, `.Values.global.firebaseSecret`, `.Values.global.env.MONGOOSE_AUTO_INDEX`, `.Values.global.env.MONGO_AUTOMATIC_INDEX_CREATION`
9595

9696
```yaml
9797
global:
@@ -115,6 +115,14 @@ global:
115115
# firebaseSecretSecretKeyRef:
116116
# name: my-secret
117117
# key: firebase-secret
118+
119+
# -- Enable index creation in MongoDB
120+
# This is required for first-time installations!
121+
# Before usage in Production, you must set it to `false` or remove it!
122+
env:
123+
MONGOOSE_AUTO_INDEX: "true"
124+
MONGO_AUTOMATIC_INDEX_CREATION: "true"
125+
118126
```
119127

120128
- Specify `.Values.ingress.tls.cert` and `.Values.ingress.tls.key` OR `.Values.ingress.tls.existingSecret`
@@ -170,6 +178,16 @@ ingress-nginx:
170178
--timeout 15m
171179
```
172180

181+
### ⚠️ **MANDATORY** Post-Installation Action Items
182+
183+
Once your Codefresh On-Prem instance is installed, configured, and confirmed to be ready for production use, the following variables must be set to `false` or removed:
184+
185+
```yaml
186+
global:
187+
env:
188+
MONGOOSE_AUTO_INDEX: "false"
189+
MONGO_AUTOMATIC_INDEX_CREATION: "false"
190+
173191
## Chart Configuration
174192
175193
See [Customizing the Chart Before Installing](https://helm.sh/docs/intro/using_helm/#customizing-the-chart-before-installing). To see all configurable options with detailed comments, visit the chart's [values.yaml](./values.yaml), or run these configuration commands:
@@ -1204,32 +1222,6 @@ cfapi:
12041222
USE_SHA256_GITHUB_SIGNATURE: "true"
12051223
```
12061224
1207-
### Auto-index creation in MongoDB
1208-
1209-
In Codefresh On-Prem 2.6.x, the `cfapi` can create indexes in MongoDB automatically. This feature is disabled by default. To enable it, set the following environment variable:
1210-
1211-
> **Note!** Enabling this feature can cause performance degradation during the index creation process.
1212-
1213-
> **Note!** It is recommended to add indexes during a maintenance window. The indexes list is provided in `codefresh/files/indexes/<MAJOR.MINOR>/<collection_name>.json` files.
1214-
1215-
```yaml
1216-
cfapi:
1217-
container:
1218-
env:
1219-
MONGOOSE_AUTO_INDEX: "true"
1220-
```
1221-
1222-
```yaml
1223-
argo-platform:
1224-
api-graphql:
1225-
env:
1226-
MONGO_AUTOMATIC_INDEX_CREATION: "true"
1227-
```
1228-
1229-
Ref:
1230-
- [Create an Index in Atlas DB](https://www.mongodb.com/docs/atlas/atlas-ui/indexes/#create-an-index)
1231-
- [Create an Index with mongosh](https://www.mongodb.com/docs/manual/reference/method/db.collection.createIndex/)
1232-
12331225
### Image digests in containers
12341226
12351227
In Codefresh On-Prem 2.6.x all Codefresh owner microservices include image digests in the default subchart values.
@@ -1456,6 +1448,86 @@ To see all the claims supported by Codefresh OIDC provider, see `claims_supporte
14561448

14571449
Use [obtain-oidc-id-token](https://github.com/codefresh-io/steps/blob/822afc0a9a128384e76459c6573628020a2cf404/incubating/obtain-oidc-id-token/step.yaml#L27-L58) and [aws-sts-assume-role-with-web-identity](https://github.com/codefresh-io/steps/blob/822afc0a9a128384e76459c6573628020a2cf404/incubating/aws-sts-assume-role-with-web-identity/step.yaml#L29-L63) steps to exchange the OIDC token (JWT) for a cloud access token.
14581450

1451+
## Maintaining MongoDB Indexes
1452+
1453+
Sometimes, in new releases of Codefresh On-Prem, index requirements change. When this happens, it's mentioned in the [Upgrading section](#upgrading) for the specific release.
1454+
1455+
> ℹ️ If you're upgrading from version `X` to version `Y`, and index requirements were updated in any of the intermediate versions, you only need to align your indexes with the index requirements of version `Y`. To do that, follow [Index alignment](#index-alignment) instructions.
1456+
1457+
### Index alignment
1458+
1459+
The required index definitions for each release can be found at the following resources:
1460+
1461+
- `2.6` <https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.6/indexes>
1462+
- `2.7` <https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.7/indexes>
1463+
1464+
The indexes are stored in JSON files with keys and options specified.
1465+
1466+
The directory structure is:
1467+
1468+
```console
1469+
indexes
1470+
├── <DB_NAME> # MongoDB database name
1471+
│ ├── <COLLECTION_NAME>.json # MongoDB indexes for the specified collection
1472+
```
1473+
1474+
**Overview of the index alignment process:**
1475+
1476+
1. Identify the differences between the indexes in your MongoDB instance and the required index definitions.
1477+
2. Create any missing indexes one by one. (It's important not to create them in bulk.)
1478+
3. Perform the upgrade of Codefresh On-Prem installation.
1479+
4. Then remove any unnecessary indexes.
1480+
1481+
> ⚠️ **Note! Any changes to indexes should be performed during a defined maintenance window or during periods of lowest traffic to MongoDB.**
1482+
>
1483+
> Building indexes during time periods where the target collection is under heavy write load can result in reduced write performance and longer index builds. ([*Source: MongoDB official documentation*](https://www.mongodb.com/docs/manual/core/index-creation/#index-build-impact-on-database-performance))
1484+
>
1485+
> Even minor changes to indexes (e.g., index removal) can cause brief but noticeable performance degradation ([*Source: MongoDB official documentation*](https://www.mongodb.com/docs/manual/core/query-plans/#plan-cache-flushes))
1486+
1487+
#### Self-hosted MongoDB
1488+
1489+
For self-hosted MongoDB, follow the instructions below:
1490+
1491+
- Connect to the MongoDB server using the [mongosh](https://www.mongodb.com/docs/mongodb-shell/install/) shell. Open your terminal or command prompt and run the following command, replacing `<connection_string>` with the appropriate MongoDB connection string for your server:
1492+
1493+
```shell
1494+
mongosh "<connection_string>"
1495+
```
1496+
1497+
- Retrieve the list of indexes for a specific collection:
1498+
1499+
```js
1500+
db.getSiblingDB('<db_name>').getCollection('<collection_name>').getIndexes()
1501+
```
1502+
1503+
- Compare your indexes with the required indexes for the target release, and adjust them by creating any missing indexes or removing any unnecessary ones
1504+
1505+
**Index creation**
1506+
1507+
> ⚠ **Note! Always create indexes sequentially, one by one. Don't create them in bulk.**
1508+
1509+
- To create an index, use the `createIndex()` method:
1510+
1511+
```js
1512+
db.getSiblingDB('<db_name>').getCollection('<collection_name>').createIndex(<keys_object>, <options_object>)
1513+
```
1514+
1515+
After executing the `createIndex()` command, you should see a result indicating that the index was created successfully.
1516+
1517+
**Index removal**
1518+
1519+
- To remove an index, use the `dropIndex()` method with `<index_name>`:
1520+
1521+
```js
1522+
db.getSiblingDB('<db_name>').getCollection('<collection_name>').dropIndex('<index_name>')
1523+
```
1524+
1525+
#### Atlas Database
1526+
1527+
If you're hosting MongoDB on [Atlas](https://www.mongodb.com/atlas/database), use the following [Manage Indexes](https://www.mongodb.com/docs/atlas/atlas-ui/indexes/) guide to View, Create or Remove indexes.
1528+
1529+
> ⚠️ **Important!** In Atlas, for production environments, it is recommended to use rolling index builds by enabling the "Build index via rolling process" checkbox. ([*MongoDB official documentation*](https://www.mongodb.com/docs/v6.0/tutorial/build-indexes-on-replica-sets/))
1530+
14591531
## Upgrading
14601532

14611533
### To 2.0.0
@@ -1991,18 +2063,22 @@ cfapi:
19912063

19922064
### To 2.6.0
19932065

2066+
> ⚠️ **WARNING! MongoDB indexes changed!**
2067+
>
2068+
> Please, follow [Maintaining MongoDB indexes](#maintaining-mongodb-indexes) guide to meet index requirements **BEFORE** the upgrade process.
2069+
19942070
### [What's new in 2.6.x](https://codefresh.io/docs/docs/whats-new/on-prem-release-notes/#on-premises-version-26)
19952071

19962072
#### Affected values
19972073

19982074
[Image digests in containers](#image-digests-in-containers)
19992075

2000-
#### Auto-index creation in MongoDB
2001-
2002-
[Auto-index creation in MongoDB](#auto-index-creation-in-mongodb)
2003-
20042076
### To 2.7.0
20052077

2078+
> ⚠️ **WARNING! MongoDB indexes changed!**
2079+
>
2080+
> Please, follow [Maintaining MongoDB indexes](#maintaining-mongodb-indexes) guide to meet index requirements **BEFORE** the upgrade process.
2081+
20062082
### [What's new in 2.7.x](https://codefresh.io/docs/docs/whats-new/on-prem-release-notes/#on-premises-version-27)
20072083

20082084
#### Affected values

‎codefresh/README.md.gotmpl

Lines changed: 108 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -33,9 +33,9 @@ Helm chart for deploying [Codefresh On-Premises](https://codefresh.io/docs/docs/
3333
- [Projects pipelines limit](#projects-pipelines-limit)
3434
- [Enable session cookie](#enable-session-cookie)
3535
- [X-Frame-Options response header](#x-frame-options-response-header)
36-
- [Auto-index creation in MongoDB](#auto-index-creation-in-mongodb)
3736
- [Image digests in containers](#image-digests-in-containers)
3837
- [Configuring OIDC Provider](#configuring-oidc-provider)
38+
- [Maintaining MongoDB Indexes](#maintaining-mongodb-indexes)
3939
- [Upgrading](#upgrading)
4040
- [To 2.0.0](#to-200)
4141
- [To 2.0.12](#to-2012)
@@ -92,7 +92,7 @@ imageCredentials:
9292
password: '{ "type": "service_account", "project_id": "codefresh-enterprise", "private_key_id": ... }'
9393
```
9494

95-
- Specify `.Values.global.appUrl`, `.Values.global.firebaseUrl` and `.Values.global.firebaseSecret`
95+
- Specify `.Values.global.appUrl`, `.Values.global.firebaseUrl`, `.Values.global.firebaseSecret`, `.Values.global.env.MONGOOSE_AUTO_INDEX`, `.Values.global.env.MONGO_AUTOMATIC_INDEX_CREATION`
9696

9797
```yaml
9898
global:
@@ -116,6 +116,14 @@ global:
116116
# firebaseSecretSecretKeyRef:
117117
# name: my-secret
118118
# key: firebase-secret
119+
120+
# -- Enable index creation in MongoDB
121+
# This is required for first-time installations!
122+
# Before usage in Production, you must set it to `false` or remove it!
123+
env:
124+
MONGOOSE_AUTO_INDEX: "true"
125+
MONGO_AUTOMATIC_INDEX_CREATION: "true"
126+
119127
```
120128

121129
- Specify `.Values.ingress.tls.cert` and `.Values.ingress.tls.key` OR `.Values.ingress.tls.existingSecret`
@@ -171,6 +179,16 @@ ingress-nginx:
171179
--timeout 15m
172180
```
173181

182+
### ⚠️ **MANDATORY** Post-Installation Action Items
183+
184+
Once your Codefresh On-Prem instance is installed, configured, and confirmed to be ready for production use, the following variables must be set to `false` or removed:
185+
186+
```yaml
187+
global:
188+
env:
189+
MONGOOSE_AUTO_INDEX: "false"
190+
MONGO_AUTOMATIC_INDEX_CREATION: "false"
191+
174192
## Chart Configuration
175193

176194
See [Customizing the Chart Before Installing](https://helm.sh/docs/intro/using_helm/#customizing-the-chart-before-installing). To see all configurable options with detailed comments, visit the chart's [values.yaml](./values.yaml), or run these configuration commands:
@@ -1209,32 +1227,6 @@ cfapi:
12091227
USE_SHA256_GITHUB_SIGNATURE: "true"
12101228
```
12111229

1212-
### Auto-index creation in MongoDB
1213-
1214-
In Codefresh On-Prem 2.6.x, the `cfapi` can create indexes in MongoDB automatically. This feature is disabled by default. To enable it, set the following environment variable:
1215-
1216-
> **Note!** Enabling this feature can cause performance degradation during the index creation process.
1217-
1218-
> **Note!** It is recommended to add indexes during a maintenance window. The indexes list is provided in `codefresh/files/indexes/<MAJOR.MINOR>/<collection_name>.json` files.
1219-
1220-
```yaml
1221-
cfapi:
1222-
container:
1223-
env:
1224-
MONGOOSE_AUTO_INDEX: "true"
1225-
```
1226-
1227-
```yaml
1228-
argo-platform:
1229-
api-graphql:
1230-
env:
1231-
MONGO_AUTOMATIC_INDEX_CREATION: "true"
1232-
```
1233-
1234-
Ref:
1235-
- [Create an Index in Atlas DB](https://www.mongodb.com/docs/atlas/atlas-ui/indexes/#create-an-index)
1236-
- [Create an Index with mongosh](https://www.mongodb.com/docs/manual/reference/method/db.collection.createIndex/)
1237-
12381230
### Image digests in containers
12391231

12401232
In Codefresh On-Prem 2.6.x all Codefresh owner microservices include image digests in the default subchart values.
@@ -1463,6 +1455,86 @@ To see all the claims supported by Codefresh OIDC provider, see `claims_supporte
14631455

14641456
Use [obtain-oidc-id-token](https://github.com/codefresh-io/steps/blob/822afc0a9a128384e76459c6573628020a2cf404/incubating/obtain-oidc-id-token/step.yaml#L27-L58) and [aws-sts-assume-role-with-web-identity](https://github.com/codefresh-io/steps/blob/822afc0a9a128384e76459c6573628020a2cf404/incubating/aws-sts-assume-role-with-web-identity/step.yaml#L29-L63) steps to exchange the OIDC token (JWT) for a cloud access token.
14651457

1458+
## Maintaining MongoDB Indexes
1459+
1460+
Sometimes, in new releases of Codefresh On-Prem, index requirements change. When this happens, it's mentioned in the [Upgrading section](#upgrading) for the specific release.
1461+
1462+
> ℹ️ If you're upgrading from version `X` to version `Y`, and index requirements were updated in any of the intermediate versions, you only need to align your indexes with the index requirements of version `Y`. To do that, follow [Index alignment](#index-alignment) instructions.
1463+
1464+
### Index alignment
1465+
1466+
The required index definitions for each release can be found at the following resources:
1467+
1468+
- `2.6` <https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.6/indexes>
1469+
- `2.7` <https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.7/indexes>
1470+
1471+
The indexes are stored in JSON files with keys and options specified.
1472+
1473+
The directory structure is:
1474+
1475+
```console
1476+
indexes
1477+
├── <DB_NAME> # MongoDB database name
1478+
│ ├── <COLLECTION_NAME>.json # MongoDB indexes for the specified collection
1479+
```
1480+
1481+
**Overview of the index alignment process:**
1482+
1483+
1. Identify the differences between the indexes in your MongoDB instance and the required index definitions.
1484+
2. Create any missing indexes one by one. (It's important not to create them in bulk.)
1485+
3. Perform the upgrade of Codefresh On-Prem installation.
1486+
4. Then remove any unnecessary indexes.
1487+
1488+
> ⚠️ **Note! Any changes to indexes should be performed during a defined maintenance window or during periods of lowest traffic to MongoDB.**
1489+
>
1490+
> Building indexes during time periods where the target collection is under heavy write load can result in reduced write performance and longer index builds. ([*Source: MongoDB official documentation*](https://www.mongodb.com/docs/manual/core/index-creation/#index-build-impact-on-database-performance))
1491+
>
1492+
> Even minor changes to indexes (e.g., index removal) can cause brief but noticeable performance degradation ([*Source: MongoDB official documentation*](https://www.mongodb.com/docs/manual/core/query-plans/#plan-cache-flushes))
1493+
1494+
#### Self-hosted MongoDB
1495+
1496+
For self-hosted MongoDB, follow the instructions below:
1497+
1498+
- Connect to the MongoDB server using the [mongosh](https://www.mongodb.com/docs/mongodb-shell/install/) shell. Open your terminal or command prompt and run the following command, replacing `<connection_string>` with the appropriate MongoDB connection string for your server:
1499+
1500+
1501+
```shell
1502+
mongosh "<connection_string>"
1503+
```
1504+
1505+
- Retrieve the list of indexes for a specific collection:
1506+
1507+
```js
1508+
db.getSiblingDB('<db_name>').getCollection('<collection_name>').getIndexes()
1509+
```
1510+
1511+
- Compare your indexes with the required indexes for the target release, and adjust them by creating any missing indexes or removing any unnecessary ones
1512+
1513+
**Index creation**
1514+
1515+
> ⚠ **Note! Always create indexes sequentially, one by one. Don't create them in bulk.**
1516+
1517+
- To create an index, use the `createIndex()` method:
1518+
1519+
```js
1520+
db.getSiblingDB('<db_name>').getCollection('<collection_name>').createIndex(<keys_object>, <options_object>)
1521+
```
1522+
1523+
After executing the `createIndex()` command, you should see a result indicating that the index was created successfully.
1524+
1525+
**Index removal**
1526+
1527+
- To remove an index, use the `dropIndex()` method with `<index_name>`:
1528+
1529+
```js
1530+
db.getSiblingDB('<db_name>').getCollection('<collection_name>').dropIndex('<index_name>')
1531+
```
1532+
1533+
#### Atlas Database
1534+
1535+
If you're hosting MongoDB on [Atlas](https://www.mongodb.com/atlas/database), use the following [Manage Indexes](https://www.mongodb.com/docs/atlas/atlas-ui/indexes/) guide to View, Create or Remove indexes.
1536+
1537+
> ⚠️ **Important!** In Atlas, for production environments, it is recommended to use rolling index builds by enabling the "Build index via rolling process" checkbox. ([*MongoDB official documentation*](https://www.mongodb.com/docs/v6.0/tutorial/build-indexes-on-replica-sets/))
14661538

14671539
## Upgrading
14681540

@@ -2000,18 +2072,22 @@ cfapi:
20002072

20012073
### To 2.6.0
20022074

2075+
> ⚠️ **WARNING! MongoDB indexes changed!**
2076+
>
2077+
> Please, follow [Maintaining MongoDB indexes](#maintaining-mongodb-indexes) guide to meet index requirements **BEFORE** the upgrade process.
2078+
20032079
### [What's new in 2.6.x](https://codefresh.io/docs/docs/whats-new/on-prem-release-notes/#on-premises-version-26)
20042080

20052081
#### Affected values
20062082

20072083
[Image digests in containers](#image-digests-in-containers)
20082084

2009-
#### Auto-index creation in MongoDB
2010-
2011-
[Auto-index creation in MongoDB](#auto-index-creation-in-mongodb)
2012-
20132085
### To 2.7.0
20142086

2087+
> ⚠️ **WARNING! MongoDB indexes changed!**
2088+
>
2089+
> Please, follow [Maintaining MongoDB indexes](#maintaining-mongodb-indexes) guide to meet index requirements **BEFORE** the upgrade process.
2090+
20152091
### [What's new in 2.7.x](https://codefresh.io/docs/docs/whats-new/on-prem-release-notes/#on-premises-version-27)
20162092

20172093
#### Affected values

0 commit comments

Comments
 (0)