Revamped Upgrade Agent for Litmus 3.x.x (#4740)

* initialise upgrade-agent

Signed-off-by: Kartikay <[email protected]>

* dockerfile revert

Signed-off-by: Kartikay <[email protected]>

* go.mod

Signed-off-by: Kartikay <[email protected]>

* litmus-portal to chaoscenter

Signed-off-by: Kartikay <[email protected]>

* dummy versions and log

Signed-off-by: Kartikay <[email protected]>

* CONTRIBUTING.md init

Signed-off-by: Kartikay <[email protected]>

* added checks while upgrading

Signed-off-by: Kartikay <[email protected]>

* transactions

Signed-off-by: Kartikay <[email protected]>

* transaction implemented in v3.4.0

Signed-off-by: Kartikay <[email protected]>

* added upgrades for version 3.9.0

Signed-off-by: Kartikay <[email protected]>

* nit: gofmt

Signed-off-by: Kartikay <[email protected]>

* Template for upgrades

Signed-off-by: Kartikay <[email protected]>

* refactoring and dumping dummy versions

Signed-off-by: Kartikay <[email protected]>

* gofmt

Signed-off-by: Kartikay <[email protected]>

* best practices

Signed-off-by: Kartikay <[email protected]>

* removed commented code

Signed-off-by: Kartikay <[email protected]>

* added flowchart

Signed-off-by: Kartikay <[email protected]>

* removed dummy version from map

Signed-off-by: Kartikay <[email protected]>

* refactor: better logs and comments

Signed-off-by: Kartikay <[email protected]>

---------

Signed-off-by: Kartikay <[email protected]>
Signed-off-by: Kartikay <[email protected]>
Signed-off-by: Kartikay <[email protected]>
Co-authored-by: Saranya Jena <[email protected]>
Co-authored-by: Kartikay <[email protected]>

chaoscenter/upgrade-agents/control-plane/CONTRIBUTING.MD

@@ -0,0 +1,73 @@
+## Steps to Contribute
+
+Fixes and improvements can be directly addressed by sending a Pull Request on GitHub. Pull requests will be reviewed by one or more maintainers and merged when acceptable.
+
+We ask that before contributing, please make the effort to coordinate with the maintainers of the project before submitting large or high impact PRs. This will prevent you from doing extra work that may or may not be merged.
+
+Use your judgement about what constitutes a large change. If you aren't sure, send a message to the **#litmus-dev** slack or submit an issue on GitHub.
+
+
+### **Sign your work with Developer Certificate of Origin**
+
+To contribute to this project, you must agree to the Developer Certificate of Origin (DCO) for each commit you make. The DCO is a simple statement that you, as a contributor, have the legal right to make the contribution.
+
+See the [DCO](https://developercertificate.org/) file for the full text of what you must agree to.
+
+To successfully sign off your contribution you just add a line to every git commit message:
+
+```git
+Signed-off-by: Joe Smith <[email protected]>
+```
+
+Use your real name (sorry, no pseudonyms or anonymous contributions.)
+
+If you set your `user.name` and `user.email` git configs, you can sign your commit automatically with `git commit -s`. You can also use git [aliases](https://git-scm.com/book/tr/v2/Git-Basics-Git-Aliases) like `git config --global alias.ci 'commit -s'`. Now you can commit with git ci and the commit will be signed.
+
+## **Development Guide**
+
+Start MongoDB, Auth and GraphQL server as mentioned in the [ChaosCenter Development Guide](https://github.com/litmuschaos/litmus/wiki/ChaosCenter-Development-Guide) to set up the basic structure of the DB. Then start the upgrade manager located in `chaoscenter/upgrade-agents/control-plane` by setting up the environment variables
+
+```
+export DB_SERVER="mongodb://m1:27015,m2:27016,m3:27017/?replicaSet=rs0"
+export DB_USER=admin
+export DB_PASSWORD=1234
+export VERSION=<Version you need to upgrade to>
+```
+To run the upgrade-manager, run
+
+```
+go run main.go
+```
+The upgrade-manager would get the current version of Litmus through the DB.
+
+## **Version Upgrade Files**
+
+`<version>/` folder contains files for the upgrade logic
+
+- `upgrade-manager.go` - Contains a map of versions with their corresponding version-managers for the versions.
+- `vx.y.z/manager.go` - Instantiates the Version Manager and runs the upgradeExecutor in transactions which can be omitted if the operation doesn't support a transaction.
+- `vx.y.z/upgrade.go` - Contains the logic of upgradeExecutor of what schema changes are to be made in the specific version.
+
+There are some other files not mentioned here.
+
+## **Example: Upgrade to version 3.9.0**
+
+In version 3.9.0 the following changes are done in DB schema
+
+- In projects collection (auth DB), member role is updated from `Editor` to `Executor`
+- New `is_initial_login` field in users collection (auth DB) whose value is set as false
+
+Run the upgrade-agent while specifying the version `VERSION=3.9.0` in the environmental variable, and it should now be upgraded.
+
+## **Best Practices**
+
+1) If upgrade volume is huge, then transaction is not favorable.
+2) Split the upgrades into proprietary functions depending on their purpose and database/collection names.
+3) Use logging techniques as used in other versions to provide the user an insight of how the upgrades are being done.
+4) While using a transaction, make sure the session context is properly passed into the relevant mongo operations and related functions.
+5) Script should be written in a way that even if run twice, should not affect the already updated documents.
+
+
+Below is the basic technical flow for the upgrade-agent
+
+![image info](./Upgrade-Agent-Flow.png)

chaoscenter/upgrade-agents/control-plane/go.mod

@@ -1,29 +1,40 @@
-module github.com/litmuschaos/litmus/litmus-portal/upgrader-agents/control-plane
+module github.com/litmuschaos/litmus/chaoscenter/upgrader-agents/control-plane
 
 go 1.22
 
 require (
 	github.com/kelseyhightower/envconfig v1.4.0
-	go.mongodb.org/mongo-driver v1.7.1
-	go.uber.org/zap v1.18.1
+	github.com/sirupsen/logrus v1.4.2
+	go.mongodb.org/mongo-driver v1.11.9
+)
+
+require (
+	github.com/jessevdk/go-flags v1.5.0 // indirect
+	github.com/montanaflynn/stats v0.7.1 // indirect
+	golang.org/x/exp v0.0.0-20240529005216-23cca8864a10 // indirect
+	golang.org/x/net v0.25.0 // indirect
+	golang.org/x/term v0.20.0 // indirect
+	gopkg.in/tomb.v2 v2.0.0-20161208151619-d5d1b5820637 // indirect
+	gopkg.in/yaml.v2 v2.4.0 // indirect
 )
 
 require (
 	github.com/go-stack/stack v1.8.0 // indirect
-	github.com/golang/snappy v0.0.1 // indirect
-	github.com/google/go-cmp v0.5.5 // indirect
-	github.com/klauspost/compress v1.9.5 // indirect
+	github.com/golang/snappy v0.0.4 // indirect
+	github.com/google/go-cmp v0.6.0 // indirect
+	github.com/klauspost/compress v1.17.8 // indirect
+	github.com/konsorten/go-windows-terminal-sequences v1.0.2 // indirect
+	github.com/mongodb/mongo-tools v0.0.0-20240711192303-088725fbaf4b
 	github.com/pkg/errors v0.9.1 // indirect
+	github.com/stretchr/testify v1.9.0 // indirect
 	github.com/xdg-go/pbkdf2 v1.0.0 // indirect
-	github.com/xdg-go/scram v1.0.2 // indirect
-	github.com/xdg-go/stringprep v1.0.2 // indirect
-	github.com/youmark/pkcs8 v0.0.0-20181117223130-1be2e3e5546d // indirect
-	go.uber.org/atomic v1.9.0 // indirect
-	go.uber.org/multierr v1.7.0 // indirect
-	golang.org/x/crypto v0.17.0 // indirect
-	golang.org/x/lint v0.0.0-20200302205851-738671d3881b // indirect
-	golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e // indirect
-	golang.org/x/text v0.14.0 // indirect
+	github.com/xdg-go/scram v1.1.2 // indirect
+	github.com/xdg-go/stringprep v1.0.4 // indirect
+	github.com/youmark/pkcs8 v0.0.0-20201027041543-1326539a0a0a // indirect
+	golang.org/x/crypto v0.23.0 // indirect
+	golang.org/x/sync v0.7.0 // indirect
+	golang.org/x/sys v0.20.0 // indirect
+	golang.org/x/text v0.15.0 // indirect
 	golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
-	gopkg.in/yaml.v2 v2.4.0 // indirect
+	gopkg.in/yaml.v3 v3.0.1 // indirect
 )