From 2c5f81d48101d86dfc6cf61c83f0049d56b1d74a Mon Sep 17 00:00:00 2001 From: Tristan Barrow Date: Thu, 18 Dec 2025 15:08:12 -0700 Subject: [PATCH 1/4] use data generation tools --- practices/use-data-generation-tools.md | 89 ++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) create mode 100644 practices/use-data-generation-tools.md diff --git a/practices/use-data-generation-tools.md b/practices/use-data-generation-tools.md new file mode 100644 index 0000000..4132f83 --- /dev/null +++ b/practices/use-data-generation-tools.md @@ -0,0 +1,89 @@ +# Use Data Generation Tools + +Data generation tools are namely about properly managing testing data. No +matter what type of test you are running there is an appropriate way to handle +your data so your tests are fast and reliable. + +For unit tests and other isolated tests data management might be as simple +defining variables or objects. This should almost always be your first choice. +When a need arises a simple [factory +method](https://refactoring.guru/design-patterns/factory-method) or factory +library like [Fishery](https://github.com/thoughtbot/fishery) will improve the +maintainability and readability. If your solution doesn't feel like it's making +things simpler, redirect to one that does. + +For integrated tests like E2E tests or integration tests, data generation might +be as simple as sql scripts to initialize and tear down test data before and +after your tests. Once data has grown in complexity or if you are dealing with +a complex legacy solution, introducing specialized tools like RedGate, dbForge, +SSDT or one of the many other SQL tool sets that fits your companies needs will +greatly improve. + +The core idea here whether in an isolated environment or in an integrated +environment is to ensure you keep your tests clear and to follow the "Arrange, +Act, Assert" pattern and that each test cleans up after its self so they are +reliable and easy to understand. Introduce tools as needs arise waiting to feel +the need for that tool before pulling the trigger and increasing the complexity +dependency structure. + +## When to Experiment + +You are a Developer and need to ensure that test data is easily managed so that +you can maintain a high quality developer experience and retain the users +positive experience. + +## How to Gain Traction + +First, bring up the need you have for data generation tools with your team so +you can gain consensus and ensure you are thinking of everyone's needs. + +Suggest a few tools and discuss different options with your team while being +understanding of the needs of your DevOps and DBA teams. + +Implement the agreed upon solution. + +## Lessons From The Field + +### Be careful about tests that depend on each other + +When dealing with data that can cross test boundaries like data inside your +database or global variables (`window` & `document` in a web context) make sure +that each test you write does not end up dependent on the setup or result of +another test. You can easily check this by running each of your tests in +isolation. If a test only passes when other tests are also run, some +modification needs to be made to decouple the tests from each other. + +## Deciding to Polish or Pitch + +After experimenting with this practice for 2-3 weeks, bring the team together to determine whether the following metrics and/or signals have changed in a positive direction: + +### Feedback Cycles + +Are your tests providing faster feedback? + +### Maintainability + +Are your tests easier to write and work with? + +### Improved Test Reliability + +Are your tests less flaky and more reliable? + +## Supported Capabilities + +### Test Data Management + +[Test Data Management](/capabilities/test-data-management.md) + +### Database Change Management + +[Database Change Management](/capabilities/database-change-management.md) + +### Continuous Delivery + +[Continuous Delivery](/capabilities/continuous-delivery.md) + +### Continuous Integration + +[Continuous Integration](/capabilities/continuous-integration.md) + From c7945c680ba15c720601c271b6340b1c8203718e Mon Sep 17 00:00:00 2001 From: Tristan Barrow Date: Wed, 31 Dec 2025 11:37:38 -0800 Subject: [PATCH 2/4] Data Gen Tools: simplify intro --- practices/use-data-generation-tools.md | 55 ++++++++------------------ 1 file changed, 17 insertions(+), 38 deletions(-) diff --git a/practices/use-data-generation-tools.md b/practices/use-data-generation-tools.md index 4132f83..e20197d 100644 --- a/practices/use-data-generation-tools.md +++ b/practices/use-data-generation-tools.md @@ -1,57 +1,36 @@ # Use Data Generation Tools -Data generation tools are namely about properly managing testing data. No -matter what type of test you are running there is an appropriate way to handle -your data so your tests are fast and reliable. - -For unit tests and other isolated tests data management might be as simple -defining variables or objects. This should almost always be your first choice. -When a need arises a simple [factory -method](https://refactoring.guru/design-patterns/factory-method) or factory -library like [Fishery](https://github.com/thoughtbot/fishery) will improve the -maintainability and readability. If your solution doesn't feel like it's making -things simpler, redirect to one that does. - -For integrated tests like E2E tests or integration tests, data generation might -be as simple as sql scripts to initialize and tear down test data before and -after your tests. Once data has grown in complexity or if you are dealing with -a complex legacy solution, introducing specialized tools like RedGate, dbForge, -SSDT or one of the many other SQL tool sets that fits your companies needs will -greatly improve. - -The core idea here whether in an isolated environment or in an integrated -environment is to ensure you keep your tests clear and to follow the "Arrange, -Act, Assert" pattern and that each test cleans up after its self so they are -reliable and easy to understand. Introduce tools as needs arise waiting to feel -the need for that tool before pulling the trigger and increasing the complexity -dependency structure. +Data generation tools reduce the complexity of generating complex data types or data rows. + +*Isolated Tests:* When a need arises use a simple [factory method](https://refactoring.guru/design-patterns/factory-method) or factory library like [Fishery](https://github.com/thoughtbot/fishery) to improve the maintainability and readability. + +*Integrated Tests:* Integrated Tests will usually need more data management. There are many good tools for this. For example: + +- RedGate +- dbForge +- SSDT ## When to Experiment -You are a Developer and need to ensure that test data is easily managed so that -you can maintain a high quality developer experience and retain the users -positive experience. +You are a Developer and need to ensure that test data is easily managed so that you can maintain a high quality developer experience and retain the users positive experience. ## How to Gain Traction -First, bring up the need you have for data generation tools with your team so -you can gain consensus and ensure you are thinking of everyone's needs. +First, bring up the need you have for data generation tools with your team so you can gain consensus and ensure you are thinking of everyone's needs. -Suggest a few tools and discuss different options with your team while being -understanding of the needs of your DevOps and DBA teams. +Suggest a few tools and discuss different options with your team while being understanding of the needs of your DevOps and DBA teams. Implement the agreed upon solution. ## Lessons From The Field +### Doint use them until you feel a need + +While data generation tools can be helpful in reducing complexity, if you don't see that complexity yet, consider defering the decision to add new tools until that complexity arises. + ### Be careful about tests that depend on each other -When dealing with data that can cross test boundaries like data inside your -database or global variables (`window` & `document` in a web context) make sure -that each test you write does not end up dependent on the setup or result of -another test. You can easily check this by running each of your tests in -isolation. If a test only passes when other tests are also run, some -modification needs to be made to decouple the tests from each other. +When dealing with data that can cross test boundaries like data inside your database or global variables (`window` & `document` in a web context) make sure that each test you write does not end up dependent on the setup or result of another test. You can easily check this by running each of your tests in isolation. If a test only passes when other tests are also run, some modification needs to be made to decouple the tests from each other. ## Deciding to Polish or Pitch From c06f91a63690e60be9fc2bcb4aeca197dd4e89ae Mon Sep 17 00:00:00 2001 From: Tristan Barrow Date: Wed, 31 Dec 2025 15:08:53 -0800 Subject: [PATCH 3/4] Data Gen Tools: improved descriptions --- practices/use-data-generation-tools.md | 28 ++++++++------------------ 1 file changed, 8 insertions(+), 20 deletions(-) diff --git a/practices/use-data-generation-tools.md b/practices/use-data-generation-tools.md index e20197d..9eb9548 100644 --- a/practices/use-data-generation-tools.md +++ b/practices/use-data-generation-tools.md @@ -36,33 +36,21 @@ When dealing with data that can cross test boundaries like data inside your data After experimenting with this practice for 2-3 weeks, bring the team together to determine whether the following metrics and/or signals have changed in a positive direction: -### Feedback Cycles +## Fast & Intangible -Are your tests providing faster feedback? - -### Maintainability - -Are your tests easier to write and work with? - -### Improved Test Reliability - -Are your tests less flaky and more reliable? +Your tests should be more *maintainable* after implementing this practice. Specifically, this means you should find yourself less likely to be fiddling with tests for lengthy periods to setup large sets of data. If you have not improved the time or energy to setup data, consider removing the tool and using factories for in memory data structures and raw sql scripts for sql data. ## Supported Capabilities -### Test Data Management - -[Test Data Management](/capabilities/test-data-management.md) - -### Database Change Management +### [Test Data Management](/capabilities/test-data-management.md) -[Database Change Management](/capabilities/database-change-management.md) +The reason we should use Data Generation Tools is pimerily for Test Data Management but Test Data Management can be done without Tools and that should be considered depending on the use case. -### Continuous Delivery +### [Database Change Management](/capabilities/database-change-management.md) -[Continuous Delivery](/capabilities/continuous-delivery.md) +Based on your strategy for Database Change Management or a lack there of in the past, tooling might be an essential part of how you continue or start doing Database Change Management. -### Continuous Integration +### [Continuous Delivery](/capabilities/continuous-delivery.md) & [Continuous Integration](/capabilities/continuous-integration.md) -[Continuous Integration](/capabilities/continuous-integration.md) +Any tests that endup in your pipelines will need Test Data Management which might be done by tools depending on your needs. From 17b568cffc637696b9de91bb70fca088d2afe051 Mon Sep 17 00:00:00 2001 From: Tristan Barrow Date: Wed, 31 Dec 2025 15:19:16 -0800 Subject: [PATCH 4/4] Data Gen Tools: minor fixes --- practices/use-data-generation-tools.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/practices/use-data-generation-tools.md b/practices/use-data-generation-tools.md index 9eb9548..ea6e731 100644 --- a/practices/use-data-generation-tools.md +++ b/practices/use-data-generation-tools.md @@ -6,9 +6,11 @@ Data generation tools reduce the complexity of generating complex data types or *Integrated Tests:* Integrated Tests will usually need more data management. There are many good tools for this. For example: -- RedGate -- dbForge -- SSDT +- [RedGate](https://www.red-gate.com/) +- [dbForge](https://www.devart.com/) +- [SSDT](https://learn.microsoft.com/en-us/sql/ssdt/sql-server-data-tools?view=sql-server-ver17) + +These are just examples, it is recommended to research many different tool options before committing to them. ## When to Experiment @@ -24,9 +26,9 @@ Implement the agreed upon solution. ## Lessons From The Field -### Doint use them until you feel a need +### Don't use them until you feel a need -While data generation tools can be helpful in reducing complexity, if you don't see that complexity yet, consider defering the decision to add new tools until that complexity arises. +While data generation tools can be helpful in reducing complexity, if you don't see that complexity yet, consider deferring the decision to add new tools until that complexity arises. ### Be careful about tests that depend on each other @@ -44,7 +46,7 @@ Your tests should be more *maintainable* after implementing this practice. Speci ### [Test Data Management](/capabilities/test-data-management.md) -The reason we should use Data Generation Tools is pimerily for Test Data Management but Test Data Management can be done without Tools and that should be considered depending on the use case. +The reason we should use Data Generation Tools is primarily for Test Data Management but Test Data Management can be done without Tools and that should be considered depending on the use case. ### [Database Change Management](/capabilities/database-change-management.md) @@ -52,5 +54,5 @@ Based on your strategy for Database Change Management or a lack there of in the ### [Continuous Delivery](/capabilities/continuous-delivery.md) & [Continuous Integration](/capabilities/continuous-integration.md) -Any tests that endup in your pipelines will need Test Data Management which might be done by tools depending on your needs. +Any tests that end up in your pipelines will need Test Data Management which might be done by tools depending on your needs.