Skip to content

Pipeline Use

Jump to bottom
Magic edited this page Dec 25, 2024 · 2 revisions

Important Note I am by no means a pipeline expert. I've learned what I have out of necessity. If you have suggestions or can do better. Then help me out! I'm more than open to suggestions, put in a pull request, or contact me. Improving the process will only help everyone involved!

How I Use Magic EF In Pipelines

I utilize Magic EF within my Azure pipeline process. I would imagine there's more refine ways to implement what I have, but it's good enough for me! I'll share parts of my personal company pipeline yml file to show and explain what I'm doing.

Step 1 - Install dependencies

# Step 2: Install Required Workloads and Tools
- script: |
    dotnet tool install --global dotnet-ef
    dotnet tool install --global MagicEf
  displayName: 'Install Required Workloads and Tools'

Step 2 - Run the Migration Runner

CRITICAL NOTE - This feature has not yet been released yet in Magic EF. It is part of the protocol, but will be released when further refined and higher confidence of stability is achieved. So, you will need to have your own migration process for now. As the process will likely change as the goal is to make it as simple as possible in the future.

Utilizing Magic EF's migration runner that's attached to the protocol. Within the pipeline it easily runs the migration process on your behalf based on the repositories specifications you've added within the protocol. It's simple, easy, and will produce error logs throughout to keep you updated and protect you throughout the process. Automatic transaction rollbacks on failure has already been implemented on your behalf as well. You can customize it further if you have a custom process.

# removed restore dependencies
- script: |
    dotnet restore ./DatabaseMigrationRunner/DatabaseMigrationRunner.csproj
  displayName: 'Restore Dependencies'

- script: |
    dotnet publish $(databaseMigrationRunnerProjectPath) --configuration $(buildConfiguration) --output "$(databaseMigrationRunnersOutputDir)" --no-restore
  displayName: 'Build and Publish Database Migration Runner'

# Run and Verify Database Migration Runner
- task: PowerShell@2
  displayName: 'Run and Verify Database Migration Runner'
  inputs:
    targetType: 'inline'
    script: |
      # Set environment variable for the CLI app
      $env:MyDbString= "$(MyDbString)"
      $env:AnotherDatabaseDbString= "$(AnotherDatabaseDbString)"

      # Navigate to the published output directory
      cd "$(Build.ArtifactStagingDirectory)\DatabaseMigrationRunner"

      # Define the log file path
      $logFile = "$(Build.ArtifactStagingDirectory)\DatabaseMigrationRunner.log"

      # Run the CLI app and capture logs
      Write-Host "Starting Database Migration Runner..."
      ./DatabaseMigrationRunner.exe 2>&1 | Tee-Object -FilePath $logFile

      # Log file output
      Write-Host "Log Contents:"
      Get-Content $logFile | Out-String | Write-Host

      # Check for failure message in the log
      Write-Host "Checking log file for failure message: $logFile"
      $failureMessages = Select-String -Path $logFile -Pattern "migration failed:" -SimpleMatch -CaseSensitive:$false

      if ($failureMessages) {
          Write-Host "Migration failed detected! Outputting error message(s):"
          foreach ($message in $failureMessages) {
              Write-Host "Error: $($message.Line)"
          }

          # Exit with an error
          Write-Host "Database migration failed. See log for details."
          exit 1
      }

      Write-Host "No migration failures detected. Migration completed successfully."

Note - I read the logs due to the fact I'm still learning to properly utilize Azure pipeline. This will be further refined before being added to the Magic EF protocol officially. Feel free to contact me and request early access to the protocol if you want. I'm only hiding it because I want to prevent headaches. But if you understand the risk, I'm more than happy to share!

Step 3 - Run the Scaffold With Magic EF

The next step for the pipeline is to run the dotnet scaffolding process. Then right after, we must run some Magic EF commands to remedy common dotnet scaffolding issues:

# Step 4: Run EF Scaffold and MagicEF
- task: PowerShell@2
  displayName: 'Run EF Scaffold and MagicEF'
  inputs:
    targetType: 'inline'
    script: |
      $ErrorActionPreference = "Stop"

      $env:MyDbString = "$(MyDbString)"

      $baseDirectory = Split-Path -Parent $MyInvocation.MyCommand.Definition

      $projectPath = {Removed}
      $outputDir = {Removed}
      $contextDir = {Removed}

      dotnet ef dbcontext scaffold $env:MyDbString 'Microsoft.EntityFrameworkCore.SqlServer' `
          --project $projectPath `
          --context ReadOnlyDbContext `
          --output-dir $outputDir `
          --context-dir $contextDir `
          --namespace DataAccess `
          --force `
          --verbose `
          --data-annotations

      MagicEF --ambiguousIndex --directoryPath {Removed}
      MagicEF --removeOnConfiguring --filePath {Removed}
      MagicEF --separateVirtualProperties --directoryPath {Removed}

As shown, after we run the standard Microsoft provided dotnet ef scaffolding. I run the "ambiguousIndex", "removeOnConfiguring", and optionally (if you use this in your standard process) "separateVirtualProperties". As the protocol was created to remedy various pipeline issues during the development process. Therefore, only 2-3 of the magic commands after a dotnet scaffold is required to remedy breaking issues that would otherwise have occurred.

Conclusion

That's it! You can see that in the pipeline I was able to safely migrate my changes to the database. Then I could use dotnet scaffold. And finally I would remedy issues that would have occurred with dotnet scaffold if it were alone. Therefore, we've successfully forced the database first codebase to update to the current environment we're moving too.

Then after those tasks run, you follow the rest of your normal procedures by building the projects, publishing them, etc. As they will all build now utilizing the updated code, each having the new updated DbContext and so on. If anything goes wrong, it was very likely because Magic EF was protecting you. Something was not aligned correctly and it should fail on your behalf.

Clone this wiki locally