Skip to content

Commit b916bb7

Browse files
LeonardHdLeonardHd
authored
feat!: use data factory testing framework expressions evaluator (#118)
1 parent 91c352e commit b916bb7

File tree

68 files changed

+921
-3791
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

68 files changed

+921
-3791
lines changed

‎.github/workflows/ci-cd.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
name: CI-CD
2+
name: CI/CD
33

44
on: [push] # yamllint disable-line rule:truthy
55

‎.github/workflows/ci.yml

+6-3
Original file line numberDiff line numberDiff line change
@@ -59,14 +59,17 @@ jobs:
5959
- name: Install dependencies
6060
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
6161
run: poetry install --no-interaction --no-root
62+
- name: Build dotnet dependencies
63+
run: dotnet build src/data_factory_testing_framework/_pythonnet/Evaluator.csproj -o src/data_factory_testing_framework/_pythonnet/bin
6264
- name: Install project
63-
run: poetry install --no-interaction
65+
run: poetry run pip install -e .
6466
- name: Run linting and tests
6567
run: poetry run pre-commit run --all-files --show-diff-on-failure
6668
- name: Set build version and build package
6769
run: |
68-
poetry version $BUILD_VERSION
69-
poetry build
70+
# create a version file with the build version
71+
echo $BUILD_VERSION > VERSION
72+
poetry run python -m build
7073
env:
7174
BUILD_VERSION: ${{ needs.version.outputs.version }}
7275
- name: Upload dist

‎.gitignore

+10
Original file line numberDiff line numberDiff line change
@@ -401,3 +401,13 @@ FodyWeavers.xsd
401401

402402
# Distributions / Releases
403403
dist
404+
*.egg-info
405+
406+
# Dotnet Evaluator
407+
src/data_factory_testing_framework/DataFactoryTestingFrameworkEvaluator
408+
test_sdist
409+
test_wheel
410+
VERSION
411+
build
412+
dotnet_build
413+
.mono

‎.pre-commit-config.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ repos:
99
args: [--config, ./.conventional-commits/commitlint.precommit.config.js]
1010
additional_dependencies: ['@commitlint/[email protected]']
1111
- repo: https://github.com/python-poetry/poetry
12-
rev: 1.6.1
12+
rev: 1.8.2
1313
hooks:
1414
- id: poetry-check
1515
args: []

‎DataFactoryTestingFramework.sln

+33
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
2+
Microsoft Visual Studio Solution File, Format Version 12.00
3+
# Visual Studio Version 17
4+
VisualStudioVersion = 17.9.34616.47
5+
MinimumVisualStudioVersion = 10.0.40219.1
6+
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "src", "src", "{35B3C0BC-05AE-4FA2-8250-DC710420F84A}"
7+
EndProject
8+
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Evaluator", "src/data_factory_testing_framework/_pythonnet/Evaluator.csproj", "{636E6B65-E552-41D1-94D5-63B8E6D09F73}"
9+
EndProject
10+
Global
11+
GlobalSection(SolutionConfigurationPlatforms) = preSolution
12+
Release|Any CPU = Release|Any CPU
13+
Release|x64 = Release|x64
14+
Release|x86 = Release|x86
15+
EndGlobalSection
16+
GlobalSection(ProjectConfigurationPlatforms) = postSolution
17+
{636E6B65-E552-41D1-94D5-63B8E6D09F73}.Release|Any CPU.ActiveCfg = DataFactoryTestingFrameworkEvaluator|Any CPU
18+
{636E6B65-E552-41D1-94D5-63B8E6D09F73}.Release|Any CPU.Build.0 = DataFactoryTestingFrameworkEvaluator|Any CPU
19+
{636E6B65-E552-41D1-94D5-63B8E6D09F73}.Release|x64.ActiveCfg = DataFactoryTestingFrameworkEvaluator|Any CPU
20+
{636E6B65-E552-41D1-94D5-63B8E6D09F73}.Release|x64.Build.0 = DataFactoryTestingFrameworkEvaluator|Any CPU
21+
{636E6B65-E552-41D1-94D5-63B8E6D09F73}.Release|x86.ActiveCfg = DataFactoryTestingFrameworkEvaluator|Any CPU
22+
{636E6B65-E552-41D1-94D5-63B8E6D09F73}.Release|x86.Build.0 = DataFactoryTestingFrameworkEvaluator|Any CPU
23+
EndGlobalSection
24+
GlobalSection(SolutionProperties) = preSolution
25+
HideSolutionNode = FALSE
26+
EndGlobalSection
27+
GlobalSection(NestedProjects) = preSolution
28+
{636E6B65-E552-41D1-94D5-63B8E6D09F73} = {35B3C0BC-05AE-4FA2-8250-DC710420F84A}
29+
EndGlobalSection
30+
GlobalSection(ExtensibilityGlobals) = postSolution
31+
SolutionGuid = {63AF779A-5A49-4A39-B72B-22512AB23EFE}
32+
EndGlobalSection
33+
EndGlobal

‎MANIFEST.in

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
include VERSION
2+
include DataFactoryTestingFramework.sln
3+
recursive-include src *.cs *.cs
4+
recursive-include src *.csproj *.csproj
5+
prune src/data_factory_testing_framework/_pythonnet/obj

‎README.md

-9
Original file line numberDiff line numberDiff line change
@@ -84,15 +84,6 @@ Azure Data Factory:
8484
1. [Copy blobs example](examples/data_factory/copy_blobs/README.md)
8585
2. [Batch job example](examples/data_factory/batch_job/README.md)
8686

87-
## Limitations :warning:
88-
89-
The framework has the following limitations at the moment:
90-
91-
1. The framework reimplements the Data Factory expression language in Python. This means that it might not be 100% accurate with the actual Data Factory language, especially when it comes to xml functions.
92-
2. The framework does not support automatic type conversion of arguments passed to functions in expressions.
93-
94-
> These limitations are top priority to be resolved in the future.
95-
9687
## Contributing :handshake:
9788

9889
This project welcomes contributions and suggestions. Most contributions require you to agree to a

‎docs/basic/installing_and_initializing_framework.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,9 @@
22

33
## Install dotnet runtime
44

5-
Install the dotnet runtime (not SDK) from [here](https://dotnet.microsoft.com/en-us/download/dotnet/8.0). This is required to run some expression functions on dotnet just like in Data Factory.
5+
Install the dotnet runtime (not SDK) from [here](https://dotnet.microsoft.com/en-us/download/dotnet/8.0).
6+
Version 8.0 is recommended (the minimum version required is 5.0).
7+
The dotnet runtime is required to evaluate the Data Factory expression functions on dotnet just like in Data Factory.
68

79
## Installing the framework
810

‎examples/fabric/batch_job/test_fabric_batchjob_unit.py

-1
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,6 @@ def pipeline(test_framework: TestFramework) -> Pipeline:
2424

2525

2626
def test_set_job_container_url(test_framework: TestFramework, pipeline: Pipeline) -> None:
27-
int(1.1)
2827
# Arrange
2928
activity = pipeline.get_activity_by_name("Set Job Container URL")
3029
state = PipelineRunState(

0 commit comments

Comments
 (0)