Skip to content

feat: move ci test #137

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
shijinpjlab wants to merge 8 commits into from
Closed

feat: move ci test #137

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
9289ea2
feat: ci update
shijinpjlab Jul 31, 2025
47f6ff1
feat: update example input path
shijinpjlab Jul 31, 2025
733d2ef
feat: move example ci test
shijinpjlab Jul 31, 2025
a650110
feat: OPENAI_KEY set check
shijinpjlab Jul 31, 2025
1443fca
feat: Test set check branch
shijinpjlab Jul 31, 2025
505a68a
feat: set ci environment
shijinpjlab Jul 31, 2025
2fae6b9
feat: set ci environment
shijinpjlab Jul 31, 2025
a082f01
feat: use pull_request_target
shijinpjlab Jul 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 13 additions & 3 deletions .github/workflows/IntegrationTest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ name: Test

on:
push:
branches: [ "*" ]
pull_request:
branches: [ "*" ]
branches: [ "main", "dev" ]
pull_request_target:
branches: [ "main", "dev" ]
workflow_dispatch:


Expand Down Expand Up @@ -58,3 +58,13 @@ jobs:
- name: Run unit tests
run: |
pytest test/scripts --ignore=test/scripts/data
- name: Run examples
env:
OPENAI_KEY: ${{ secrets.OPENAI_KEY }}
run: |
python examples/core/score.py
python examples/custom/sdk_custom_rule.py
python examples/llm_and_rule/llm_and_rule_mix.py
python examples/register/sdk_register_llm.py
python examples/register/sdk_register_prompt.py
python examples/register/sdk_register_rule.py
13 changes: 0 additions & 13 deletions .github/workflows/publish-to-pypi.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,19 +24,6 @@ jobs:
python -m pip install --upgrade pip
pip install build wheel==0.43.0 twine==6.0.1 packaging==24.1 pkginfo==1.12.0

- name: Run examples
env:
OPENAI_KEY: ${{ secrets.OPENAI_KEY }}
run: |
python examples/core/score.py
python examples/custom/sdk_custom_rule.py
python examples/llm_and_rule/llm_and_rule_mix.py
python examples/register/sdk_register_llm.py
python examples/register/sdk_register_prompt.py
python examples/register/sdk_register_rule.py
python examples/multi_turn_dialogues/sdk_mtbench101_llm.py
python examples/multi_turn_dialogues/sdk_mtbench101_rule_all.py

- name: Build package
run: python setup.py bdist_wheel

Expand Down
2 changes: 2 additions & 0 deletions examples/core/score.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
OPENAI_MODEL = 'deepseek-chat'
OPENAI_URL = 'https://api.deepseek.com/v1'
OPENAI_KEY = os.getenv("OPENAI_KEY")
if not OPENAI_KEY:
raise ValueError('OPENAI_KEY is not set correctly.')


def llm():
Expand Down
2 changes: 1 addition & 1 deletion examples/custom/sdk_custom_llm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
from dingo.exec import Executor

input_data = {
"input_path": "../../test/data/test_local_jsonl.jsonl",
"input_path": "test/data/test_local_jsonl.jsonl",
"dataset": {
"source": "local",
"format": "jsonl",
Expand Down
2 changes: 1 addition & 1 deletion examples/custom/sdk_custom_rule.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
from dingo.exec import Executor

input_data = {
"input_path": "../../test/data/test_local_json.json",
"input_path": "test/data/test_local_json.json",
"dataset": {
"source": "local",
"format": "json",
Expand Down
4 changes: 3 additions & 1 deletion examples/llm_and_rule/llm_and_rule_mix.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,11 @@
OPENAI_MODEL = 'deepseek-chat'
OPENAI_URL = 'https://api.deepseek.com/v1'
OPENAI_KEY = os.getenv("OPENAI_KEY")
if not OPENAI_KEY:
raise ValueError('OPENAI_KEY is not set correctly.')

input_data = {
"input_path": "../../test/data/test_local_jsonl.jsonl",
"input_path": "test/data/test_local_jsonl.jsonl",
"dataset": {
"source": "local",
"format": "jsonl",
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_and_rule/local_llm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
from dingo.exec import Executor

input_data = {
"input_path": "../../test/data/test_local_jsonl.jsonl",
"input_path": "test/data/test_local_jsonl.jsonl",
"dataset": {
"source": "local",
"format": "jsonl",
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_and_rule/remote_llm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
from dingo.exec import Executor

input_data = {
"input_path": "../../test/data/test_local_jsonl.jsonl",
"input_path": "test/data/test_local_jsonl.jsonl",
"dataset": {
"source": "local",
"format": "jsonl",
Expand Down
2 changes: 1 addition & 1 deletion examples/multi_turn_dialogues/sdk_mtbench101_llm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
OPENAI_KEY = os.getenv("OPENAI_KEY")

input_data = {
"input_path": "../../test/data/test_mtbench101_jsonl.jsonl",
"input_path": "test/data/test_mtbench101_jsonl.jsonl",
"dataset": {
"source": "local",
"format": "multi_turn_dialog",
Expand Down
2 changes: 1 addition & 1 deletion examples/multi_turn_dialogues/sdk_mtbench101_rule_all.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
from dingo.exec import Executor

input_data = {
"input_path": "../../test/data/test_mtbench101_jsonl.jsonl",
"input_path": "test/data/test_mtbench101_jsonl.jsonl",
"dataset": {
"source": "local",
"format": "multi_turn_dialog",
Expand Down
4 changes: 3 additions & 1 deletion examples/register/sdk_register_llm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@
OPENAI_MODEL = 'deepseek-chat'
OPENAI_URL = 'https://api.deepseek.com/v1'
OPENAI_KEY = os.getenv("OPENAI_KEY")
if not OPENAI_KEY:
raise ValueError('OPENAI_KEY is not set correctly.')


@Model.llm_register('LlmTextQualityRegister')
Expand Down Expand Up @@ -54,7 +56,7 @@ def process_response(cls, response: str) -> ModelRes:
from dingo.exec import Executor

input_data = {
"input_path": "../../test/data/test_local_jsonl.jsonl",
"input_path": "test/data/test_local_jsonl.jsonl",
"dataset": {
"source": "local",
"format": "jsonl",
Expand Down
4 changes: 3 additions & 1 deletion examples/register/sdk_register_prompt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
OPENAI_MODEL = 'deepseek-chat'
OPENAI_URL = 'https://api.deepseek.com/v1'
OPENAI_KEY = os.getenv("OPENAI_KEY")
if not OPENAI_KEY:
raise ValueError('OPENAI_KEY is not set correctly.')


@Model.prompt_register("QUALITY_BAD_SIMILARITY", [])
Expand All @@ -24,7 +26,7 @@ class PromptRepeatDemo(BasePrompt):
from dingo.exec import Executor

input_data = {
"input_path": "../../test/data/test_local_jsonl.jsonl",
"input_path": "test/data/test_local_jsonl.jsonl",
"dataset": {
"source": "local",
"format": "jsonl",
Expand Down
2 changes: 1 addition & 1 deletion examples/register/sdk_register_rule.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ def eval(cls, input_data: Data) -> ModelRes:
from dingo.exec import Executor

input_data = {
"input_path": "../../test/data/test_local_json.json",
"input_path": "test/data/test_local_json.json",
"dataset": {
"source": "local",
"format": "json",
Expand Down