Skip to content

tune the properties #43439

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 26 commits into
base: main
Choose a base branch
from
Open

tune the properties #43439

wants to merge 26 commits into from

Conversation

@Zyysurely
Copy link
Contributor

Description

Please add an informative description that covers that changes made by the pull request and link all relevant issues.

If an SDK is being regenerated based on a new API spec, a link to the pull request containing these API spec changes should be included above.

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

Neehar Duvvuri and others added 26 commits September 30, 2025 12:36
skeleton code
fa2f8aa
add function that logs eval results to app insights
8dfde5b
format
771e19d
log red team data
7174bc4
get agent info from app insights config
f25760c
merge with main
08d3e36
@YoYoJa
add eval result converter (#43233)
73fb9c7 
* add eval result converter

* Add result converter

* update converter params to optional

* add eval meta data

* fix type

* remove useless file

* get eval meta data as input

* fix build errors

* remove useless import

* resolve comments

* update

* update comments
rename function
46e0504
Merge branch 'main' of https://github.com/Azure/azure-sdk-for-python …
9cae8cf 
…into needuv/structured-results-otel-logging
fix a thing
17cdd1e
@YoYoJa @singankit
@catalinaperalta @cperaltah @pvaneck @v-dharmarajv @annatisch
Jessli/convert (#43342)
1f13f56 
* add eval result converter

* Add result converter

* update converter params to optional

* add eval meta data

* fix type

* remove useless file

* get eval meta data as input

* fix build errors

* remove useless import

* resolve comments

* update

* update comments

* fix checker failure

* Groundedness Evaluator to not add tool result to tool call message (#43290)

* Groundededness Evalautor to not add tool result to tool call message

* Fixing reformatting issues

* Add ledger certificate package (#43278)

* add ledger certificate package

* regen

* update changelog

---------

Co-authored-by: catalinaperalta <[email protected]>

* [Identity] Update test-resources bicep (#43304)

The vmSize for the AKS resource was updated to an SKU that is available
in our subscription/location.

Explicit PrincipalType fields were removed from role assignments that could
potentially be user principals. Azure can automatically determine the
type.

Signed-off-by: Paul Van Eck <[email protected]>

* [Communication Shared] Adding the mypy fixes (#42925)

* Adding the mypy fixes

* addressing the comments

* addressing comments

* Make docs happy

* Updated docstring references

---------

Co-authored-by: antisch <[email protected]>

* add error msg and error code

* Surface evaluator error msg

---------

Signed-off-by: Paul Van Eck <[email protected]>
Co-authored-by: Ankit Singhal <[email protected]>
Co-authored-by: catalinaperalta <[email protected]>
Co-authored-by: catalinaperalta <[email protected]>
Co-authored-by: Paul Van Eck <[email protected]>
Co-authored-by: Vinothini Dharmaraj <[email protected]>
Co-authored-by: antisch <[email protected]>
@YoYoJa @singankit
@catalinaperalta @cperaltah @pvaneck @v-dharmarajv @annatisch
Fix usage (#43355)
6104b0c 
* add eval result converter

* Add result converter

* update converter params to optional

* add eval meta data

* fix type

* remove useless file

* get eval meta data as input

* fix build errors

* remove useless import

* resolve comments

* update

* update comments

* fix checker failure

* Groundedness Evaluator to not add tool result to tool call message (#43290)

* Groundededness Evalautor to not add tool result to tool call message

* Fixing reformatting issues

* Add ledger certificate package (#43278)

* add ledger certificate package

* regen

* update changelog

---------

Co-authored-by: catalinaperalta <[email protected]>

* [Identity] Update test-resources bicep (#43304)

The vmSize for the AKS resource was updated to an SKU that is available
in our subscription/location.

Explicit PrincipalType fields were removed from role assignments that could
potentially be user principals. Azure can automatically determine the
type.

Signed-off-by: Paul Van Eck <[email protected]>

* [Communication Shared] Adding the mypy fixes (#42925)

* Adding the mypy fixes

* addressing the comments

* addressing comments

* Make docs happy

* Updated docstring references

---------

Co-authored-by: antisch <[email protected]>

* add error msg and error code

* Surface evaluator error msg

* update UT

* fix usage

---------

Signed-off-by: Paul Van Eck <[email protected]>
Co-authored-by: Ankit Singhal <[email protected]>
Co-authored-by: catalinaperalta <[email protected]>
Co-authored-by: catalinaperalta <[email protected]>
Co-authored-by: Paul Van Eck <[email protected]>
Co-authored-by: Vinothini Dharmaraj <[email protected]>
Co-authored-by: antisch <[email protected]>
save
7858ee8
Merge branch 'needuv/structured-results-otel-logging' of https://gith…
0916d62 
…ub.com/Azure/azure-sdk-for-python into needuv/structured-results-otel-logging
add _type to evals/aoai graders
e8942cb
@YoYoJa @singankit
@catalinaperalta @cperaltah @pvaneck @v-dharmarajv @annatisch
Jessli/convert make eval_meta_data optional (#43376)
920c964 
* add eval result converter

* Add result converter

* update converter params to optional

* add eval meta data

* fix type

* remove useless file

* get eval meta data as input

* fix build errors

* remove useless import

* resolve comments

* update

* update comments

* fix checker failure

* Groundedness Evaluator to not add tool result to tool call message (#43290)

* Groundededness Evalautor to not add tool result to tool call message

* Fixing reformatting issues

* Add ledger certificate package (#43278)

* add ledger certificate package

* regen

* update changelog

---------

Co-authored-by: catalinaperalta <[email protected]>

* [Identity] Update test-resources bicep (#43304)

The vmSize for the AKS resource was updated to an SKU that is available
in our subscription/location.

Explicit PrincipalType fields were removed from role assignments that could
potentially be user principals. Azure can automatically determine the
type.

Signed-off-by: Paul Van Eck <[email protected]>

* [Communication Shared] Adding the mypy fixes (#42925)

* Adding the mypy fixes

* addressing the comments

* addressing comments

* Make docs happy

* Updated docstring references

---------

Co-authored-by: antisch <[email protected]>

* add error msg and error code

* Surface evaluator error msg

* update UT

* fix usage

* make eval_meta_data optional

* remove useless lines

---------

Signed-off-by: Paul Van Eck <[email protected]>
Co-authored-by: Ankit Singhal <[email protected]>
Co-authored-by: catalinaperalta <[email protected]>
Co-authored-by: catalinaperalta <[email protected]>
Co-authored-by: Paul Van Eck <[email protected]>
Co-authored-by: Vinothini Dharmaraj <[email protected]>
Co-authored-by: antisch <[email protected]>
add error logging for otel event emission
24e5cc1
Merge branch 'needuv/structured-results-otel-logging' of https://gith…
f60e574 
…ub.com/Azure/azure-sdk-for-python into needuv/structured-results-otel-logging
fix merge conflicts
58035f5
add input/output tokens for prompty evals
4a5ffbf
@YoYoJa
Jessli/convert - update param name to add underscore (#43411)
776224c 
* add eval result converter

* Add result converter

* update converter params to optional

* add eval meta data

* fix type

* remove useless file

* get eval meta data as input

* fix build errors

* remove useless import

* resolve comments

* update

* update comments

* fix checker failure

* add error msg and error code

* Surface evaluator error msg

* update UT

* fix usage

* make eval_meta_data optional

* remove useless lines

* update param name to add underscore
exclude token counts from aggregation
7e2969f
Merge branch 'needuv/structured-results-otel-logging' of https://gith…
81ad396 
…ub.com/Azure/azure-sdk-for-python into needuv/structured-results-otel-logging
add total token count to prompty output
f70e979
fix prompty tests
e6e0746
@yingyingzhaozyying
tune the properties
73fcf23
@Zyysurely Zyysurely requested a review from a team as a code owner October 15, 2025 06:38
@github-actions github-actions bot added the Evaluation Issues related to the client library for Azure AI Evaluation label Oct 15, 2025
Base automatically changed from needuv/structured-results-otel-logging to main October 22, 2025 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Evaluation Issues related to the client library for Azure AI Evaluation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Zyysurely @YoYoJa @yingyingzhaozyying