[AutoParallel] Add_llama_dpo_auto_parallel_intermediate

[Xing-lil]

PR types

New features

PR changes

Others

Description

Support auto_parallel in llama DPO with intermediate API.
The loss curve is same over 300 steps:

The loss diff is zero:

The performance:

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: Patch coverage is 8.50767% with 656 lines in your changes missing coverage. Please review.

Project coverage is 49.73%. Comparing base (da96c72) to head (ab4b733).
Report is 39 commits behind head on develop.

Files with missing lines	Patch %	Lines
paddlenlp/trl/dpo_auto_trainer.py	7.91%	628 Missing ⚠️
paddlenlp/transformers/dpo_criterion.py	12.50%	21 Missing ⚠️
paddlenlp/transformers/llama/modeling_network.py	0.00%	5 Missing ⚠️
paddlenlp/trl/trl_data.py	50.00%	2 Missing ⚠️

❌ Your patch check has failed because the patch coverage (8.50%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.
❌ Your project check has failed because the head coverage (49.73%) is below the target coverage (58.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop   #10044      +/-   ##
===========================================
- Coverage    50.21%   49.73%   -0.48%     
===========================================
  Files          757      758       +1     
  Lines       122398   123211     +813     
===========================================
- Hits         61466    61285     -181     
- Misses       60932    61926     +994     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jeff41404]

The description are too simple, the description needs to include a comparison of loss and performance of DPO between manual parallel and auto parallel.