Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seeking reference for the shrinkage parameter in RandomOversampler #1030

Open
EssamWisam opened this issue Jul 15, 2023 · 2 comments
Open

Seeking reference for the shrinkage parameter in RandomOversampler #1030

EssamWisam opened this issue Jul 15, 2023 · 2 comments

Comments

@EssamWisam
Copy link

I was trying to implement ROSE (Random Oversampling Examples) in Julia, and after considering the paper I decided to look at imbalanced-learn's implementation. There seems to be a shrinkage parameter that has no mention in the paper. I see that its multiplied by the smoothing matrix used for the kernel as in here near line 214 and understand its effect; however, why the specific name "shrinkage"? Especially when it causes the points to spread farther apart...

@glemaitre
Copy link
Member

I think I found that this parameter was behaving the same way as the learning_rate in GBDT, also known as shrinkage. You also have the concept of shrunk covariance that based on the value of the shrinkage your "shrink" the empirical covariance to an identity matrix.

Here, you use the shrinkage to "shrink" towards the simple random oversampling. Then, it is true that it might be counter-intuitive that the value of shrinkage is actually the opposite of what you would expect :).

@glemaitre
Copy link
Member

And the reason why this parameter is not in the paper is just because this single parameter can switch from the normal random over-sampling to ROSE without the need to create a dedicated class.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants