Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow Customizing Spot Allocation Strategy #427

Open
djmetzle opened this issue Oct 31, 2023 · 2 comments · May be fixed by #481
Open

Allow Customizing Spot Allocation Strategy #427

djmetzle opened this issue Oct 31, 2023 · 2 comments · May be fixed by #481
Labels
enhancement good first issue

Comments

@djmetzle
Copy link

djmetzle commented Oct 31, 2023 •
edited
Loading

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Description

EC2 allows choosing multiple allocation strategies for spot requests. It would be great to be able to specify the Allocation strategy for packer builds on Spot.

Use Case(s)

In our use case (building AMIs in CI/CD) we are seeing frequent evictions of spot instances mid-build, or even before launching the build instances. It could help to choose a strategy like capacity-optimized or price-capacity-optimized, in order to see less frequent evictions and failures due to lack of capacity.

Potential configuration

An option that exposes spot_allocation_strategy on the AWS builder would do the trick. That option could still default to lowest-price for backwards behavioral compatibility. But price-capacity-optimized is the recommended value.

Potential References

It looks like the AWS SDK exposes this configuration point as an Enum (sorry, this file is to big to permalink to):
https://github.com/aws/aws-sdk-go/blob/main/service/ec2/api.go

I think it would only be a matter of adding that option to the spot fleet request here?

packer-plugin-amazon/builder/common/step_run_spot_instance.go

Lines 368 to 384 in e8a4526

createFleetInput := &ec2.CreateFleetInput{
LaunchTemplateConfigs: []*ec2.FleetLaunchTemplateConfigRequest{
{
LaunchTemplateSpecification: &ec2.FleetLaunchTemplateSpecificationRequest{
LaunchTemplateName: aws.String(launchTemplateName),
Version: aws.String("1"),
},
Overrides: overrides,
},
},
ReplaceUnhealthyInstances: aws.Bool(false),
TargetCapacitySpecification: &ec2.TargetCapacitySpecificationRequest{
TotalTargetCapacity: aws.Int64(1),
DefaultTargetCapacityType: aws.String("spot"),
},
Type: aws.String("instant"),
}

From there, a configuration option would need to be exposed from input and threaded to the fleet request.

Apologies if this is too vague, as i'm not very well versed in Go.
@lbajolet-hashicorp
Copy link
Contributor

lbajolet-hashicorp commented Nov 1, 2023 •
edited
Loading

Hi @djmetzle,

This may be a good addition indeed! This wouldn't be too hard to implement as well, so I figure this would be well suited for a first contribution.

If you're interested in this, please let me know, and we can provide guidance on that.

I'll mark the issue as such, and place it in our backlog in the meantime, we can revisit it later.

Thanks for bringing this up!

PS: for reference, the allocation strategy is not exposed as an enum but as a *string, but there's a couple of constants we can use to support them, they're exposed as the ec2.AllocationStrategy_Values() function, which we can rely on for validation.

@djmetzle
Copy link
Author

djmetzle commented Nov 1, 2023

Hello @lbajolet-hashicorp! Thanks for the fast feedback.

I dont think i will personally be able to contribute a solution here. I'm relatively familiar with the EC2 SDK (from other languages) which is how i searched out the link from the description. But i do not think i will have the time to learn all the input validation and testing paradigms needed to make the feature add here.

But i do think this would be an easy to add, and very useful feature. Mostly as the lowest-price strategy is not the recommended default. Even just switching to price-capacity-optimized as the default strategy could be a worthwhile change.

@vucong2409 vucong2409 linked a pull request May 10, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement good first issue
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: allow customizing spot allocation strategy vucong2409/packer-plugin-amazon
2 participants
@djmetzle @lbajolet-hashicorp