Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minimum bookable free mcp #1306

Conversation

ramonfigueiredo
Copy link
Collaborator

Prevent booking frames on hosts with full MCP

Change the host hardwareState from REPAIR to UP, when the amount of free space in the /mcp directory is greater or equals to the minimum required and delete the comment associated with the host. Check if the host with REPAIR status has comments with subject=SUBJECT_COMMENT_FULL_MCP_DIR and user=CUEBOT_COMMENT_USER and delete the comments. If the comment exists, change the host hardwareState from REPAIR to UP.

ramonfigueiredo and others added 4 commits July 17, 2023 18:00
Change the host hardwareState from REPAIR to UP, when the amount of free space in the /mcp directory is greater or equals to the minimum required and delete the comment associated with the host. Check if the host with REPAIR status has comments with subject=SUBJECT_COMMENT_FULL_MCP_DIR and user=CUEBOT_COMMENT_USER and delete the comments. If the comment exists, change the host hardwareState from REPAIR to UP.
(cherry picked from commit 89bdecc11b236745dbea33cf7ec83c25eaa83226)
- Set the host state to REPAIR, after create the comment about minimum bookable free MCP
- Fix freeMCP logic
- Set the host state to UP if host.hardwareState == HardwareState.REPAIR && freeMcp >= minBookableFreeMCP and use return
- Fix unit tests to the Opencue open source code
- Set dispatcher.min_bookable_free_mcp_kb=-1 (default = -1 = deactivated)
@DiegoTavares
Copy link
Collaborator

@bcipriano as I was part of the development of this feature, I think I'm biased to review it.

- Set dispatcher.min_bookable_free_mcp_kb=1048576 on cuebot/src/test/resources
- Fix tests: testHandleHostReportWithFullMCPDirectories() and testHandleHostReportWithHardwareStateRepairNotRelatedToFullMCPdirectories()
- Fix tests in the HostReportHandlerTests class
private CommentManager commentManager;
// Comment constants
private static final String SUBJECT_COMMENT_FULL_MCP_DIR = "Host set to REPAIR for not having enough storage " +
"space on /mcp";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's my understanding that for most users, the thing we call "MCP" will not actually be the /mcp directory:

def getTempPath(self):
"""Returns the correct mcp path for the given machine"""
if os.path.isdir("/mcp/"):
return "/mcp/"
return '%s/' % tempfile.gettempdir()

Ideally we would update all of OpenCue to use more fitting terminology, but that's a big project and not something we need to do right now.

However, in things like user-readable strings, and code comments, is there a different term we can use? This line is one example but there are a few others in this PR.

The current string would seem to be a bit misleading, if users have RQD hosts that aren't actually using the literal /mcp directory.

Copy link
Collaborator Author

@ramonfigueiredo ramonfigueiredo Jul 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @bcipriano

I propose we use the term TempDir to replace Mcp. What do you think?

I will replace all the Mcp by TempDir in my pull resquest.

However, as you said, we can create a new task later on to update all of OpenCue code to fit the terminology.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

I changed the comments, variables and methods reference from "mcp" to "temporary directory" or "TempDir".

See commit: 04ec106

private void changeHardwareState(DispatchHost host,
HardwareState reportState, boolean isBoot) {
private void changeHardwareState(DispatchHost host, HardwareState reportState, boolean isBoot, long freeMcp,
Long minBookableFreeMCP) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we call env.getRequiredProperty("dispatcher.min_bookable_free_mcp_kb", Long.class) to fetch this value instead of passing it as a function parameter?

Fetching properties should be cheap.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Brian.

I updated the code following you recommendation. Now the solution is calling env.getRequiredProperty("dispatcher.min_bookable_free_mcp_kb", Long.class) twice. Inside the changeHardwareState() method and inside handleHostReport() method.

Please note that currently the only place the changeHardwareState() method is called is in the handleHostReport() method. My previous implementation did only one call of env.getRequiredProperty("dispatcher.min_bookable_free_mcp_kb", Long.class) because the minBookableFreeMCP variable was used later on in the handleHostReport() method to log the message of minimum amount of free space.

  • Use of minBookableFreeMCP latter in the handleHostReport() method:
if (minBookableFreeMCP != -1 && report.getHost().getFreeMcp() < minBookableFreeMCP) {
                msg = String.format("%s doens't have enough free space in the /mcp directory, %dMB needs %dMB",
                        host.name, (report.getHost().getFreeMcp()/1024),  (minBookableFreeMCP/1024));
            }

However, I understand your point of view that the changeHardwareState method should know the min_bookable_free_mcp_kb value and that etching properties should be cheap.

- Call env.getRequiredProperty("dispatcher.min_bookable_free_mcp_kb", Long.class) inside the method changeHardwareState() to fetch this value instead of passing it as a function parameter. Fetching properties should be cheap.
- Code refactoring to log messages when the host is not bookable
- Code refactoring
- Change comments, variables and methods reference from "mcp" to "temporary directory" or "TempDir"
- Changing tests to use .setFreeMcp(CueUtil.GB) and .setTotalMcp(CueUtil.GB4)
- AbstractTreeWidget.py > _removeItem(): Check if object ID exists before delete it
Copy link
Collaborator

@bcipriano bcipriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change LGTM. Thank you for the detailed tests!

Let's just do a merge from master so we get some fresh test runs.

@ramonfigueiredo
Copy link
Collaborator Author

Hi @bcipriano,

I need write access to merge the code. Can you give me write access?

I am seeing the message below and I can't see the merge button.
Only those with write access to this repository can merge pull requests.

@bcipriano
Copy link
Collaborator

Sorry to clarify, I meant you should update this PR's branch by pulling in the latest changes from master. Once you push the updated branch it will trigger the tests to run again.

@bcipriano bcipriano merged commit b606a59 into AcademySoftwareFoundation:master Sep 27, 2023
11 checks passed
@ramonfigueiredo ramonfigueiredo deleted the minimum_bookable_free_mcp branch September 27, 2023 22:45
@ramonfigueiredo ramonfigueiredo restored the minimum_bookable_free_mcp branch September 27, 2023 22:51
@ramonfigueiredo ramonfigueiredo deleted the minimum_bookable_free_mcp branch September 29, 2023 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants