Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More configuration flexibility in InputDataPolicy #6820

Open
panta-123 opened this issue Feb 16, 2023 · 1 comment
Open

More configuration flexibility in InputDataPolicy #6820

panta-123 opened this issue Feb 16, 2023 · 1 comment

Comments

@panta-123
Copy link

Some additional flexibility will be good to add.

  1. Choose InputDataResolution policy according to jobType specified in CS.
    -> Lets say i want only User job to use InputDataByProtocol and other job to use DownloadInputData

  2. If there is no locol protcol that we want to use for streaming fallback to download policy.
    -> Curently for streaming p[rotocols we can specify Local and Remote protocols as:

     # List of protocols to use for streaming
    Protocols
    {
      # This list is used if the we are getting a file from a
      # StorageElement local to the site we are running on
      Local = file, xroot, root
      # This list is used if the SE is not local
      Remote = xroot, root
    }
    

    Lets say if the Remote part is empty and no local protocol available, then use the default policy.(i.e. DownloadInputData)

@chaen
Copy link
Contributor

chaen commented Feb 16, 2023

These are nice ideas, and we welcome PR !
The input data resolution being based on plugins, it should not be too difficult for you to extend. You should find a good starting point here https://dirac.readthedocs.io/en/latest/AdministratorGuide/Systems/WorkloadManagement/InputDataResolution.html?highlight=inputdatapolicy
Also, what you offer seems generic enough, and would probably benefit to other communities, so please consider implementing this in vanilla DIRAC.
Let us know if you need more guidance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants