Unify all .md resources loading approach

[jdneo]

See: #136 (comment)

Currently we have

In the preference.

An idea is make it affect to other md files as well. For example the skills.

And we can extract it out of the custom instructions block, because it ultimately affects how the workspace folders are passed to CLS. And CLS searches those md files based on the passed folder.

[travkin79]

I'm trying to collect all the details that we need to consider and some open questions to answer.

Files to consider

• copilot-instructions.md
• *.instructions.md
• *.prompt.md
• SKILL.md
• *.agent.md
• AGENTS.md
• CLAUDE.md, GEMINI.md

Copilot scopes according to documentation

• repository-wide (files in git repository root folder or in .github/, .claude/, or .agents/ folders)
• project- / path-specific (files in a project's directory or its sub-folders)
• user-specific / local (files in ~/.copilot/, ~/.claude/, or ~/.agents/)

Scopes in copilot-for-eclipse

• instructions from all Eclipse projects in Eclipse workspace
• instructions only from Eclipse projects derived from referenced files in context (selected in Chat View)

Current state

• Scope for loading copilot-instructions.md can now be set to all (from all projects) or from referenced files when using the chat view (see Add Copilot preference for a chat's custom instructions loading (from all projects or from referenced projects only) #136)
• Scope for loading *.prompt.md and SKILL.md is always all (from all projects)
• Scope for loading *.agent.md is always all (from all projects)
• The scope settings also apply to the search (by limiting the set of workspace folders passed to CLS)

Questions

• Which of the listed files need scoping and which should always be loaded? (E.g. always load all user-specific instructions/skills?)
• Which scope settings do we need in copilot-for-eclipse?
• Do we need a separate scope setting for the search feature (e.g. loading instructions from a limited scope, but search in all projects in the Eclipse workspace)? Is that even possible or would that require changes on LS-side?
• Where should we place the new / adapted preferences (which preferences page)?
• Does the CLS load user-specific and/or repository-wide instructions or should loading them be a client-side feature?

Considerations
As a user, I'd expect copilot to

• always load all user-specific instructions --> no scoping here or only a switch for turning user-specific instructions on and off via Eclipse preferences
• always load repository-wide instructions from the root directory, since they apply to whole git repository -> no scoping here or only a switch for turning repository-wide instructions on and off
• decide based on a user's preferences whether to load all path-specific instructions or only those derived from files in the context (in the chat view). This is exactly what we introduced in PR Add Copilot preference for a chat's custom instructions loading (from all projects or from referenced projects only) #136 for copilot-instructions.md files.

In this case, we could stay with the two scopes all and derived from referenced files, I guess. Optionally, we could introduce two additional switches in the preferences for additionally loading or for not loading user-specific and repository-wide instructions.

What do you think, @jdneo and @iloveeclipse? Did I miss some file type, scope, special case?

References

• Custom instructions in VSCode: https://docs.github.com/en/copilot/concepts/prompting/response-customization?tool=vscode
• Custom instructions in Copilot on GitHub: https://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/add-custom-instructions/add-repository-instructions
• Custom instructions in GitHub Copilot CLI: https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/add-custom-instructions#types-of-custom-instructions

[iloveeclipse]

In this case, we could stay with the two scopes all and derived from referenced files, I guess. Optionally, we could introduce two additional switches in the preferences for additionally loading or for not loading user-specific and repository-wide instructions.

Regarding user specific instructions: note, "workspace" instructions are also there:

Here is the list of scopes I'm aware of:

1. "User" scope: files from ~/.copilot etc
2. "Git repo" scope: files from git repository root
3. "Workspace" scope: settings as shown above (not sure why do we have them)
4. "Project" scope: projects from referenced files in context or all projects

IMO, ideally, the files should be always loaded based on current scope, from smallest to highest one, based on reachability.

My real SDK workspace has N projects from M repositories. Repositories I have are from different organizations, contain projects of fully different nature and with fiully diferent toolchains. Makes no sense to load maven related instructions or skills into fully unrelated bazel based project etc, or instructions from open source repo licensed under GPL into closed source repo licensed under proprietary license etc.

So let see the example. If I have foo.txt file in context from project "A" from repo "B", I expect following to be loaded:

1. Settings from "A" project
2. Git repo files from "B" git repository root
3. Workspace settings as shown above (not sure why do we have them)
4. My user files from ~/.copilot etc

If loading order is important, the "smallest" scope should always "win" and be more important as "wider" scopes.

From the preferences point of view it would be:

Custom files (skills, instructions, agents) loading

• Load files from projects referenced in context (optionally: from all projects)
• Load files from repositories referenced in context (optionally: from all repos)
• Load files from referenced by workspace instructions
• Load files from user home directory

[travkin79]

Thank you @iloveeclipse for your feedback. I agree with that, I'm just not sure about the suggested preferences.

How about the following preference options, i.e. either use all for project-specific and repository-wide instructions or derived for both?

• Load instructions from (choice via drop-down list)
  • all projects in workspace and corresponding repositories
  • projects and repositories derived from files in context (derived from chat-attached files)
• [ ] Load workspace instructions (checkbox)
• [ ] Load user-specific instructions (checkbox)

[jdneo]

Hi all, I was busy with preparing the token based billing support, will discuss this topic after it's done.

[jdneo]

I hope to take this into consideration: #24

The root cause of it is:

The CLS will spawn a new child process when doing search for each of the workspace folder, and the max number is 20.

But for a search tool, it is unnecessary to pass each submodule as a workspace folder. Just the top level project is enough.

The option all projects in workspace and corresponding repositories still breaks this case because it adds more items, not reduce the items.

How about:

1. all project and repos
2. all repos
3. projects and repos derived from files in context

[iloveeclipse]

How about:

1. all project and repos
2. all repos
3. projects and repos derived from files in context

This sounds OK and would be either/or choice (radio boxes).
What is about workspace / user instructions? They could be additionally enabled/disabled as checkboxes?

[jdneo]

What is about workspace / user instructions? They could be additionally enabled/disabled as checkboxes?

I need to check how CLS handle this...

[travkin79]

Hi @jdneo,
Did you have some time to check the CLS code?