Different Mappings per namespace, get overwritten due to embedded-git-config.json

[pisani]

Take an enterprise solution consisting of 2 interoperability production, running in two different namespaces., and one git repository, (called for argument's sake - "MySolution").

The IRIS instance defines a git initialise root folder: c:\MySolution.
The IRIS instance defines namespaces APPA and APPB where these productions are developed, and I want code edited/saved in each namespace to go to a namespace appropriate sub-folder.

The export mappings as therefore defined in Settings UI say,

• for APPA, that CLS files export to /APPA/src/cls, INC to /APPA/src/inc, etc.
• for APPB, that CLS files export to /APPB/src/cls, INC to /APPB/src/inc, etc. - you get he idea...

This works fine - initially...., but changing these settings via the UI creates an export file: embedded-git-config.json in the ROOT work folder, with the most recently updated namespace values (which could be APPA or APPB), and re-entering into the other namespace's Settings UI, loads and overwrites these previously defined namespace-specific mappings.

What is the use of this embedded-git-config.json ? Am I forced to a one-to-one mapping of GIT Repository to Namespace ?

[isc-pbarton]

For now, the way to do this is to add embedded-git-config.json to the .gitignore file for the repository. You'll then be able to manage the mapping configuration separately in each namespace.

We do have an improvement on the roadmap to allow namespace-specific mappings in the configuration file: #710

[pisani]

I couldn't see how adding embedded-git-config.json to .gitignore allow me to manage the mapping configurations seperately for each namespace, but I tried this anyway (and it fails).

The process:
when you pull up the Settings (say for APPA), and make a small change and Save the settings - a default green bar at the top of the settings UI windows says settings have been exported to embedded-git-config.json.
When you then switch the other namespace (eg APPB), pull up its settings and make some minor change, and Save - the green bar says:

" embedded-git-config.GSC has been imported from C:\MySolution\embedded-git-config.json exporting new version of embedded-git-config.GSC to C:\MySolution\embedded-git-config.json "

The first part is tell-tale clue to the issue. It seems, on every SAVE, the settings are first imported from file. This is unrelated to whether the file name is added to .gitignore or not.

[pisani]

to generate a namespace-specific embedded-git-config.json file I made these changes to SourceControl.Git.Settings.Document:

Parameter INTERNALNAME = {"embedded-git-config-"_$namespace_".GSC"};
Parameter EXTERNALNAME = {"embedded-git-config-"_$namespace_".json"};

This embeds the namespace as part of the filename, and are kep seperate. Is this a valid solution to the namespace-specific config file that is worthy of a pull request ?

[isc-pbarton]

Apologies, I misunderstood your use case somewhat. I was assuming that each namespace had a separate "local" git repository but shared the same "remote". In that case you would want to add the settings file to the .gitignore so pushing on one namespace and pulling on the other will not overwrite the mapping configuration.

For backwards compatibility we'd rather not change the filename of the settings file. I would suggest instead adding a namespace filter as another field in the mapping configuration. We would absolutely welcome a PR for that (though I can prioritize doing it myself since you are not the first person to ask).

[pisani]

I could make it that when the settings are loaded from JSON, if there are not defined under a {namespace:..{..},} element, they will be loaded as defined (this way existing users on old versions of the file continue). If there are nanespace-specifix element, those will be loaded.

Then when saved, the settings file will be exported under the namespace specific element, this automatically 'upgrading' the format of this file, whose name stays the same.

Will this approach work ? (Sorry, I did not understand your suggestion of ' adding a namespace filter as a other field in the mapping configuration'. And that might be a simpler/quicker approach you might be better off doing?)

[isc-pbarton]

That's very similar to what I was getting at (though put much more clearly than I did, thank you).

There's another layer of backwards compatibility that we want: say I have an existing system with "DEV" and "PROD" namespaces cloning the same repository. Both namespaces should have the same mapping configuration. So I don't want all my mappings to be upgraded to namespace-specific if I save them. Instead, there should be an input on the settings page specifying the namespace for each mapping. By default the value will be * so it applies to any namespace. That's what I meant by adding a namespace filter.

[pisani]

I'm struggling for a 'clean' implementation approach for all the use cases as I'm sure you are aware.
So far it seems we need the following:

1. namespace(s) in IRIS that updates 1 GIT repository and use exactly the same embedded-git-config.json settings: (pullEventClass, percentClassReplace, itemMappings and decomposeProduction).

2. namespace(s) in IRIS that updates 1 GIT repository, but defines different embedded-git-config.json settings (especially itemMappings specific /src paths) - used when a solutions is made up of different parts running in different namespaces

3. one namespace in IRIS that updates multiple GIT repositories. this would require multiple GIT work folders (not just one 'root' work folder path), and specific mappings that includes the root path

** Ignoring number 3. as less likely scenario...

I believe the problem was introduced with the introduction of embedded-git-config.json, because, throughout git-source-control, ALL settings are namespace-specific and set in the ^SYS("SourceControl". The introduction of this file, however - generated and exported by the settings from the most recently modified namespace, make no distinction of the source, only to re-load them into another namespace if you switched to work somewhere else - then namespace setting placed in this file bleed and get copied around which is very bad.

Whichever way I look at this - I can't expect the the namespace global ^SYS("SourceControl","Git") to be different as it is between namespace carrying different settings, and still generate/export/import a non-namespace specific version of git settings everytime I switch namespaces. To make matters worse embedded-git-config.json only records SOME (not ALL) of the setting properties in ^SYS("SourceContro","Git"..) so hence I end up with a mix of namespace-specific and namespace non-specific settings.

I'm increasingly feeling there needs to be a wholesale re-thinkg of GIT Settings in order to be make this tool usefull for a wide set of scenarios. Prior to that though... -

I propose the following change:

(a) that embedded-git-config.json always export namespace specific section, and latest update in a non-specific group of settings
(b) that when populating on import, only namespace-specific settings will be used, if found, else, the non-specific group of settings
(c) a button on the SettingUI "Force Import of Generic settings", could force the import of the non-namespace specific section.

proposed new file layout to be generated on export: showing the generic, always-updated settings with most recent updates, first, then specific section last.

{
"pullEventClass":"SourceControl.Git.PullEventHandler.Default",
"percentClassReplace":"",
"Mappings":{
"CLS":{
"": { "directory":"cls/" },
"UnitTest": { "directory":"test/" }
},
"DFI": {
"": { "directory":"dfi/" }
},
"INC": {
"": { "directory":"inc/" }
},
"MAC": {
"": { "directory":"rtn/" }
},
"PTD": {
"": { "directory":"ptd/" }
}
},
"decomposeProductions":false
},
[
{
namespace: "IRISAPP1",
settings: {
"pullEventClass":"SourceControl.Git.PullEventHandler.Default",
"percentClassReplace":"",
"Mappings":{
"CLS":{
"": { "directory":"cls/" },
"UnitTest":{ "directory":"test/" }
},
"DFI":{
"": { "directory":"dfi/" }
},
"INC": {
"": { "directory":"inc/" }
},
"MAC": {
"": { "directory":"rtn/" }
},
"PTD": {
"": { "directory":"ptd/" }
}
},
"decomposeProductions":false
}
},
{
namespace: "IRISAPP2",
settings: {
...
}
}

]

Would be good for this to be reviewed before I implement though.

Steve

[isc-pbarton]

Hi Steve,
@isc-tleavitt and I discussed this yesterday and put some notes on the issue #215. The gist of it is that we want to start off by supporting parameter expansion of the token in mapping configurations. So your embedded-git-config.json file could look like this:

{
   "Mappings":{
     "CLS":{
       "*":{
        "directory":"<namespace>/cls/"
       },
}

Classes specific to IRISAPP1 would be exported to IRISAPP1/cls/... and classes specific to IRISAPP2 would be exported to IRISAPP2/cls/...

I just pushed up a PR that implements this: #762
Would that work for the scenarios you are thinking of? Also happy to chat offline if this could benefit from some more conversation.

[pisani]

Hi.

Yes, for this 'monorepo' situation, including the namespace in the path is key, and it is what I'm doing, except in hard coding it:

{
"Mappings":{
"CLS":{
"*":{
"directory":"IRISAPP1/cls/"
},
}

Not using placeholder

{
"Mappings":{
"CLS":{
"*":{
"directory":"/cls/"
},
}

And because I'm hard coding, and, the embedded-git-settings are reloaded in every namespace, this poses a problem.

Yes - using a placeholder in the path that resolves to namespace at the time of export, will work but note that you still have pullEventClass, percentClassReplace decomposeProduction - that will have to be the same for all namespaces (and it's why I did not suggest this option originally as I thought we where heading for total independence of settings per namespace, not just item mappings).

For me those other parameters are the same, so your PR will resolve my situation.

I'll try and find a time to chat and will email you separately. (I can think of other placeholders sorry from, that could be useful).

[pisani]

Hi -

In a shared-developer environment - are developers intended to have their own NAMESPACE to work in ? if so - then this change, will make working in a dev's namespace export to the workfolder /<devNamespace/src/... which is not what we want ?

If the idea of shared dev, is everyone logs into in the same namespace to work, then, this change will be ok for the 'mono-repo' approach - but then - when different developers log into VSCode to work in this common namespace, there is no Warning that they are stepping on other people's developement and someone may commit someone else's work in progress by mistake see issue #765