[BUG] Defining saltenv options in gitfs_remotes are not rendered correctly.

[bennodepenno]

Your setup

Formula commit hash / release tag

f10a85f

Versions reports (master & minion)

Salt Version:
Salt: 3000.3

Dependency Versions:
cffi: 1.6.0
cherrypy: Not Installed
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 2.7.2
libgit2: 0.26.3
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.6.2
mysql-python: Not Installed
pycparser: 2.14
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: 0.26.4
Python: 2.7.5 (default, Apr 2 2020, 13:16:51)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 15.3.0
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.1.4

System Versions:
dist: centos 7.8.2003 Core
locale: ANSI_X3.4-1968
machine: x86_64
release: 3.10.0-1127.8.2.el7.x86_64
system: Linux
version: CentOS Linux 7.8.2003 Core

Pillar / config used

salt:
  release: '3000.3'
  master_remove_config: True
  master:
    interface: 0.0.0.0
    worker_threads: 15
    enable_gpu_grains: True
    autosign_file: /etc/salt/autosign.conf
    minion_data_cache: True
    fileserver_backend:
      - git
      - roots
    file_roots:
      base:
        - /srv/uv/salt
        - /srv/uv/formulas
    pillar_roots:
      base:
        - /srv/uv/pillar
    ext_pillar:
      - git:
        - develop https://myurl/pillar.git:
          - password: 'secret'
          - user: 'saltuser'
          - env: uv
          - root: uv
    gitfs_saltenv_whitelist:
      - base
    gitfs_update_interval: 60
    gitfs_provider: pygit2
    gitfs_insecure_auth: True
    git_pillar_insecure_auth: True
    file_client: local
    gitfs_remotes:
      - https://myurl/salt.git:
        - user: 'saltuser'
        - password: 'secret'
        - mountpoint: salt://
        - root: uv
        - saltenv:
          - uv:
            - ref: develop
      - https://myurl/formulas/bind-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: bind
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://bind
      - https://myurl/formulas/dhcpd-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: dhcpd
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://dhcpd
      - https://myurl/formulas/salt-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: salt
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://salt
      - https://myurl/formulas/users-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: users
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://users
      - https://myurl/formulas/sudoers-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: sudoers
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://sudoers
      - https://myurl/formulas/grafana-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: grafana
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://grafana
      - https://myurl/formulas/rsyslog-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: rsyslog
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://rsyslog
      - https://myurl/formulas/prometheus-formula.git:
        - user: 'saltuser'
        - password: 'secret'
        - root: prometheus
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://prometheus
    decrypt_pillar:
      - 'certificates:vault'
    decrypt_pillar_delimiter: '|'
    use_superseded:
      - module.run
    external_auth:
      pam:
        saltuser:
          - '@runner'
        foreman:
          - '@runner'
          - '@wheel'
        foreman-proxy:
          - '@runner'
          - '@wheel'

    reactors:
      - 'master/deploy':
        - /srv/salt/reactors/deploy.sls

---

Bug details

Describe the bug

Hi,

I'm having issues using the salt-formula when using gitfs_remotes. In my pillar, I've specified:

gitfs_remotes:
      - https://myurl/formulas/bind-formula.git:
        - user: 'saltuser@myurl'
        - password: 'secret'
        - root: bind
        - saltenv:
          - uv:
            - ref: master
            - mountpoint: salt://bind

When rendered, the output shows:

gitfs_remotes:
    - https://myurl/formulas/bind-formula.git:
      - user: saltuser
      - password: secret
      - root: bind
      - saltenv: [{u'uv': [{u'ref': u'master'}, {u'mountpoint': u'salt://bind'}]}]

It's this line which is incorrect: [{u'uv': [{u'ref': u'master'}, {u'mountpoint': u'salt://bind'}]}].

Steps to reproduce the bug

Use the pillar set and execute the salt formula on target host. The salt run will succeed but the resulting file /etc/salt/master.d/f_defaults.conf contains the incorrect configuration.

Expected behaviour

gitfs_remotes:
    - https://myurl/formulas/bind-formula.git:
      - user: saltuser
      - password: secret
      - root: bind
      - saltenv:
         - uv:
           - ref: master
           - mountpoint: salt://bind

Attempts to fix the bug

I've attempted some suggestions to use "{ -uv: }" and "{[ -uv ]}" without luck.

Additional context

[myii]

@bennodepenno Thanks for the report. This is known problem with the pillar approach, so we're in the process of moving to providing master/minion configuration files via. TOFS instead, which was implemented in #398 (specifically starting from #398 (comment)). It's working but there are steps remaining to make it easier for users to adopt (#417). An example of helping someone getting it working has been captured around here:

• https://freenode.logbot.info/saltstack-formulas/20200320#c3444154

Hopefully, there's enough there to help you work out how to provide the configuration files via. TOFS. We were hoping for some documentation to be provided but that's not the case so far. If you need further help, feel free to start a conversation in the #formulas channel in Slack (also available via. IRC).

[bennodepenno]

Thank you for your reply. I will look into TOFS and the examples provided. Thank you for your time and effort.
I would like to participate on Slack, but I do not know how to join the #formulas channel.

[myii]

@bennodepenno This should be the correct link, please let me know if it doesn't work:

3. Please direct questions to the #formulas channel on Slack, which is bridged to #saltstack-formulas on Freenode.

[bennodepenno]

Thanks for all the help.