Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modifying existing resources crashes with Puppet 7 #261

Open
supertassu opened this issue Dec 1, 2022 · 2 comments
Open

Modifying existing resources crashes with Puppet 7 #261

supertassu opened this issue Dec 1, 2022 · 2 comments

Comments

@supertassu
Copy link

supertassu commented Dec 1, 2022 •
edited
Loading

Description of problem

  • What did you do? Made a change to an existing resource in a manifest, and used octocatalog-diff to compile the changes for a host where that manifest is applied.
  • What happened? I get an error, /usr/lib/ruby/vendor_ruby/octocatalog-diff/catalog-diff/filter/compilation_dir.rb:72:in new': tried to create Proc object without a block (ArgumentError)`
  • What did you expect to happen? I see the difference in the catalog
  • How can someone reproduce the problem?

Command used and debugging output

taavi@runko:~/src/jquery/infrastructure-puppet production(+1/-1) ± octocatalog-diff --environment production -n codeorigin-02.ops.jquery.net --debug
D, [2022-12-01T20:44:42.844628 #845048] DEBUG -- : Running octocatalog-diff 2.1.0-1 with ruby 3.0.4
D, [2022-12-01T20:44:42.844666 #845048] DEBUG -- : Command line arguments: ["--environment", "production", "-n", "codeorigin-02.ops.jquery.net", "--debug"]
D, [2022-12-01T20:44:42.844676 #845048] DEBUG -- : Running on host runko (x86_64-linux-gnu)
D, [2022-12-01T20:44:42.844688 #845048] DEBUG -- : Compiling catalogs for codeorigin-02.ops.jquery.net
D, [2022-12-01T20:44:42.844757 #845048] DEBUG -- : Initialized OctocatalogDiff::Catalog::Computed for from-catalog
D, [2022-12-01T20:44:42.844767 #845048] DEBUG -- : Initialized OctocatalogDiff::Catalog::Computed for to-catalog
D, [2022-12-01T20:44:42.844784 #845048] DEBUG -- : Initialized parallel task result array: size=2
D, [2022-12-01T20:44:42.845559 #845048] DEBUG -- : Launched pid=845049 for index=0
D, [2022-12-01T20:44:42.845790 #845049] DEBUG -- : Begin build_catalog for origin/staging
D, [2022-12-01T20:44:42.845870 #845049] DEBUG -- : Setting up Puppet catalog build for origin/staging
D, [2022-12-01T20:44:42.845895 #845049] DEBUG -- : Catalog for origin/staging will be built with OctocatalogDiff::Catalog::Computed
D, [2022-12-01T20:44:42.845941 #845049] DEBUG -- : Calling build for object OctocatalogDiff::Catalog::Computed
D, [2022-12-01T20:44:42.846011 #845049] DEBUG -- : Start retrieving facts for codeorigin-02.ops.jquery.net from OctocatalogDiff::Catalog::Computed
D, [2022-12-01T20:44:42.846042 #845049] DEBUG -- : Retrieving facts from PuppetDB
D, [2022-12-01T20:44:42.846220 #845048] DEBUG -- : Launched pid=845050 for index=1
D, [2022-12-01T20:44:42.846672 #845050] DEBUG -- : Begin build_catalog for .
D, [2022-12-01T20:44:42.846802 #845050] DEBUG -- : Setting up Puppet catalog build for .
D, [2022-12-01T20:44:42.846825 #845050] DEBUG -- : Catalog for . will be built with OctocatalogDiff::Catalog::Computed
D, [2022-12-01T20:44:42.846870 #845050] DEBUG -- : Calling build for object OctocatalogDiff::Catalog::Computed
D, [2022-12-01T20:44:42.846968 #845050] DEBUG -- : Start retrieving facts for codeorigin-02.ops.jquery.net from OctocatalogDiff::Catalog::Computed
D, [2022-12-01T20:44:42.847014 #845050] DEBUG -- : Retrieving facts from PuppetDB
D, [2022-12-01T20:44:43.468163 #845050] DEBUG -- : Success retrieving facts for codeorigin-02.ops.jquery.net from OctocatalogDiff::Catalog::Computed
D, [2022-12-01T20:44:43.468565 #845050] DEBUG -- : Symlinked /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/environments/production -> /home/taavi/src/jquery/infrastructure-puppet
D, [2022-12-01T20:44:43.469546 #845050] DEBUG -- : Installed hiera.yaml from /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/environments/production/test_data/hiera.yaml to /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/hiera.yaml
D, [2022-12-01T20:44:43.473271 #845050] DEBUG -- : Installed fact file at /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/var/yaml/facts/codeorigin-02.ops.jquery.net.yaml
D, [2022-12-01T20:44:43.473641 #845050] DEBUG -- : Installed CA certificate in /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/var/ssl/certs/ca.pem
D, [2022-12-01T20:44:43.489164 #845049] DEBUG -- : Success retrieving facts for codeorigin-02.ops.jquery.net from OctocatalogDiff::Catalog::Computed
D, [2022-12-01T20:44:43.489303 #845049] DEBUG -- : Begin git checkout /home/taavi/src/jquery/infrastructure-puppet:origin/staging -> /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-bootstrap-checkout-20221201-845049-9mfo6l
D, [2022-12-01T20:44:43.495187 #845049] DEBUG -- : ["Exit status: 0"]
D, [2022-12-01T20:44:43.495247 #845049] DEBUG -- : Success git archive /home/taavi/src/jquery/infrastructure-puppet:origin/staging
D, [2022-12-01T20:44:43.495261 #845049] DEBUG -- : Success git checkout /home/taavi/src/jquery/infrastructure-puppet:origin/staging -> /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-bootstrap-checkout-20221201-845049-9mfo6l
D, [2022-12-01T20:44:43.495291 #845049] DEBUG -- : Begin install bootstrap script in target directory
D, [2022-12-01T20:44:43.495468 #845049] DEBUG -- : Success: copied /home/taavi/src/jquery/infrastructure-puppet/test_data/bootstrap.sh to /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-bootstrap-checkout-20221201-845049-9mfo6l/test_data/bootstrap.sh
D, [2022-12-01T20:44:43.495487 #845049] DEBUG -- : Begin bootstrap with 'test_data/bootstrap.sh' in /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-bootstrap-checkout-20221201-845049-9mfo6l
D, [2022-12-01T20:44:43.505189 #845049] DEBUG -- : Success bootstrap in /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-bootstrap-checkout-20221201-845049-9mfo6l
D, [2022-12-01T20:44:43.505482 #845049] DEBUG -- : Symlinked /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/environments/production -> /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-bootstrap-checkout-20221201-845049-9mfo6l
D, [2022-12-01T20:44:43.506695 #845049] DEBUG -- : Installed hiera.yaml from /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/environments/production/test_data/hiera.yaml to /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/hiera.yaml
D, [2022-12-01T20:44:43.510244 #845049] DEBUG -- : Installed fact file at /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/var/yaml/facts/codeorigin-02.ops.jquery.net.yaml
D, [2022-12-01T20:44:43.510558 #845049] DEBUG -- : Installed CA certificate in /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/var/ssl/certs/ca.pem
D, [2022-12-01T20:44:43.994814 #845050] DEBUG -- : (to) Try 1 executing Puppet 7.20.0: /usr/bin/puppet catalog compile codeorigin-02.ops.jquery.net --no-storeconfigs --factpath=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/var/yaml/facts --facts_terminus=yaml --no-daemonize --color=false --environment=production --hiera_config=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/hiera.yaml --environmentpath=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/environments --vardir=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/var --logdir=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/var --ssldir=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/var/ssl --confdir=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l
D, [2022-12-01T20:44:43.995072 #845050] DEBUG -- : ["Execute: /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-scriptrunner20221201-845050-l867uc/puppet.sh catalog compile codeorigin-02.ops.jquery.net --no-storeconfigs --factpath\\=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/var/yaml/facts --facts_terminus\\=yaml --no-daemonize --color\\=false --environment\\=production --hiera_config\\=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/hiera.yaml --environmentpath\\=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/environments --vardir\\=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/var --logdir\\=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/var --ssldir\\=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l/var/ssl --confdir\\=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845050-9mfo6l"]
D, [2022-12-01T20:44:44.012554 #845049] DEBUG -- : (from) Try 1 executing Puppet 7.20.0: /usr/bin/puppet catalog compile codeorigin-02.ops.jquery.net --no-storeconfigs --factpath=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/var/yaml/facts --facts_terminus=yaml --no-daemonize --color=false --environment=production --hiera_config=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/hiera.yaml --environmentpath=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/environments --vardir=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/var --logdir=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/var --ssldir=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/var/ssl --confdir=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc
D, [2022-12-01T20:44:44.012826 #845049] DEBUG -- : ["Execute: /tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-scriptrunner20221201-845049-o6vqzm/puppet.sh catalog compile codeorigin-02.ops.jquery.net --no-storeconfigs --factpath\\=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/var/yaml/facts --facts_terminus\\=yaml --no-daemonize --color\\=false --environment\\=production --hiera_config\\=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/hiera.yaml --environmentpath\\=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/environments --vardir\\=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/var --logdir\\=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/var --ssldir\\=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc/var/ssl --confdir\\=/tmp/ocd-ipc-20221201-845048-fx3dm5/ocd-builddir-20221201-845049-l867uc"]
D, [2022-12-01T20:44:45.644370 #845050] DEBUG -- : ["Exit status: 0"]
D, [2022-12-01T20:44:45.644442 #845050] DEBUG -- : (to) Catalog succeeded on try 1 in 2.170742998 seconds
D, [2022-12-01T20:44:45.648713 #845050] DEBUG -- : Catalog for . successfully built with OctocatalogDiff::Catalog::Computed in 2.801828166 seconds
D, [2022-12-01T20:44:45.648798 #845050] DEBUG -- : Success build_catalog for .
D, [2022-12-01T20:44:45.659457 #845048] DEBUG -- : PID=845050 completed in 2.813199822 seconds, 358893 bytes
D, [2022-12-01T20:44:45.693843 #845049] DEBUG -- : ["Exit status: 0"]
D, [2022-12-01T20:44:45.693912 #845049] DEBUG -- : (from) Catalog succeeded on try 1 in 2.183305034 seconds
D, [2022-12-01T20:44:45.696800 #845049] DEBUG -- : Catalog for origin/staging successfully built with OctocatalogDiff::Catalog::Computed in 2.850866101 seconds
D, [2022-12-01T20:44:45.696839 #845049] DEBUG -- : Success build_catalog for origin/staging
D, [2022-12-01T20:44:45.704691 #845048] DEBUG -- : PID=845049 completed in 2.859106061 seconds, 358796 bytes
D, [2022-12-01T20:44:45.704718 #845048] DEBUG -- : All child processes completed with no exceptions raised
I, [2022-12-01T20:44:45.704779 #845048]  INFO -- : Catalogs compiled for codeorigin-02.ops.jquery.net
D, [2022-12-01T20:44:45.704794 #845048] DEBUG -- : Begin compute diffs between catalogs
D, [2022-12-01T20:44:45.704858 #845048] DEBUG -- : Entering catdiff; catalog sizes: 204, 204
D, [2022-12-01T20:44:45.704867 #845048] DEBUG -- : Entering preprocess_diff; catalog sizes: 204, 204
D, [2022-12-01T20:44:45.706920 #845048] DEBUG -- : Exiting preprocess_diff; added 0, removed 0
D, [2022-12-01T20:44:45.706935 #845048] DEBUG -- : Entering hashdiff_initial; catalog sizes: 204, 204
D, [2022-12-01T20:44:45.706944 #845048] DEBUG -- : HashDiff configuration: (use_lcs: true)
D, [2022-12-01T20:44:45.742015 #845048] DEBUG -- : Exiting hashdiff_initial; changes: 3, nested changes: 1
/usr/lib/ruby/vendor_ruby/octocatalog-diff/catalog-diff/filter/compilation_dir.rb:72:in `new': tried to create Proc object without a block (ArgumentError)
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/catalog-diff/filter/compilation_dir.rb:72:in `block in traverse'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/catalog-diff/filter/compilation_dir.rb:72:in `map'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/catalog-diff/filter/compilation_dir.rb:72:in `traverse'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/catalog-diff/filter/compilation_dir.rb:63:in `remove_compilation_dir'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/catalog-diff/filter/compilation_dir.rb:40:in `filtered?'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/catalog-diff/filter.rb:62:in `block in filter'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/catalog-diff/filter.rb:62:in `reject!'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/catalog-diff/filter.rb:62:in `filter'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/catalog-diff/filter.rb:42:in `block in apply_filters'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/catalog-diff/filter.rb:42:in `each'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/catalog-diff/filter.rb:42:in `apply_filters'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/catalog-diff/differ.rb:207:in `catdiff'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/catalog-diff/differ.rb:83:in `diff'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/cli/diffs.rb:34:in `diffs'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/api/v1/catalog-diff.rb:54:in `catalog_diff'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/api/v1.rb:19:in `catalog_diff'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/cli.rb:152:in `run_octocatalog_diff'
	from /usr/lib/ruby/vendor_ruby/octocatalog-diff/cli.rb:126:in `cli'
	from /usr/bin/octocatalog-diff:34:in `<main>'

Platform and version information

  • Your OS: Debian GNU/Linux, running the testing release
  • Your Ruby version: ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux-gnu]
  • Your version of Puppet: 7.20.0, the open source version as packaged in Debian
  • Your version of octocatalog-diff: octocatalog-diff 2.1.0-1, again installed from the packages in Debian repositories

Do the tests pass from a clean checkout?

No,

$ bundle exec rake
rake aborted!
SyntaxError: /home/taavi/src/octocatalog-diff/vendor/bundle/ruby/3.0.0/gems/mime-types-3.2.2/lib/mime/types/logger.rb:30: _1 is reserved for numbered parameter

Anything else to add that you think will be helpful?
@supertassu
Copy link
Author

This seems to be an issue with a newer Ruby version, not with a newer Puppet version. The following patch seems to fix it, but I'm not very familiar with Ruby and therefore not sure if it's the correct or proper fix:

diff --git a/lib/octocatalog-diff/catalog-diff/filter/compilation_dir.rb b/lib/octocatalog-diff/catalog-diff/filter/compilation_dir.rb
index 5e2747d..b105f8a 100644
--- a/lib/octocatalog-diff/catalog-diff/filter/compilation_dir.rb
+++ b/lib/octocatalog-diff/catalog-diff/filter/compilation_dir.rb
@@ -66,12 +66,12 @@ module OctocatalogDiff
           value
         end
 
-        def traverse(a)
+        def traverse(a, &proc)
           case a
           when Array
-            a.map { |v| traverse(v, &Proc.new) }
+            a.map { |v| traverse(v, &proc) }
           when Hash
-            traverse(a.values, &Proc.new)
+            traverse(a.values, &proc)
           else
             yield a
           end

@mika
Copy link

mika commented Mar 24, 2023

I can confirm the problem, also seeing it with ruby 3.1.2p20 (as present in Debian/bookworm) and puppet-agent v7.23.0-1bullseye (from puppetlabs upstream).

Thanks @supertassu for providing a patch, this also works for me!

ghost pushed a commit to makandra/octocatalog-diff that referenced this issue Jan 12, 2024
Hotfix compilation_dir.rb with change from upstream github#261

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode
e1c2a9d
Krinkle added a commit to jquery/infrastructure-puppet that referenced this issue Mar 31, 2024
@Krinkle
bin: Hotpatch octocatalog-diff to not crash on Ruby 3.1.2
e24cafc 
Ref github/octocatalog-diff#261
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mika @supertassu