Merge pull request #172 from github/kpaulisse-release-branch

Release version 1.5.2

Author: kpaulisse

.version

@@ -1 +1 @@
-1.5.1
+1.5.2

doc/CHANGELOG.md

@@ -8,6 +8,16 @@
 </tr>
 </thead><tbody>
 
+<tr valign=top>
+<td>1.5.2</td>
+<td>2017-12-19</td>
+<td>
+<li><a href="https://github.com/github/octocatalog-diff/pull/169">#169</a>: (Enhancement) Puppet Enterprise RBAC token to authenticate to PuppetDB</li>
+<li><a href="https://github.com/github/octocatalog-diff/pull/170">#170</a>: (Enhancement) Filter to treat an object the same as a single array containing that object</li>
+<li><a href="https://github.com/github/octocatalog-diff/pull/165">#165</a>: (Bug Fix) Override of fact file via CLI now has precedence over value set in configuration file</li>
+</td>
+</tr>
+
 <tr valign=top>
 <td>1.5.1</td>
 <td>2017-11-16</td>

doc/advanced-filter.md

@@ -11,6 +11,7 @@ Here is the list of available filters and an explanation of each:
 
 - [Absent File](/doc/advanced-filter.md#absent-file) - Ignore parameter changes of a file that is declared to be absent
 - [JSON](/doc/advanced-filter.md#json) - Ignore whitespace differences if JSON parses to the same object
+- [SingleItemArray](/doc/advanced-filter.md#SingleItemArray) - Ignore differences between object and array containing only that object
 - [YAML](/doc/advanced-filter.md#yaml) - Ignore whitespace/comment differences if YAML parses to the same object
 
 ## Absent File
@@ -84,6 +85,28 @@ If a file resource has extension `.json` and a difference in its content is obse
 
 This allows you to ignore changes in whitespace, comments, etc., that are not meaningful to a machine parsing the file. Note that changes to files may still trigger Puppet to restart services even though these changes are not displayed in the octocatalog-diff output.
 
+## Single Item Array
+
+#### Usage
+
+```
+--filters SingleItemArray
+```
+
+#### Description
+
+When enabling the future parser or upgrading between certain versions of Puppet, the internal structure of the catalog for certain parameters can change as shown in the following example:
+
+```
+Old: { "notify": "Service[foo]" }
+New: { "notify": [ "Service[foo]" ] }
+```
+
+This filter will suppress differences for the value of a parameter when:
+
+- The value in one catalog is an object, AND
+- The value in the other catalog is an array containing *only* that same object
+
 ## YAML
 
 #### Usage

doc/configuration-puppetdb.md

@@ -36,6 +36,7 @@ The following settings can be used in a [configuration file](/doc/configuration.
 | `settings[:puppetdb_ssl_client_key]` | TEXT of the private key of the client SSL keypair used to authenticate to PuppetDB. Note: This variable is not set to a file path, which means you will likely want to use means you will likely want to use `File.read(...)` if you are configuring this to be read from a file. |
 | `settings[:puppetdb_ssl_client_pem]` | Concatenation of the text of `puppetdb_ssl_client_key` and `puppetdb_ssl_client_cert` as previously described. This is a good alternative if your certificate chain is complex and it's easier just to put everything in a single place. Note: this option is second in precedence; if `settings[:puppetdb_ssl_client_cert]` and `settings[:puppetdb_ssl_client_key]` are both set, this will be ignored. |
 | `settings[:puppetdb_ssl_client_password]` | Plain text string containing the password to unlock the private key. For keys generated by the Puppet Master CA, this is not required and should be left undefined. |
+| `settings[:puppetdb_token]` | TEXT containing the PE RBAC token used to authenticate to PuppetDB. Note: This variable is not set to a file path, which means you will likely want to use `File.read(...)` if you are configuring this to be read from a file. |
 
 ## Supplying necessary information via the command line
 
@@ -48,6 +49,8 @@ The following arguments can be used on the command line.
 | --puppetdb-ssl-client-cert FILENAME | Path to the certificate of the client SSL keypair. |
 | --puppetdb-ssl-client-key FILENAME | Path to the private key of the client SSL keypair. |
 | --puppetdb-ssl-client-password PASSWORD_STRING | Plain text string containing the password to unlock the private key. For keys generated by the Puppet Master CA, this is not required. |
+| --puppetdb-token STRING | String containing the PE RBAC token used to authenticate to PuppetDB. |
+| --puppetdb-token-file FILENAME | Path to the PE RBAC token file used to authenticate to PuppetDB. |
 
 ## Supplying necessary information via the environment
 

doc/optionsref.md

@@ -87,6 +87,8 @@ Usage: octocatalog-diff [command line options]
         --to-puppet-binary STRING    Full path to puppet binary for the to branch
         --from-puppet-binary STRING  Full path to puppet binary for the from branch
         --facts-terminus STRING      Facts terminus: one of yaml, facter
+        --puppetdb-token TOKEN       Token to access the PuppetDB API
+        --puppetdb-token-file PATH   Path containing token for PuppetDB API, relative or absolute
         --puppetdb-url URL           PuppetDB base URL
         --puppetdb-ssl-ca FILENAME   CA certificate that signed the PuppetDB certificate
         --puppetdb-ssl-client-cert FILENAME
@@ -1374,6 +1376,36 @@ the text of the password won't appear in the process list. (<a href="../lib/octo
     </td>
   </tr>
 
+  <tr>
+    <td valign=top>
+      <pre><code>--puppetdb-token TOKEN</code></pre>
+    </td>
+    <td valign=top>
+      Token to access the PuppetDB API
+    </td>
+    <td valign=top>
+      Specify the PE RBAC token to access the PuppetDB API. Refer to
+https://puppet.com/docs/pe/latest/rbac/rbac_token_auth_intro.html#generate-a-token-using-puppet-access
+for details on generating and obtaining a token. Use this option to specify the text
+of the token. (Use --puppetdb-token-file to read the content of the token from a file.) (<a href="../lib/octocatalog-diff/cli/options/puppetdb_token.rb">puppetdb_token.rb</a>)
+    </td>
+  </tr>
+
+  <tr>
+    <td valign=top>
+      <pre><code>--puppetdb-token-file PATH</code></pre>
+    </td>
+    <td valign=top>
+      Path containing token for PuppetDB API, relative or absolute
+    </td>
+    <td valign=top>
+      Specify the PE RBAC token to access the PuppetDB API. Refer to
+https://puppet.com/docs/pe/latest/rbac/rbac_token_auth_intro.html#generate-a-token-using-puppet-access
+for details on generating and obtaining a token. Use this option to specify the text
+in a file, to read the content of the token from the file. (<a href="../lib/octocatalog-diff/cli/options/puppetdb_token_file.rb">puppetdb_token_file.rb</a>)
+    </td>
+  </tr>
+
   <tr>
     <td valign=top>
       <pre><code>--puppetdb-url URL</code></pre>

lib/octocatalog-diff/catalog-diff/filter.rb

@@ -2,6 +2,7 @@
 require_relative 'filter/absent_file'
 require_relative 'filter/compilation_dir'
 require_relative 'filter/json'
+require_relative 'filter/single_item_array'
 require_relative 'filter/yaml'
 
 require 'stringio'
@@ -13,7 +14,7 @@ class Filter
       attr_accessor :logger
 
       # List the available filters here (by class name) for use in the validator method.
-      AVAILABLE_FILTERS = %w(AbsentFile CompilationDir JSON YAML).freeze
+      AVAILABLE_FILTERS = %w(AbsentFile CompilationDir JSON SingleItemArray YAML).freeze
 
       # Public: Determine whether a particular filter exists. This can be used to validate
       # a user-submitted filter.

lib/octocatalog-diff/catalog-diff/filter/single_item_array.rb

@@ -0,0 +1,44 @@
+# frozen_string_literal: true
+
+require_relative '../filter'
+
+module OctocatalogDiff
+  module CatalogDiff
+    class Filter
+      # Filter out changes in parameters when one catalog has a parameter that's an object and
+      # the other catalog has that same parameter as an array containing the same object.
+      # For example, under this filter, the following is not a change:
+      #   catalog1: notify => "Service[foo]"
+      #   catalog2: notify => ["Service[foo]"]
+      class SingleItemArray < OctocatalogDiff::CatalogDiff::Filter
+        # Public: Implement the filter for single-item arrays whose item exactly matches the
+        # item that's not in an array in the other catalog.
+        #
+        # @param diff [OctocatalogDiff::API::V1::Diff] Difference
+        # @param _options [Hash] Additional options (there are none for this filter)
+        # @return [Boolean] true if this should be filtered out, false otherwise
+        def filtered?(diff, _options = {})
+          # Skip additions or removals - focus only on changes
+          return false unless diff.change?
+          old_value = diff.old_value
+          new_value = diff.new_value
+
+          # Skip unless there is a single-item array under consideration
+          return false unless
+            (old_value.is_a?(Array) && old_value.size == 1) ||
+            (new_value.is_a?(Array) && new_value.size == 1)
+
+          # Skip if both the old value and new value are arrays
+          return false if old_value.is_a?(Array) && new_value.is_a?(Array)
+
+          # Do comparison
+          if old_value.is_a?(Array)
+            old_value.first == new_value
+          else
+            new_value.first == old_value
+          end
+        end
+      end
+    end
+  end
+end

lib/octocatalog-diff/cli/options.rb

@@ -23,7 +23,6 @@ def self.classes
 
       # Define the Option class and newoption() method for use by cli/options/*.rb files
       class Option
-        DEFAULT_WEIGHT = 999
         def self.has_weight(w) # rubocop:disable Style/PredicateName
           @weight = w
         end
@@ -38,7 +37,9 @@ def self.weight
           elsif @weight
             @weight
           else
-            DEFAULT_WEIGHT
+            # :nocov:
+            raise ArgumentError, "Option #{name} does not have a weight specified. Add 'has_weight NNN' to control ordering."
+            # :nocov:
           end
         end
 

lib/octocatalog-diff/cli/options/pe_enc_token_file.rb

@@ -12,7 +12,7 @@
   def parse(parser, options)
     parser.on('--pe-enc-token-file PATH', 'Path containing token for PE node classifier, relative or absolute') do |x|
       proposed_token_path = x.start_with?('/') ? x : File.join(options[:basedir], x)
-      raise Errno::ENOENT, "Provided token (#{proposed_token_path}) does not exist" unless File.file?(proposed_token_path)
+      raise Errno::ENOENT, "Provided PE ENC token (#{proposed_token_path}) does not exist" unless File.file?(proposed_token_path)
       options[:pe_enc_token] = File.read(proposed_token_path)
     end
   end

lib/octocatalog-diff/cli/options/puppetdb_token.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+# Specify the PE RBAC token to access the PuppetDB API. Refer to
+# https://puppet.com/docs/pe/latest/rbac/rbac_token_auth_intro.html#generate-a-token-using-puppet-access
+# for details on generating and obtaining a token. Use this option to specify the text
+# of the token. (Use --puppetdb-token-file to read the content of the token from a file.)
+# @param parser [OptionParser object] The OptionParser argument
+# @param options [Hash] Options hash being constructed; this is modified in this method.
+OctocatalogDiff::Cli::Options::Option.newoption(:puppetdb_token) do
+  has_weight 310
+
+  def parse(parser, options)
+    parser.on('--puppetdb-token TOKEN', 'Token to access the PuppetDB API') do |token|
+      options[:puppetdb_token] = token
+    end
+  end
+end

lib/octocatalog-diff/cli/options/puppetdb_token_file.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+# Specify the PE RBAC token to access the PuppetDB API. Refer to
+# https://puppet.com/docs/pe/latest/rbac/rbac_token_auth_intro.html#generate-a-token-using-puppet-access
+# for details on generating and obtaining a token. Use this option to specify the text
+# in a file, to read the content of the token from the file.
+# @param parser [OptionParser object] The OptionParser argument
+# @param options [Hash] Options hash being constructed; this is modified in this method.
+OctocatalogDiff::Cli::Options::Option.newoption(:puppetdb_token_file) do
+  has_weight 310
+
+  def parse(parser, options)
+    parser.on('--puppetdb-token-file PATH', 'Path containing token for PuppetDB API, relative or absolute') do |x|
+      proposed_token_path = x.start_with?('/') ? x : File.join(options[:basedir], x)
+      unless File.file?(proposed_token_path)
+        raise Errno::ENOENT, "Provided PuppetDB API token (#{proposed_token_path}) does not exist"
+      end
+      options[:puppetdb_token] = File.read(proposed_token_path)
+    end
+  end
+end

lib/octocatalog-diff/puppetdb.rb

@@ -42,6 +42,7 @@ class PuppetDB
     # @param :puppetdb_ssl_client_p12 [String] pkcs12-encoded client key and certificate
     # @param :puppetdb_ssl_client_password [String] Path to file containing password for SSL client key (any format)
     # @param :puppetdb_ssl_client_auth [Boolean] Override the client-auth that is guessed from parameters
+    # @param :puppetdb_token [String] PE RBAC token to authenticate to PuppetDB API
     # @param :timeout [Integer] Connection timeout for PuppetDB (default=10)
     def initialize(options = {})
       @connections =
@@ -107,7 +108,10 @@ def _get(path)
         ].join('')
 
         begin
-          more_options = { headers: { 'Accept' => 'application/json' }, timeout: @timeout }
+          headers = { 'Accept' => 'application/json' }
+          headers['X-Authentication'] = @options[:puppetdb_token] if @options[:puppetdb_token]
+          more_options = { headers: headers, timeout: @timeout }
+
           if connection[:username] || connection[:password]
             more_options[:basic_auth] = { username: connection[:username], password: connection[:password] }
           end

spec/octocatalog-diff/fixtures/catalogs/filter-single-item-array-1.json

@@ -0,0 +1,80 @@
+{
+  "document_type": "Catalog",
+  "data": {
+    "tags": [
+      "settings"
+    ],
+    "name": "my.rspec.node",
+    "version": "production",
+    "environment": "production",
+    "resources": [
+      {
+        "type": "File",
+        "title": "/tmp/amazing",
+        "file": "/environments/production/modules/foo/manifests/init.pp",
+        "line": 10,
+        "exported": false,
+        "parameters": {
+          "content": "This is my file.\nMy file is amazing.\n",
+          "group": "root",
+          "mode": "0755",
+          "notify": "Service[foo]",
+          "owner": "root"
+        }
+      },
+      {
+        "type": "File",
+        "title": "/tmp/awesome",
+        "file": "/environments/production/modules/foo/manifests/init.pp",
+        "line": 20,
+        "exported": false,
+        "parameters": {
+          "content": "This is my file.\nMy file is awesome.\n",
+          "group": "root",
+          "mode": "0755",
+          "notify": [
+            "Service[foo]",
+            "Service[bar]"
+          ],
+          "owner": "root",
+          "subscribe": [
+            "Service[baz]"
+          ]
+        }
+      },
+      {
+        "type": "File",
+        "title": "/tmp/fizzbuzz",
+        "file": "/environments/production/modules/foo/manifests/init.pp",
+        "line": 30,
+        "exported": false,
+        "parameters": {
+          "content": "1\n2\nfizz\n4\nbuzz\nfizz\n7\n8\nfizz\nbuzz\n",
+          "group": "root",
+          "mode": "0755",
+          "owner": "root"
+        }
+      },
+      {
+        "type": "File",
+        "title": "/tmp/foobar",
+        "file": "/environments/production/modules/foo/manifests/init.pp",
+        "line": 40,
+        "exported": false,
+        "parameters": {
+          "content": "foo\nbar\n",
+          "group": "root",
+          "mode": "0755",
+          "owner": "root",
+          "notify": "Service[foobar]"
+        }
+      }
+    ],
+    "classes": [
+      "settings"
+    ]
+  },
+  "metadata": {
+    "api_version": 1
+  }
+}