Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support array indexers with names and not just ordinal values #1853

Open
johndowns opened this issue Mar 12, 2021 · 10 comments
Open

Support array indexers with names and not just ordinal values #1853

johndowns opened this issue Mar 12, 2021 · 10 comments
Labels
enhancement New feature or request

Comments

@johndowns
Copy link
Contributor

When creating resources like virtual networks with subnets, I often want to refer to the subnet resource ID in another resource or as an output. Because the subnet property of the VNet is an array, the way I can reference that today is:

vnet.properties.subnets[0].id

I don't like this, because it is brittle to changes in the ordering of the subnets. What I'd love to be able to do instead is something like this:

vnet.properties.subnets[vmSubnetName].id

Currently the workaround I'm using is to use the resourceId() function directly, but it'd be nicer if Bicep supported this using the symbolic referencing syntax.

@alex-frankel
Copy link
Collaborator

I think the challenge with this is knowing which property is the primary key so that it can be provided in the array. It could be different for any given array property. I do agree it would be nice if we could figure this out.

@anthony-c-martin
Copy link
Member

anthony-c-martin commented Mar 18, 2021

I think the challenge with this is knowing which property is the primary key so that it can be provided in the array. It could be different for any given array property. I do agree it would be nice if we could figure this out.

If you treat an object as an iterable set of key-value pairs (quite a few languages do this), then something like the below may be possible:

var myProps = {
  first: {
    prop: 'firstVal'
  }
  second: {
    prop: 'secondVal'
  }
}

output propVals array = [for (key, val) in myProps: '${key}: ${val.prop}']
// when evaluated, this should be equivalent to
// output propVals array = [
//   'first: firstVal'
//   'second: secondVal'
// ]

EDIT: @johndowns - I'm not totally sure if what I posted solves your problem or is a whole different issue. It would be helpful if you could share a larger code sample including the resourceId() workaround you mentioned.

@majastrz
Copy link
Member

It sounds like the array of subnets has a bunch of subnets that could be re-ordered over time. One of the subnets has a specific name. I think the ask here is to have some mechanism that allows us to find an item in the array whose id property equals vmSubnet. @johndowns is using the resourceId function to construct the resource ID from vmSubnet avoids indexing over the array because the index may not be stable or is brittle.

I'm not sure array indexers would be the right place to solve this. The main issue is that we don't know which property is the "key" property in an array. With something like vnet.properties.subnets[vmSubnetName], how do we know that the id property is what should equal vmSubnetName and not name or some other property?

Since subnets are modeled as a child resource, it should be possible to use existing to create a symbolic name for the specific resource. This should work today.

One option would be to introduce some sort of capability to the runtime to find items in arrays. Maybe a function like find(input: array, property: string, value: any). The behavior would be similar to FirstOrDefault() in C#. The return value would be an item in the array or null if not found. There may also be a need for a function that returns all items in the array that match the "predicate".

The above would only allow for a limited set of predicates (mainly equality). A more powerful (and expensive) option would be to implement lambdas for use as predicates.

@fluffy-cakes
Copy link

Looping over objects could be another solution to this. Objects have keys that can be used as indexer. This would solve problems when referencing one resource that loops from inside another trying to attach the id to a value.

param param1 object = {
    key1: {
        subkey1: 'value'
        subkey2: 'value'
    }
    key2: {
        subkey1: 'value'
        subkey2: 'value'
    }
}

param param2 object = {
    key1: {
        whatevs: 'value'
        asdfValue: 'key1'
    }
    key2: {
        whatevs: 'value'
        asdfValue: 'key2'
    }
}

resource asdf 'theAPI' = (for_each param1): {
    //do some stuff
}

resource qwer 'theAPI' = (for_each param2): {
    name: param2.whatevs
    id: asdf[param2.asdfValue].id
}

@askew
Copy link

askew commented Apr 22, 2021

Whenever there is an array of objects in ARM each object always has a name property. It has to because this is how the ARM ids are constructed. vnet.properties.subnets[vmSubnetName].id would have to work for the subnet to have an id.

@bmoore-msft
Copy link
Contributor

@askew - not always, there are quite a few array properties in ARM that don't have a name propertie (e.g. accessPolicies, serviceEndpoints, failOver locations) they do usually have a key but the property (or properties) vary.

@miqm
Copy link
Collaborator

miqm commented Dec 31, 2021

Isn't this problem solved by #4456 that introduced items function?

@johndowns
Copy link
Contributor Author

@miqm No, I don't believe so the items function helps here, because the subnets property is an array rather than a dictionary. Please correct me if I'm wrong though.

However, the workaround I've been using (which isn't bad) is to define the subnet list in the subnets property and then using an existing nested resource to get a hard reference to the subnet:

var subnet1Name = 'subnet1'
var subnet2Name = 'subnet2'

resource vnet 'Microsoft.Network/virtualNetworks@2019-11-01' = {
  name: 'myvnet'
  location: resourceGroup().location
  properties: {
    addressSpace: {
      addressPrefixes: [
        '10.0.0.0/16'
      ]
    }
    subnets: [
      {
        name: subnet1Name
        properties: {
          addressPrefix: '10.0.0.0/24'
        }
      }
      {
        name: subnet2Name
        properties: {
          addressPrefix: '10.0.1.0/24'
        }
      }
    ]
  }

  resource subnet1 'subnets' existing = {
    name: subnet1Name
  }
}

output subnet1Id string = vnet::subnet1.id

I think it would still be nice to be able to use the syntax I proposed in the initial issue description (vnet.properties.subnets[vmSubnetName].id), because this workaround requires a bit of understanding of Bicep's type system as well as child resources and the way the VNet resource is defined.

@4c74356b41
Copy link

4c74356b41 commented Mar 23, 2022

jmespath queries. should be trival to just use that library. I've been asking for years (literally) in arm templates.

@NSimpragaVolur
Copy link

jmespath queries. should be trial to just use this library. I've been asking for years (literally) in arm templates.

This!

anthony-c-martin added a commit that referenced this issue Apr 23, 2024
… on lambdas (#13658)

Adds the spread operator `...` as well as various new functions +
indexes on lambdas:
1. Spread operator - usage is as follows:
    * In an object:
        ```bicep
        var objA = { bar: 'bar' }
var objB = { foo: 'foo', ...objA } // equivalent to { foo: 'foo', bar:
'bar' }
        ```
    * In an array:
        ```bicep
        var arrA = [ 2, 3 ]
        var arrB = [ 1, ...arrA, 4 ] // equivalent to [ 1, 2, 3, 4 ]
        ```
1. New functions + usage:
    * `objectKeys`: Returns the keys of an object parameter:
        ```bicep
        var example = objectKeys({ a: 1, b: 2 }) // returns [ 'a', 'b' ]
        ```
* `mapValues`: Create an object from an input object, using a custom
lambda to map values:
        ```bicep
var example = mapValues({ foo: 'foo' }, val => toUpper(val)) // returns
{ foo: 'FOO' }
        ```
* `groupBy`: Create an object with array values from an array, using a
grouping condition:
        ```bicep
var example = groupBy(['foo', 'bar', 'baz'], x => substring(x, 0, 1)) //
returns { f: [ 'foo' ], b: [ 'bar', 'baz' ]
        ```
* `shallowMerge`: Perform a shallow merge of input object parameters:
        ```bicep
var example = shallowMerge([{ foo: 'foo' }, { bar: 'bar' }]) // returns
{ foo: 'foo', bar: 'bar' }
        ```
1. Optional indices on lambdas + usage:
    * `map`:
        ```bicep
var example = map(['a', 'b'], (x, i) => { index: i, val: x }) // returns
[ { index: 0, val: 'a' }, { index: 1 val: 'b' } ]
        ```
    * `reduce`:
        ```bicep
var example = reduce([ 2, 3, 7 ], (cur, next, i) => (i % 2 == 0) ? cur +
next : cur) // returns 9
        ```
    * `filter`:
        ```bicep
var example = filter([ 'foo', 'bar', 'baz' ], (val, i) => i < 2 &&
substring(val, 0, 1) == 'b') // returns [ 'bar' ]
        ```
Closes #13560
Closes #9244
Closes #1560
Addresses some of the issues described under the following: #2082,
#1853, #387

###### Microsoft Reviewers: [Open in
CodeFlow](https://microsoft.github.io/open-pr/?codeflow=https://github.com/Azure/bicep/pull/13658)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

10 participants