feat: adds a DescribeSecurityGroups queryType

README.md

@@ -23,7 +23,8 @@ result is written to `target` (`status.<field>` or `context.<field>`).
 | `DescribeImages` | `[{imageId, name, ownerId, creationDate, architecture, state, rootDeviceType, description}]` |
 | `DescribeRouteTables` | `[{routeTableId, vpcId, ownerId, associations[{routeTableAssociationId, routeTableId, subnetId, gatewayId, main, state}], routes[{destinationCidrBlock, destinationIpv6CidrBlock, destinationPrefixListId, gatewayId, natGatewayId, transitGatewayId, vpcPeeringConnectionId, egressOnlyInternetGatewayId, carrierGatewayId, localGatewayId, coreNetworkArn, instanceId, networkInterfaceId, origin, state}], tags{}}]`. Carries the **main association ID**, which no CloudFormation schema models. `filters` **required**: `vpc-id`, `route-table-id`, `association.subnet-id`, `tag:<key>`. |
 | `DescribeSubnets` | `[{subnetId, subnetArn, vpcId, ownerId, availabilityZone, availabilityZoneId, cidrBlock, state, defaultForAz, mapPublicIpOnLaunch, availableIpAddressCount, ipv6Native, ipv6CidrBlockAssociationSet[{associationId, ipv6CidrBlock, state}], tags{}}]`. Returns only **live** subnets, unlike the Tagging API. `filters` **required**: `vpc-id`, `subnet-id`, `availability-zone`, `tag:<key>`. |
-| `DescribeSecurityGroupRules` | `[{securityGroupRuleId, securityGroupRuleArn, groupId, groupOwnerId, isEgress, ipProtocol, fromPort, toPort, cidrIpv4, cidrIpv6, prefixListId, referencedGroupId, referencedGroupUserId, referencedGroupVpcId, description, tags{}}]`. `filters` **required**: `group-id`, `security-group-rule-id`, `tag:<key>` - this operation does **not** accept `vpc-id`. |
+| `DescribeSecurityGroups` | `[{groupId, groupName, securityGroupArn, description, vpcId, ownerId, tags{}}]`. The only server-side path from a **VPC to its groups** - `DescribeSecurityGroupRules` does not accept `vpc-id`. Group-level only: the inline rules carry no rule ID, so pass the `groupId` from here to `DescribeSecurityGroupRules`. `filters` **required**: `vpc-id`, `group-id`, `group-name`, `description`, `owner-id`, `tag:<key>`. |
+| `DescribeSecurityGroupRules` | `[{securityGroupRuleId, securityGroupRuleArn, groupId, groupOwnerId, isEgress, ipProtocol, fromPort, toPort, cidrIpv4, cidrIpv6, prefixListId, referencedGroupId, referencedGroupUserId, referencedGroupVpcId, description, tags{}}]`. `filters` **required**: `group-id`, `security-group-rule-id`, `tag:<key>` - this operation does **not** accept `vpc-id`; AWS errors on it only where the region holds rules, and returns `[]` otherwise. |
 | `ListServiceQuotas` | `[{quotaCode, quotaName, value, unit, adjustable, globalQuota}]` (all quotas for a `serviceCode`) |
 | `GetServiceQuota` | `{quotaCode, quotaName, value, unit, adjustable, globalQuota}` (a single quota; needs `serviceCode`+`quotaCode`) |
 
@@ -46,12 +47,12 @@ result is written to `target` (`status.<field>` or `context.<field>`).
   type including untagged ones. Caveats: needs that type's read IAM permissions,
   filters are applied client-side, and hydration costs one `GetResource` call per
   resource (use `hydrate=false` to skip it when you only want identifiers).
-- **`DescribeRouteTables` / `DescribeSubnets` / `DescribeSecurityGroupRules`** -
-  when the identifier you need is not in the resource's CloudFormation schema (a
-  route table's **main association ID** is the canonical case), or when you need
-  an **authoritative** answer. Needs `ec2:DescribeRouteTables`,
-  `ec2:DescribeSubnets` and `ec2:DescribeSecurityGroupRules` respectively;
-  without them the call fails at reconcile with `UnauthorizedOperation`. It
+- **`DescribeRouteTables` / `DescribeSubnets` / `DescribeSecurityGroups` /
+  `DescribeSecurityGroupRules`** - when the identifier you need is not in the
+  resource's CloudFormation schema (a route table's **main association ID** is
+  the canonical case), or when you need an **authoritative** answer. Each needs
+  its matching `ec2:Describe*` permission; without it the call fails at
+  reconcile with `UnauthorizedOperation`. It
   reads only what the `filters` select, server-side, so a foreign resource cannot
   fail the query. `filters` are **required**, and the accepted names differ per
   query type (see the table above); unfiltered these would be region-wide reads.
@@ -63,6 +64,10 @@ result is written to `target` (`status.<field>` or `context.<field>`).
   Tagging API can keep reporting deleted resources for a while - which, if they
   carry the same identifying tag as their live replacements, silently doubles
   the result set.
+  Security groups take two steps: `DescribeSecurityGroups` filtered by `vpc-id`
+  gives you the groups, then `DescribeSecurityGroupRules` filtered by those
+  `group-id`s gives you their rules, since that second operation cannot filter
+  by VPC.
 
 Rule of thumb: *IDs by tag →* `GetResources`; *attributes / full inventory of a
 type →* `ListResources`; *EC2 identifiers CloudFormation does not model, or an

aws.go

@@ -59,6 +59,7 @@ func (q *AWSQuery) registry() map[string]handler {
 		"DescribeImages":             q.describeImages,
 		"DescribeRouteTables":        q.describeRouteTables,
 		"DescribeSubnets":            q.describeSubnets,
+		"DescribeSecurityGroups":     q.describeSecurityGroups,
 		"DescribeSecurityGroupRules": q.describeSecurityGroupRules,
 		"ListServiceQuotas":          q.listServiceQuotas,
 		"GetServiceQuota":            q.getServiceQuota,
@@ -405,8 +406,42 @@ func routeTableRoutes(routes []ec2types.Route) []any {
 	return out
 }
 
+// describeSecurityGroups lists security groups (EC2, paginated). Projects
+// group-level fields only: the inline IpPermissions/IpPermissionsEgress carry
+// no security group rule ID, so an individual rule in them is not addressable -
+// DescribeSecurityGroupRules owns rule-level data, filtered by the groupId
+// returned here. Unlike that operation this one does accept vpc-id, which makes
+// it the only server-side path from a VPC to its groups.
+func (q *AWSQuery) describeSecurityGroups(ctx context.Context, cfg aws.Config, in *v1beta1.Input) (any, error) {
+	client, err := ec2Client(cfg, in, "DescribeSecurityGroups", "vpc-id, group-id, group-name, description, owner-id, tag:<key>")
+	if err != nil {
+		return nil, err
+	}
+	p := ec2.NewDescribeSecurityGroupsPaginator(client, &ec2.DescribeSecurityGroupsInput{Filters: toEC2Filters(in.Filters)})
+	res := []any{}
+	for p.HasMorePages() {
+		page, err := p.NextPage(ctx)
+		if err != nil {
+			return nil, errors.Wrap(err, "DescribeSecurityGroups failed")
+		}
+		for _, sg := range page.SecurityGroups {
+			res = append(res, map[string]any{
+				"groupId":          aws.ToString(sg.GroupId),
+				"groupName":        aws.ToString(sg.GroupName),
+				"securityGroupArn": aws.ToString(sg.SecurityGroupArn),
+				"description":      aws.ToString(sg.Description),
+				"vpcId":            aws.ToString(sg.VpcId),
+				"ownerId":          aws.ToString(sg.OwnerId),
+				"tags":             ec2TagsToMap(sg.Tags),
+			})
+		}
+	}
+	return res, nil
+}
+
 // describeSecurityGroupRules lists security group rules (EC2, paginated).
-// Note the filter names: this operation does NOT accept vpc-id.
+// Note the filter names: unlike DescribeSecurityGroups, this operation does NOT
+// accept vpc-id, though AWS only errors on it where the region holds rules.
 func (q *AWSQuery) describeSecurityGroupRules(ctx context.Context, cfg aws.Config, in *v1beta1.Input) (any, error) {
 	client, err := ec2Client(cfg, in, "DescribeSecurityGroupRules", "group-id, security-group-rule-id, tag:<key>")
 	if err != nil {

aws_test.go

@@ -28,7 +28,9 @@ import (
 type respStub struct {
 	bodies      []string
 	contentType string
-	calls       int
+	// status is the HTTP status of every response; zero means 200.
+	status int
+	calls  int
 	// requests records each marshalled request body, so a test can assert what
 	// was actually sent (e.g. that a filter went server-side).
 	requests []string
@@ -50,8 +52,12 @@ func (s *respStub) Do(req *http.Request) (*http.Response, error) {
 	if s.contentType != "" {
 		h.Set("Content-Type", s.contentType)
 	}
+	status := s.status
+	if status == 0 {
+		status = http.StatusOK
+	}
 	return &http.Response{
-		StatusCode: http.StatusOK,
+		StatusCode: status,
 		Header:     h,
 		Body:       io.NopCloser(strings.NewReader(body)),
 	}, nil
@@ -339,6 +345,7 @@ func TestHandlerValidationGuards(t *testing.T) {
 		"RouteTablesNoRegion": func() (any, error) { return q.describeRouteTables(ctx, noRegion, &v1beta1.Input{}) },
 		"SubnetsNoRegion":     func() (any, error) { return q.describeSubnets(ctx, noRegion, &v1beta1.Input{}) },
 		"SGRulesNoRegion":     func() (any, error) { return q.describeSecurityGroupRules(ctx, noRegion, &v1beta1.Input{}) },
+		"SGsNoRegion":         func() (any, error) { return q.describeSecurityGroups(ctx, noRegion, &v1beta1.Input{}) },
 		"SubnetsNoFilters": func() (any, error) {
 			return q.describeSubnets(ctx, stubCfg(&respStub{}), &v1beta1.Input{})
 		},
@@ -529,8 +536,30 @@ const (
 		`<item><securityGroupRuleId>sgr-3</securityGroupRuleId><groupId>sg-1</groupId><groupOwnerId>123456789012</groupOwnerId>` +
 		`<isEgress>false</isEgress><ipProtocol>icmp</ipProtocol><prefixListId>pl-1</prefixListId>` +
 		`</item></securityGroupRuleSet></DescribeSecurityGroupRulesResponse>`
+
+	securityGroupsPage1 = `<DescribeSecurityGroupsResponse xmlns="http://ec2.amazonaws.com/doc/2016-11-15/"><requestId>r</requestId>` +
+		`<securityGroupInfo><item><groupId>sg-1</groupId><groupName>default</groupName>` +
+		`<groupDescription>default VPC security group</groupDescription><vpcId>vpc-1</vpcId>` +
+		`<ownerId>123456789012</ownerId>` +
+		`<securityGroupArn>arn:aws:ec2:eu-central-1:123456789012:security-group/sg-1</securityGroupArn>` +
+		// Inline rules the SDK deserializes but the projection drops on purpose.
+		`<ipPermissions><item><ipProtocol>tcp</ipProtocol><fromPort>443</fromPort><toPort>443</toPort>` +
+		`<ipRanges><item><cidrIp>0.0.0.0/0</cidrIp></item></ipRanges></item></ipPermissions>` +
+		`<tagSet/></item></securityGroupInfo>` +
+		`<nextToken>tok</nextToken></DescribeSecurityGroupsResponse>`
+
+	securityGroupsPage2 = `<DescribeSecurityGroupsResponse xmlns="http://ec2.amazonaws.com/doc/2016-11-15/"><requestId>r</requestId>` +
+		`<securityGroupInfo><item><groupId>sg-2</groupId><groupName>app</groupName>` +
+		`<groupDescription>app tier</groupDescription><vpcId>vpc-1</vpcId><ownerId>123456789012</ownerId>` +
+		`<securityGroupArn>arn:aws:ec2:eu-central-1:123456789012:security-group/sg-2</securityGroupArn>` +
+		`<tagSet><item><key>Name</key><value>app</value></item></tagSet>` +
+		`</item></securityGroupInfo></DescribeSecurityGroupsResponse>`
 )
 
+// ec2QueryTypes are the direct EC2 describes that share the ec2Client guard
+// (region and filters required).
+var ec2QueryTypes = []string{"DescribeRouteTables", "DescribeSubnets", "DescribeSecurityGroups", "DescribeSecurityGroupRules"}
+
 // ec2Input builds an input for one query type with a filter that query type
 // actually supports. This is not cosmetic: DescribeSecurityGroupRules accepts
 // only group-id, security-group-rule-id and tag:<key>, and an unrecognised
@@ -553,6 +582,7 @@ func TestEc2Dispatches(t *testing.T) {
 		"DescribeRouteTables":        routeTablesPage2,
 		"DescribeSubnets":            subnetsBody,
 		"DescribeSecurityGroupRules": securityGroupRulesBody,
+		"DescribeSecurityGroups":     securityGroupsPage2,
 	}
 	for queryType, body := range cases {
 		t.Run(queryType, func(t *testing.T) {
@@ -578,7 +608,7 @@ func TestEc2Dispatches(t *testing.T) {
 // into XR status. It is reachable without a typo: toFilters returns a non-nil
 // empty slice, so a filtersRef resolving to [] arrives with len 0.
 func TestEc2FilterGuard(t *testing.T) {
-	for _, queryType := range []string{"DescribeRouteTables", "DescribeSubnets", "DescribeSecurityGroupRules"} {
+	for _, queryType := range ec2QueryTypes {
 		t.Run(queryType, func(t *testing.T) {
 			h := newQuery().registry()[queryType]
 			_, err := h(context.Background(), stubCfg(&respStub{}), &v1beta1.Input{})
@@ -645,12 +675,40 @@ func TestEc2RouteTablesPaginates(t *testing.T) {
 // Isolates the region guard: the shared guards table only asserts err != nil,
 // which the SDK's endpoint-resolution error satisfies on its own.
 func TestEc2RegionGuard(t *testing.T) {
-	_, err := newQuery().describeSubnets(context.Background(), aws.Config{}, ec2Input("DescribeSubnets"))
-	if err == nil {
-		t.Fatal("expected an error, got nil")
+	for _, queryType := range ec2QueryTypes {
+		t.Run(queryType, func(t *testing.T) {
+			h := newQuery().registry()[queryType]
+			_, err := h(context.Background(), aws.Config{}, ec2Input(queryType))
+			if err == nil {
+				t.Fatal("expected an error, got nil")
+			}
+			if !strings.Contains(err.Error(), "requires a region") {
+				t.Errorf("expected the region guard, got: %v", err)
+			}
+		})
 	}
-	if !strings.Contains(err.Error(), "requires a region") {
-		t.Errorf("expected the region guard, got: %v", err)
+}
+
+// TestEc2APIError pins that an AWS-side failure - typically a missing
+// ec2:Describe* permission - surfaces wrapped with the operation name and the
+// AWS error code, rather than as an empty result written to the XR.
+func TestEc2APIError(t *testing.T) {
+	const body = `<Response><Errors><Error><Code>UnauthorizedOperation</Code>` +
+		`<Message>You are not authorized to perform this operation.</Message></Error></Errors>` +
+		`<RequestID>r</RequestID></Response>`
+	for _, queryType := range ec2QueryTypes {
+		t.Run(queryType, func(t *testing.T) {
+			stub := &respStub{bodies: []string{body}, contentType: "text/xml", status: http.StatusForbidden}
+			got, err := newQuery().registry()[queryType](context.Background(), stubCfg(stub), ec2Input(queryType))
+			if err == nil {
+				t.Fatalf("expected an error, got result %#v", got)
+			}
+			for _, want := range []string{queryType + " failed", "UnauthorizedOperation"} {
+				if !strings.Contains(err.Error(), want) {
+					t.Errorf("expected %q in error, got: %v", want, err)
+				}
+			}
+		})
 	}
 }
 
@@ -727,6 +785,43 @@ func TestEc2SecurityGroupRules(t *testing.T) {
 	}
 }
 
+// TestEc2SecurityGroups pins the group-level projection across two pages.
+// ipPermissions/ipPermissionsEgress are deliberately NOT projected: they carry
+// no securityGroupRuleId, so an individual rule in them is not addressable and
+// DescribeSecurityGroupRules owns rule-level data. Page 1 carries an inbound
+// rule so that omission stays pinned rather than accidental.
+func TestEc2SecurityGroups(t *testing.T) {
+	stub := &respStub{bodies: []string{securityGroupsPage1, securityGroupsPage2}, contentType: "text/xml"}
+	got, err := newQuery().describeSecurityGroups(context.Background(), stubCfg(stub), ec2Input("DescribeSecurityGroups"))
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	want := []any{
+		map[string]any{
+			"groupId": "sg-1", "groupName": "default", "description": "default VPC security group",
+			"securityGroupArn": "arn:aws:ec2:eu-central-1:123456789012:security-group/sg-1",
+			"vpcId":            "vpc-1", "ownerId": "123456789012", "tags": map[string]any{},
+		},
+		map[string]any{
+			"groupId": "sg-2", "groupName": "app", "description": "app tier",
+			"securityGroupArn": "arn:aws:ec2:eu-central-1:123456789012:security-group/sg-2",
+			"vpcId":            "vpc-1", "ownerId": "123456789012", "tags": map[string]any{"Name": "app"},
+		},
+	}
+	if diff := cmp.Diff(want, got); diff != "" {
+		t.Errorf("-want +got:\n%s", diff)
+	}
+	if len(stub.requests) != 2 {
+		t.Fatalf("expected 2 requests (one per page), got %d", len(stub.requests))
+	}
+	// Unlike DescribeSecurityGroupRules, this operation does accept vpc-id, and
+	// that is the reason it exists: it is the only way to get from a VPC to its
+	// groups server-side.
+	if !strings.Contains(stub.requests[0], "Filter.1.Name=vpc-id") {
+		t.Errorf("vpc-id filter not sent server-side: %s", stub.requests[0])
+	}
+}
+
 func TestEc2TagsToMap(t *testing.T) {
 	got := ec2TagsToMap([]ec2types.Tag{{Key: aws.String("Name"), Value: aws.String("x")}})
 	if diff := cmp.Diff(map[string]any{"Name": "x"}, got); diff != "" {

example/Makefile

@@ -39,6 +39,7 @@ cloudcontrol-ids|xr.yaml|composition-cloudcontrol-ids.yaml|
 tagging-subnets|xr.yaml|composition-tagging-subnets.yaml|
 ec2-route-tables|xr.yaml|composition-ec2-route-tables.yaml|
 ec2-subnets|xr.yaml|composition-ec2-subnets.yaml|
+ec2-security-groups|xr.yaml|composition-ec2-security-groups.yaml|
 ec2-security-group-rules|xr.yaml|composition-ec2-security-group-rules.yaml|
 dynamic-refs|xr-dynamic.yaml|composition-dynamic-refs.yaml|
 dynamic-context|xr.yaml|composition-dynamic-context.yaml|--context-values='$(CTX)'

example/README.md

@@ -49,6 +49,7 @@ status:
 | `composition-ami-lookup.yaml` | DescribeImages | EC2 | `status.amis` |
 | `composition-ec2-route-tables.yaml` | DescribeRouteTables | EC2 | `status.routeTables` |
 | `composition-ec2-subnets.yaml` | DescribeSubnets | EC2 | `status.subnets` |
+| `composition-ec2-security-groups.yaml` | DescribeSecurityGroups | EC2 | `status.securityGroups` |
 | `composition-ec2-security-group-rules.yaml` | DescribeSecurityGroupRules | EC2 | `status.securityGroupRules` |
 | `composition-service-quotas.yaml` | ListServiceQuotas | Service Quotas | `status.ec2Quotas` |
 | `composition-cloudcontrol-vpcs.yaml` | ListResources | Cloud Control | `status.prodVpcs` |

example/composition-ec2-security-groups.yaml

@@ -0,0 +1,35 @@
+apiVersion: apiextensions.crossplane.io/v1
+kind: Composition
+metadata:
+  name: function-aws-query-ec2-security-groups
+spec:
+  compositeTypeRef:
+    apiVersion: example.io/v1alpha1
+    kind: XAccount
+  mode: Pipeline
+  pipeline:
+  # The VPC's default security group. AWS creates it, so it carries no tags and
+  # no Crossplane resource references it - neither GetResources nor a tag filter
+  # can find it. This is also the only EC2 describe that accepts vpc-id for
+  # security groups: DescribeSecurityGroupRules does not, so a rules query has
+  # to be filtered by the group-id this step returns.
+  - step: default-security-group
+    functionRef:
+      name: function-aws-query
+    input:
+      apiVersion: aws.fn.crossplane.io/v1beta1
+      kind: Input
+      queryType: DescribeSecurityGroups
+      region: eu-central-1
+      filters:
+      - name: vpc-id
+        values: ["vpc-0123456789abcdef0"]
+      - name: group-name
+        values: ["default"]
+      target: status.securityGroups
+    credentials:
+    - name: aws-creds
+      source: Secret
+      secretRef:
+        namespace: crossplane-system
+        name: aws-creds