feat(aws): CloudTrail timeline for findings #9101

MrCloudSec · 2025-10-30T15:23:23Z

Context

Add CloudTrail timeline to AWS findings as an API-only programmatic utility that provides timeline context showing who created/modified resources and when.

Description

Introduces CloudTrail timeline capability that queries AWS CloudTrail Event History to add lifecycle context to security findings.

Key Features:

141 event types across 20 AWS services: EC2, S3, RDS, Lambda, IAM, VPC, ELB, KMS, Secrets Manager, EBS, EKS, ECR, DynamoDB, SNS, SQS, CloudWatch, EventBridge, Backup, ACM, ECS
API-first design: Returns list[dict] for direct REST consumption
Session-based: Requires boto3.Session for flexible credential management
Regional support: Automatic CloudTrail client creation per region
Configurable: 90-day default lookback period (CloudTrail retention limit)
Test coverage: 42 tests (28 unit + 14 moto integration tests)

API Usage:

from prowler.providers.aws.lib.cloudtrail_timeline.cloudtrail_timeline import (
    CloudTrailTimeline,
)

timeline = CloudTrailTimeline(session=session, lookback_days=90)

timeline_events = timeline. get_resource_timeline(
    resource_id="sg-0abc123",
    resource_arn="arn:aws:ec2:us-east-1:123456789012:security-group/sg-0abc123",
    region="us-east-1"
)
# Returns: list[dict] with timestamp, event_type, principal, message, event_details

Checklist

Are there new checks included in this PR? Yes / No
- If so, do we need to update permissions for the provider? Please review this carefully.
Review if the code is being covered by tests.
Review if code is being documented following this specification https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings
Review if backport is needed.
Review if is needed to change the Readme.md
Ensure new entries are added to CHANGELOG.md, if applicable.

API

Verify if API specs need to be regenerated.
Check if version updates are required (e.g., specs, Poetry, etc.).
Ensure new entries are added to CHANGELOG.md, if applicable.

License

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

github-actions · 2025-10-30T15:23:46Z

⚠️ Changes detected in the following folders without a corresponding update to the CHANGELOG.md:

prowler

Please add an entry to the corresponding CHANGELOG.md file to maintain a clear history of changes.

github-actions · 2025-10-30T15:23:52Z

✅ Conflict Markers Resolved

All conflict markers have been successfully resolved in this pull request.

prowler/lib/scan/network_scanner.py

prowler/providers/aws/services/cloudtrail/lib/enrichment/cloudtrail_enricher.py

prowler/providers/aws/services/cloudtrail/lib/enrichment/formatters.py

github-actions · 2025-10-30T15:30:17Z

🔒 Container Security Scan

Image: prowler:4b38cae
Last scan: 2025-11-17 14:59:46 UTC

📊 Vulnerability Summary

Severity	Count
🔴 Critical	2
Total	2

2 package(s) affected

⚠️ Action Required

Critical severity vulnerabilities detected. These should be addressed before merging:

Review the detailed scan results
Update affected packages to patched versions
Consider using a different base image if updates are unavailable

📋 Resources:

Download full report (see artifacts)
View in Security tab
Scanned with Trivy

codecov · 2025-10-30T16:58:03Z

Codecov Report

❌ Patch coverage is 82.97735% with 263 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.12%. Comparing base (0f43789) to head (8e5b28a).
⚠️ Report is 6 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #9101       +/-   ##
===========================================
+ Coverage   58.94%   90.12%   +31.18%     
===========================================
  Files           8      837      +829     
  Lines         397    25077    +24680     
===========================================
+ Hits          234    22601    +22367     
- Misses        163     2476     +2313

Flag	Coverage Δ
prowler-py3.10-aws	`90.05% <82.97%> (?)`
prowler-py3.10-kubernetes	`?`
prowler-py3.11-aws	`90.08% <82.97%> (?)`
prowler-py3.11-kubernetes	`?`
prowler-py3.12-aws	`90.11% <82.97%> (?)`
prowler-py3.12-kubernetes	`?`
prowler-py3.9-aws	`90.06% <82.97%> (?)`
prowler-py3.9-kubernetes	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
prowler	`90.12% <82.97%> (+31.18%)`	⬆️
api	`∅ <ø> (∅)`

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

jfagoagas · 2025-11-04T08:06:03Z

prowler/lib/check/check.py

        )


+def _enrich_findings_post_scan(findings: list, output_options: Any) -> dict:


Why this name? Is this included in a class?

jfagoagas · 2025-11-04T08:07:51Z

prowler/lib/scan/scan.py

 from types import SimpleNamespace
 from typing import Generator

+from alive_progress import alive_bar


We can't add terminal UI things to the SDK. This is a CLI responsibility.

jfagoagas · 2025-11-04T08:08:19Z

prowler/lib/scan/scan.py

+    _enrich_findings: bool = False
+    _enrichment_config: dict = None
+    _enricher = None


Not sure about adding this to the Scan SDK. We need to think about it to have it as something external, executed afterwards.

We need to talk about the whole logic in this class. We need to move it somewhere else.

jfagoagas · 2025-11-04T08:10:20Z

prowler/providers/aws/services/cloudtrail/lib/enrichment/models.py

+    RESOURCE_MODIFIED = "resource_modified"
+    RESOURCE_DELETED = "resource_deleted"
+
+    # EC2 Instance events


Why not to add classes for each service?

jfagoagas · 2025-11-04T08:11:50Z

prowler/providers/aws/services/cloudtrail/cloudtrail_service.py

                f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
            )

+    def lookup_events_for_resource(


If we move the logic out of the Scan, maybe we'll need to create a new class for this part. We can't re-init the CloudTrail class. To me this would fit best in the enricher class.

jfagoagas · 2025-11-04T08:12:25Z

prowler/providers/common/models.py

    output_filename: str
    only_logs: bool
    unix_timestamp: bool
+    enrich_findings: bool


We need to find a way not to add more data to this object. The idea is to get rid of it as it makes some things dependant in between.

jfagoagas · 2025-11-07T08:02:44Z

prowler/CHANGELOG.md

 - `codepipeline_project_repo_private` check for AWS provider [(#5915)](https://github.com/prowler-cloud/prowler/pull/5915)
 - `cloudstorage_bucket_versioning_enabled` check for GCP provider [(#9014)](https://github.com/prowler-cloud/prowler/pull/9014)
 - `cloudstorage_bucket_soft_delete_enabled` check for GCP provider [(#9028)](https://github.com/prowler-cloud/prowler/pull/9028)
+- CloudTrail enrichment for AWS provider [(#9101)](https://github.com/prowler-cloud/prowler/pull/9101)


I'd not add this until the feature is implemented as for now is only an SDK thing.

prowler/CHANGELOG.md

puchy22

As general comment the current AWS-specific design would require complete duplication for Azure (Activity Log) and GCP (Audit Logs). I've recommended an abstraction layer with provider-agnostic base classes

puchy22 · 2025-11-07T12:16:46Z

prowler/providers/aws/services/cloudtrail/lib/__init__.py

Is this the best place to put the code? You are not really using nothing for the service and it is not expected to be used by checks or services, for me it should be at provider level like other integrations like S3 or SecurityHub in prowler/providers/aws/lib

puchy22 · 2025-11-07T12:29:33Z

prowler/providers/aws/services/cloudtrail/lib/enrichment/cloudtrail_enricher.py

+            )
+            return None
+
+    def _create_enrichment(


I believe that this method is never used

Missed this one when refactoring.

puchy22 · 2025-11-07T15:10:59Z

tests/providers/aws/services/cloudtrail/lib/enrichment/test_formatters.py

+
+        assert "1234abcd-12ab-34cd-56ef-1234567890ab" in message
+        assert "30" in message
+        assert "⚠️" in message or "DELETION SCHEDULED" in message


I am not sure if we should add this kind of emojis in the tests

puchy22 · 2025-11-07T15:20:24Z

prowler/providers/aws/lib/cloudtrail_enrichment/cloudtrail_enricher.py

@@ -0,0 +1,1269 @@
+"""CloudTrail enrichment service for Prowler findings.


I am concerned about the sustainability of this file. I think it should be split into different files for each service and we should try to use another more sustainable way of adding new events that does not involve modifying a file with 1200 lines of code

puchy22 · 2025-11-07T15:27:35Z

prowler/providers/aws/lib/cloudtrail_timeline/formatters.py

@@ -0,0 +1,1882 @@
+"""Event message formatters for CloudTrail events.


Same as with cloudtrail_enricher.py, it is not sustainable having a class that is 1800 lines of code. It's so difficult to know what is the class exactly doing. Is there any way to divide it into other parts to make it more sustainable?

puchy22 · 2025-11-07T15:40:44Z

prowler/providers/aws/services/cloudtrail/lib/enrichment/models.py

+
+
+# Union type for all event types
+TimelineEventType = (


Why are generating this union at module level and then only using it in the type hint of the TimelineEvent class.
Besides, this won't be a type as such, but rather a union. It seems like a strange way to do it, especially since it will only be used in one place.

100%, changed!

puchy22 · 2025-11-07T15:41:51Z

prowler/providers/aws/services/cloudtrail/lib/enrichment/models.py

+)
+
+
+@dataclass


Why using dataclass instead of using pydantic BaseModel?

Agree, good catch.

puchy22 · 2025-11-07T15:43:23Z

prowler/providers/aws/services/cloudtrail/lib/enrichment/models.py

+    last_modified_at: datetime | None = None
+    related_resources: list[dict[str, str]] = field(default_factory=list)
+
+    def to_dict(self) -> dict[str, Any]:


Using BaseModel saves you this method: https://docs.pydantic.dev/1.10/usage/exporting_models/#modeldict

Done, good catch!

puchy22 · 2025-11-07T15:49:23Z

prowler/providers/aws/services/cloudtrail/lib/enrichment/models.py

+        if not self.created_at:
+            return None
+
+        from datetime import timezone


Why this here and not in the top of the file?

Done, I did it inside the function because we only need to import it if we use this function.

puchy22 · 2025-11-07T15:50:06Z

prowler/providers/aws/services/cloudtrail/lib/enrichment/models.py

+        if not self.last_modified_at:
+            return None
+
+        from datetime import timezone


prowler/providers/aws/lib/cloudtrail_enrichment/models.py

prowler/providers/aws/lib/cloudtrail_timeline/models.py

api/src/backend/api/v1/views.py

github-actions · 2025-11-14T23:46:34Z

🔒 Container Security Scan

Image: prowler-api:79ea883
Last scan: 2025-11-14 23:46:33 UTC

📊 Vulnerability Summary

Severity	Count
🔴 Critical	4
Total	4

3 package(s) affected

⚠️ Action Required

Critical severity vulnerabilities detected. These should be addressed before merging:

Review the detailed scan results
Update affected packages to patched versions
Consider using a different base image if updates are unavailable

📋 Resources:

Download full report (see artifacts)
View in Security tab
Scanned with Trivy

AdriiiPRodri

You need to add dedicated timeline serializers and hook them into the viewset so the timeline action follows our current response schema, because right now the endpoint isn’t aligned with it.

You also need to clean up the timeline action: validate filter[lookback_days], use the existing provider session initialization, remove the ad-hoc JSON body handling, and return errors through the usual DRF validation flow. Make sure the endpoint is documented with extend_schema, and if there are constants involved, they should live in the view as class-level attributes. Remove the global query-parameter override since it’s no longer needed.

Finally, add the required unittests: provider validation, lookback validation, and the happy path using mocked AWS components

AdriiiPRodri · 2025-11-17T08:47:56Z

api/src/backend/api/v1/views.py

+            from api.utils import initialize_prowler_provider
+            from prowler.providers.aws.lib.cloudtrail_timeline.cloudtrail_timeline import (
+                CloudTrailTimeline,
+            )


Why are we importing this here? I mean, this code runs on every call to this endpoint. This should be at the top of the file

jfagoagas · 2025-11-17T10:05:04Z

api/src/backend/api/v1/views.py

+        url_name="timeline",
+        url_path=r"timeline(?:/(?P<lookback_days>[0-9]+))?",
+    )
+    def timeline(self, request, pk=None, lookback_days=None):


@MrCloudSec based on your tests, how long can this take?

@AdriiiPRodri @vicferpoy @josemazo @rakan-pro I'm not sure about having a synchronous endpoint to retrieve this data. From a design standpoint I think we'd have an async endpoint to enqueue a task to fetch that data. Then, another endpoint to get the status/result. What do you think?

I don't currently know how long it takes us to retrieve that data, but if it can block, then I completely agree with you, this should be a task, and we should have an endpoint to fetch the results

It takes one second or less to retrieve the timelin for each resource.

Even if the timeline is more than 90 days? Since this depends on CloudTrail and the user's events, we need to make this async.

In the API side or in the SDK?

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

github-actions bot added documentation provider/aws Issues/PRs related with the AWS provider output/csv Issues/PRs related with the CSV output format output/ocsf Issues/PRs related with the OCSF output format output/html Issues/PRs related with the HTML output format labels Oct 30, 2025

mintlify bot deployed to staging - docs October 30, 2025 15:25 View deployment

github-advanced-security bot found potential problems Oct 30, 2025

View reviewed changes

mintlify bot deployed to staging - docs November 3, 2025 20:00 View deployment

MrCloudSec marked this pull request as ready for review November 3, 2025 21:35

MrCloudSec requested review from a team as code owners November 3, 2025 21:35

jfagoagas requested changes Nov 4, 2025

View reviewed changes

github-actions bot removed documentation output/ocsf Issues/PRs related with the OCSF output format output/html Issues/PRs related with the HTML output format labels Nov 6, 2025

mintlify bot deployed to staging - docs November 6, 2025 16:22 View deployment

github-actions bot removed the output/csv Issues/PRs related with the CSV output format label Nov 6, 2025

MrCloudSec requested a review from jfagoagas November 6, 2025 20:20

jfagoagas reviewed Nov 7, 2025

View reviewed changes

MrCloudSec commented Nov 7, 2025

View reviewed changes

prowler/CHANGELOG.md Outdated Show resolved Hide resolved

puchy22 requested changes Nov 7, 2025

View reviewed changes

MrCloudSec added the no-changelog Skip including change in changelog/release notes label Nov 7, 2025

github-advanced-security bot found potential problems Nov 7, 2025

View reviewed changes

prowler/providers/aws/lib/cloudtrail_enrichment/models.py Fixed Show fixed Hide fixed

github-advanced-security bot found potential problems Nov 7, 2025

View reviewed changes

prowler/providers/aws/lib/cloudtrail_enrichment/models.py Fixed Show fixed Hide fixed

MrCloudSec requested a review from jfagoagas November 7, 2025 16:54

github-advanced-security bot found potential problems Nov 7, 2025

View reviewed changes

prowler/providers/aws/lib/cloudtrail_timeline/models.py Fixed Show fixed Hide fixed

mintlify bot deployed to staging - docs November 14, 2025 20:58 View deployment

MrCloudSec requested a review from a team as a code owner November 14, 2025 23:35

github-actions bot added the component/api label Nov 14, 2025

github-advanced-security bot found potential problems Nov 14, 2025

View reviewed changes

api/src/backend/api/v1/views.py Fixed Show fixed Hide fixed

api/src/backend/api/v1/views.py Fixed Show fixed Hide fixed

api/src/backend/api/v1/views.py Fixed Show fixed Hide fixed

AdriiiPRodri requested changes Nov 17, 2025

View reviewed changes

jfagoagas reviewed Nov 17, 2025

View reviewed changes

MrCloudSec added 15 commits November 17, 2025 09:49

feat(aws): add CloudTrail enrichment to findings

476ce71

chore: revision

8934879

fix: test

79cd1c7

fix: lib tests

39708ae

chore: refactor

1974977

chore: refactor

7b6f133

chore: refactor

6fb2543

chore: refactor

2241fe0

chore: improvement

97d38b0

chore: refactor

1b99a1f

chore: moving files

104c99d

chore: add noqa

39bbf82

chore: renaming

3a4a24e

chore: solve comment

268af61

chore: solve comment

8e5b28a

MrCloudSec force-pushed the PROWLER-297-cloud-trail-finding-enrichment branch from 20e243d to 8e5b28a Compare November 17, 2025 14:52

github-actions bot removed the component/api label Nov 17, 2025

mintlify bot deployed to staging - docs November 17, 2025 14:53 View deployment

MrCloudSec requested review from AdriiiPRodri and jfagoagas November 17, 2025 14:54

MrCloudSec mentioned this pull request Nov 17, 2025

feat(api): add CloudTrail timeline endpoint for AWS resources #9250

Merged

13 tasks

feat(api): add CloudTrail timeline endpoint for AWS resources (#9250)

cae607f

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

github-actions bot added the component/api label Nov 21, 2025

		)


		def _enrich_findings_post_scan(findings: list, output_options: Any) -> dict:

		@@ -0,0 +1,1269 @@
		"""CloudTrail enrichment service for Prowler findings.

		@@ -0,0 +1,1882 @@
		"""Event message formatters for CloudTrail events.

		)


		@dataclass

feat(aws): CloudTrail timeline for findings #9101

Are you sure you want to change the base?

feat(aws): CloudTrail timeline for findings #9101

Uh oh!

Conversation

MrCloudSec commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

Description

Checklist

API

License

Uh oh!

github-actions bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔒 Container Security Scan

📊 Vulnerability Summary

⚠️ Action Required

Uh oh!

codecov bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

puchy22 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

MrCloudSec commented Oct 30, 2025 •

edited

Loading

github-actions bot commented Oct 30, 2025 •

edited

Loading

github-actions bot commented Oct 30, 2025 •

edited

Loading

github-actions bot commented Oct 30, 2025 •

edited

Loading

codecov bot commented Oct 30, 2025 •

edited

Loading