crowdstrike: improvements and fixes to ingest pipelines for all data streams

[navnit-elastic]

Proposed commit message

crowdstrike: improvements and fixes to ingest pipelines for all data streams

fixes all the issues outlined in issue #15973.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• [ ]

How to test this PR locally

Pipeline Tests for all data streams:

--- Test results for package: crowdstrike - START ---
╭─────────────┬───────────────┬───────────┬──────────────────────────────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE     │ DATA STREAM   │ TEST TYPE │ TEST NAME                                                                    │ RESULT │ TIME ELAPSED │
├─────────────┼───────────────┼───────────┼──────────────────────────────────────────────────────────────────────────────┼────────┼──────────────┤
│ crowdstrike │ alert         │ pipeline  │ (ingest pipeline warnings test-alert.log)                                    │ PASS   │ 415.419166ms │
│ crowdstrike │ alert         │ pipeline  │ test-alert.log                                                               │ PASS   │ 265.621417ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-event-stream.log)                             │ PASS   │ 399.122209ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-audit-events.log)                      │ PASS   │ 446.721667ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-auth-activity.log)                     │ PASS   │   453.8645ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-cspmioa-streaming.log)                 │ PASS   │  491.57225ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-cspmsearch-streaming.log)              │ PASS   │ 474.186875ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-data-protection-detection-summary.log) │ PASS   │ 434.634875ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-detection-summary.log)                 │ PASS   │ 433.706167ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-epp-detection-summary.log)             │ PASS   │ 471.211625ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-events.log)                            │ PASS   │ 457.642625ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-firewall.log)                          │ PASS   │ 404.285209ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-identity-protection-incident.log)      │ PASS   │ 394.308375ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-incident-summary.log)                  │ PASS   │ 406.211708ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-ipd-summary.log)                       │ PASS   │ 408.486666ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-mobile-detection-summary.log)          │ PASS   │ 431.840333ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-recon-notification.log)                │ PASS   │ 404.128291ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-remote-response.log)                   │ PASS   │ 408.239333ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-sample.log)                            │ PASS   │   407.6305ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-tags-list.log)                         │ PASS   │ 433.912208ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-tags.log)                              │ PASS   │ 408.622958ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-user-activity.log)                     │ PASS   │ 420.845833ms │
│ crowdstrike │ falcon        │ pipeline  │ (ingest pipeline warnings test-falcon-xdr-detection-summary.log)             │ PASS   │ 448.744916ms │
│ crowdstrike │ falcon        │ pipeline  │ test-event-stream.log                                                        │ PASS   │ 145.416708ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-audit-events.log                                                 │ PASS   │ 111.738417ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-auth-activity.log                                                │ PASS   │  78.738125ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-cspmioa-streaming.log                                            │ PASS   │  78.328625ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-cspmsearch-streaming.log                                         │ PASS   │  77.802042ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-data-protection-detection-summary.log                            │ PASS   │  93.265666ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-detection-summary.log                                            │ PASS   │  90.241458ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-epp-detection-summary.log                                        │ PASS   │ 259.862917ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-events.log                                                       │ PASS   │  83.804334ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-firewall.log                                                     │ PASS   │  83.687667ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-identity-protection-incident.log                                 │ PASS   │   80.10225ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-incident-summary.log                                             │ PASS   │  68.054958ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-ipd-summary.log                                                  │ PASS   │  77.081084ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-mobile-detection-summary.log                                     │ PASS   │  85.974041ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-recon-notification.log                                           │ PASS   │  66.996458ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-remote-response.log                                              │ PASS   │  75.048334ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-sample.log                                                       │ PASS   │ 104.718625ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-tags-list.log                                                    │ PASS   │  67.588667ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-tags.log                                                         │ PASS   │  68.034917ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-user-activity.log                                                │ PASS   │  70.480083ms │
│ crowdstrike │ falcon        │ pipeline  │ test-falcon-xdr-detection-summary.log                                        │ PASS   │     77.884ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-data.log)                                     │ PASS   │ 422.628334ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr-cspm-ioa.log)                             │ PASS   │ 507.362333ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr-cspm-iom-evaluation.log)                  │ PASS   │    444.716ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr-cspm-iom.log)                             │ PASS   │ 415.004458ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr-data-protection-detection-summary.log)    │ PASS   │ 458.365542ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr-epp-detection-summary.log)                │ PASS   │ 466.797125ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr-lengthy-field-delete.log)                 │ PASS   │ 465.865333ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr-lengthy-field-index.log)                  │ PASS   │ 437.518416ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdr.log)                                      │ PASS   │ 472.625291ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-fdrv2-notmanaged.log)                         │ PASS   │ 507.205334ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-linux.log)                                    │ PASS   │ 458.793875ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-macos.log)                                    │ PASS   │   473.1535ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-tags-formats.log)                             │ PASS   │ 424.751959ms │
│ crowdstrike │ fdr           │ pipeline  │ (ingest pipeline warnings test-windows.log)                                  │ PASS   │    460.481ms │
│ crowdstrike │ fdr           │ pipeline  │ test-data.log                                                                │ PASS   │ 139.903166ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr-cspm-ioa.log                                                        │ PASS   │ 112.283542ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr-cspm-iom-evaluation.log                                             │ PASS   │  99.427125ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr-cspm-iom.log                                                        │ PASS   │ 100.011291ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr-data-protection-detection-summary.log                               │ PASS   │  99.198041ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr-epp-detection-summary.log                                           │ PASS   │  254.41975ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr-lengthy-field-delete.log                                            │ PASS   │  84.978917ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr-lengthy-field-index.log                                             │ PASS   │  82.725333ms │
│ crowdstrike │ fdr           │ pipeline  │ test-fdr.log                                                                 │ PASS   │ 1.420559792s │
│ crowdstrike │ fdr           │ pipeline  │ test-fdrv2-notmanaged.log                                                    │ PASS   │  78.446291ms │
│ crowdstrike │ fdr           │ pipeline  │ test-linux.log                                                               │ PASS   │ 175.896458ms │
│ crowdstrike │ fdr           │ pipeline  │ test-macos.log                                                               │ PASS   │ 256.396833ms │
│ crowdstrike │ fdr           │ pipeline  │ test-tags-formats.log                                                        │ PASS   │  77.859875ms │
│ crowdstrike │ fdr           │ pipeline  │ test-windows.log                                                             │ PASS   │ 1.279709542s │
│ crowdstrike │ host          │ pipeline  │ (ingest pipeline warnings test-host.log)                                     │ PASS   │ 397.009542ms │
│ crowdstrike │ host          │ pipeline  │ test-host.log                                                                │ PASS   │   65.42125ms │
│ crowdstrike │ vulnerability │ pipeline  │ (ingest pipeline warnings test-vulnerability.log)                            │ PASS   │ 417.808917ms │
│ crowdstrike │ vulnerability │ pipeline  │ test-vulnerability.log                                                       │ PASS   │  99.117083ms │
╰─────────────┴───────────────┴───────────┴──────────────────────────────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: crowdstrike - END   ---
Done

Related issues

• Closes [CrowdStrike] Pipeline improvements and fixes #15973

Screenshots

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: c507690

History

• 💚 Build #35982 succeeded c98679a

cc @navnit-elastic