Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
103 changes: 68 additions & 35 deletions docs/deployment/ecosystem/collectors.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,46 +4,33 @@

If you want to learn more about the concept and features of collectors, you can have more info [here](../../usage/collectors.md).

## Installation
!!! question "Collectors list"

### External (Python) collectors
You are looking for the available collectors? The list is in the [OpenAEV Ecosystem](https://filigran.notion.site/OpenAEV-Ecosystem-30d8eb73d7d04611843e758ddef8941b).

#### Configuration

All external collectors have to be able to access the OpenAEV API. To allow this connection, they have 2 mandatory configuration parameters, the `OPENAEV_URL` and the `OPENAEV_TOKEN`. In addition to these 2 parameters, collectors have other mandatory parameters that need to be set in order to get them work.
## Installing a collector

!!! info "Collector tokens"
There are multiple ways to deploy a collector from OpenAEV:

You can use your administrator token or create another administrator service account to put in your collectors. It is not necessary to have one dedicated user for each collector.
- Integration Manager (Recommended)
- Docker deployment
- Manual deployment

Here is an example of a collector `docker-compose.yml` file:
```yaml
- OPENAEV_URL=http://localhost
- OPENAEV_TOKEN=ChangeMe
- COLLECTOR_ID=ChangeMe # Specify a valid UUIDv4 of your choice
- "COLLECTOR_NAME=MITRE ATT&CK"
- COLLECTOR_LOG_LEVEL=error
```
!!! info

Here is an example in a collector `config.yml` file:
All collectors require access to the OpenAEV API. See [Configuration](#configuration) for required parameters.

```yaml
openaev:
url: 'http://localhost:3001'
token: 'ChangeMe'
### Integration Manager (Recommended)
The easiest way to deploy collectors is through the Integration Manager, which allows automatic deployment directly from the OpenAEV interface.

collector:
id: 'ChangeMe'
name: 'MITRE ATT&CK'
log_level: 'info'
```

## Docker activation
👉 See the [Integration Manager documentation](integration-manager/overview.md) for detailed instructions.

You can either directly run the Docker image of collectors or add them to your current `docker-compose.yml` file.

### Add a collector to your deployment
### Docker Deployment
Several options are available for Docker deployment:

#### Add a collector to your existing deployment
For instance, to enable the MITRE ATT&CK collector, you can add a new service to your `docker-compose.yml` file:

```docker
Expand All @@ -57,10 +44,10 @@ For instance, to enable the MITRE ATT&CK collector, you can add a new service to
- COLLECTOR_LOG_LEVEL=error
restart: always
```
Note: Collector images and available versions can be found on Docker Hub.

### Launch a standalone collector

To launch standalone collector, you can use the `docker-compose.yml` file of the collector itself. Just download the latest [release](https://github.com/OpenAEV-Platform/collectors/releases) and start the collector:
#### Launch a standalone collector
To launch a standalone collector, you can use the `docker-compose.yml` file of the collector itself. Just download the latest [release](https://github.com/OpenAEV-Platform/collectors/releases) and start the collector:

```
$ wget https://github.com/OpenAEV-Platform/collectors/archive/{RELEASE_VERSION}.zip
Expand All @@ -74,9 +61,8 @@ Change the configuration in the `docker-compose.yml` according to the parameters
$ docker compose up
```

## Manual activation

If you want to manually launch collector, you just have to install Python 3 and pip3 for dependencies:
### Manual deployment
If you want to manually launch collector without docker, you just have to install Python 3 and pip3 for dependencies:

```
$ apt install python3 python3-pip
Expand All @@ -97,12 +83,59 @@ $ pip3 install -r requirements.txt
$ cp config.yml.sample config.yml
```

Change the `config.yml` content according to the parameters of the platform and of the targeted service and launch the collector:
Change the `config.yml` content according to the parameters of the platform and of the targeted service.
For example :

```yaml

openaev:
url: 'http://localhost:3001'
token: 'ChangeMe'

collector:
id: 'ChangeMe'
name: 'MITRE ATT&CK'
log_level: 'info'

```


Finally : launch the collector:

```
$ python3 openaev_mitre.py
```

### Configuration

All external collectors have to be able to access the OpenAEV API. To allow this connection, they have 2 mandatory configuration parameters, the `OPENAEV_URL` and the `OPENAEV_TOKEN`. In addition to these 2 parameters, collectors have other mandatory parameters that need to be set to make them work.

!!! info "Collector tokens"

You can use your administrator token or create another administrator service account to put in your collectors. It is not necessary to have one dedicated user for each collector.

Here is an example of a collector `docker-compose.yml` file:
```yaml
- OPENAEV_URL=http://localhost
- OPENAEV_TOKEN=ChangeMe
- COLLECTOR_ID=ChangeMe # Specify a valid UUIDv4 of your choice
- "COLLECTOR_NAME=MITRE ATT&CK"
- COLLECTOR_LOG_LEVEL=error
```

Here is an example in a collector `config.yml` file:

```yaml
openaev:
url: 'http://localhost:3001'
token: 'ChangeMe'

collector:
id: 'ChangeMe'
name: 'MITRE ATT&CK'
log_level: 'info'
```

## Collectors status

The collector status can be displayed in the dedicated section of the platform available in Integration > collectors. You will be able to see the statistics of the RabbitMQ queue of the collector:
Expand Down
113 changes: 70 additions & 43 deletions docs/deployment/ecosystem/injectors.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,53 +17,26 @@ just add the proper configuration parameters in your platform configuration.

### External (Python) injectors

#### Configuration
There are multiple ways to deploy an external injectors from OpenAEV:

All external injectors have to be able to access the OpenAEV API. To allow this connection, they have 2 mandatory configuration parameters, the `OPENAEV_URL` and the `OPENAEV_TOKEN`. In addition to these 2 parameters, injectors have other mandatory parameters that need to be set in order to get them work.
- Integration Manager (Recommended)
- Docker deployment
- Manual deployment

!!! info "Injector tokens"
!!! info

You can use your administrator token or create another administrator service account to put in your injectors. It is not necessary to have one dedicated user for each injector.
⚠️ All external injectors must be able to access the OpenAEV API. They require 2 mandatory configuration parameters: OPENAEV_URL and OPENAEV_TOKEN. In addition, each collector has specific mandatory parameters that need to be configured.

Here is an example of a injector `docker-compose.yml` file:
```yaml
- OPENAEV_URL=http://localhost
- OPENAEV_TOKEN=ChangeMe
- INJECTOR_ID=ChangeMe # Specify a valid UUIDv4 of your choice
- "INJECTOR_NAME=HTTP query"
- INJECTOR_LOG_LEVEL=error
```
#### Integration Manager (Recommended)
The easiest way to deploy injectors is through the Integration Manager, which allows automatic deployment directly from the OpenAEV interface.

Here is an example in a injector `config.yml` file:
👉 See the [Integration Manager documentation](integration-manager/overview.md) for detailed instructions.

```yaml
openaev:
url: 'http://localhost:3001'
token: 'ChangeMe'

injector:
id: 'ChangeMe'
name: 'HTTP query'
log_level: 'info'
```

#### Networking

Be aware that all injectors are reaching RabbitMQ based the RabbitMQ configuration provided by the OpenAEV platform. The injector must be able to reach RabbitMQ on the specified hostname and port. If you have a specific Docker network configuration, please be sure to adapt your `docker-compose.yml` file in such way that the injector container gets attached to the OpenAEV Network, e.g.:

```yaml
networks:
default:
external: true
name: openaev-docker_default
```

## Docker activation

You can either directly run the Docker image of injectors or add them to your current `docker-compose.yml` file.

### Add an injector to your deployment
#### Docker Deployment
Several options are available for Docker deployment:

##### Add an injector to your existing deployment
For instance, to enable the HTTP query injector, you can add a new service to your `docker-compose.yml` file:

```docker
Expand All @@ -77,9 +50,9 @@ For instance, to enable the HTTP query injector, you can add a new service to yo
- INJECTOR_LOG_LEVEL=error
restart: always
```
Note: Injector images and available versions can be found on Docker Hub.

### Launch a standalone injector

##### Launch a standalone collector
To launch standalone injector, you can use the `docker-compose.yml` file of the injector itself. Just download the latest [release](https://github.com/OpenAEV-Platform/injectors/releases) and start the injector:

```
Expand All @@ -94,7 +67,7 @@ Change the configuration in the `docker-compose.yml` according to the parameters
$ docker compose up
```

## Manual activation
#### Manual activation

If you want to manually launch injector, you just have to install Python 3 and pip3 for dependencies:

Expand All @@ -117,12 +90,66 @@ $ pip3 install -r requirements.txt
$ cp config.yml.sample config.yml
```

Change the `config.yml` content according to the parameters of the platform and of the targeted service and launch the injector:
Change the `config.yml` content according to the parameters of the platform and of the targeted service.
For example :
```yaml
openaev:
url: 'http://localhost:3001'
token: 'ChangeMe'

injector:
id: 'ChangeMe'
name: 'HTTP query'
log_level: 'info'
```

Finally : launch the injector:

```
$ python3 openaev_http.py
```

#### Configuration

All external injectors have to be able to access the OpenAEV API. To allow this connection, they have 2 mandatory configuration parameters, the `OPENAEV_URL` and the `OPENAEV_TOKEN`. In addition to these 2 parameters, injectors have other mandatory parameters that need to be set in order to get them work.

!!! info "Injector tokens"

You can use your administrator token or create another administrator service account to put in your injectors. It is not necessary to have one dedicated user for each injector.

Here is an example of a injector `docker-compose.yml` file:
```yaml
- OPENAEV_URL=http://localhost
- OPENAEV_TOKEN=ChangeMe
- INJECTOR_ID=ChangeMe # Specify a valid UUIDv4 of your choice
- "INJECTOR_NAME=HTTP query"
- INJECTOR_LOG_LEVEL=error
```

Here is an example in a injector `config.yml` file:

```yaml
openaev:
url: 'http://localhost:3001'
token: 'ChangeMe'

injector:
id: 'ChangeMe'
name: 'HTTP query'
log_level: 'info'
```

#### Networking

Be aware that all injectors are reaching RabbitMQ based the RabbitMQ configuration provided by the OpenAEV platform. The injector must be able to reach RabbitMQ on the specified hostname and port. If you have a specific Docker network configuration, please be sure to adapt your `docker-compose.yml` file in such way that the injector container gets attached to the OpenAEV Network, e.g.:

```yaml
networks:
default:
external: true
name: openaev-docker_default
```

## Injectors status

The injector status can be displayed in the dedicated section of the platform available in Integration > injectors. You will be able to see the statistics of the RabbitMQ queue of the injector:
Expand Down
Loading