Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
12 changes: 10 additions & 2 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1,9 +1,15 @@
# Build outputs — regenerated inside the image, never shipped from context.
dist/
dist-connector/
build/
**/build/

# VCS
# VCS and gradle caches (the wrapper is not used; system gradle lives in the
# builder image).
.git/
.gitignore
.gradle/
**/.gradle/

# Non-Java SDKs (not included in Gradle build)
eventmesh-sdks/eventmesh-sdk-rust/
Expand All @@ -14,10 +20,12 @@ eventmesh-sdks/eventmesh-sdk-c/
**/target/
**/node_modules/

# IDE
# IDE / editor cruft.
.idea/
.vscode/
*.iml
*.ipr
*.iws

# Docs & misc
docs/
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
fail-fast: false
matrix:
os: [ ubuntu-latest, macOS-latest ]
java: [ 8, 11 ]
java: [ 21 ]
language: [ 'java' ]
runs-on: ${{ matrix.os }}

Expand All @@ -49,11 +49,11 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e #v6.1.0

- name: Set up JDK 11
- name: Set up JDK 21
uses: actions/setup-java@v5
with:
distribution: 'zulu'
java-version: 11
java-version: 21

- name: GenerateGrammarSource
run: ./gradlew clean generateGrammarSource --parallel --daemon --scan
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/code-scanning.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,12 +51,12 @@ jobs:
# queries: ./path/to/local/query, your-org/your-repo/queries@main
languages: ${{ matrix.language }}

- name: Set up JDK 11
- name: Set up JDK 21
if: matrix.language == 'java'
uses: actions/setup-java@v5
with:
distribution: 'zulu'
java-version: 11
java-version: 21

- name: Setup Gradle
if: matrix.language == 'java'
Expand Down
17 changes: 13 additions & 4 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,16 @@ on:
jobs:
docker:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
# EventMesh unified runtime (traffic HTTP + admin).
- image: apache/eventmesh
dockerfile: ./docker/Dockerfile
# Connector-runtime (separate process, talks HTTP+CloudEvents to the runtime).
- image: apache/eventmesh-connector
dockerfile: ./docker/Dockerfile.connector
steps:
- name: Checkout repository
uses: actions/checkout@v6
Expand All @@ -37,14 +47,13 @@ jobs:
id: meta
uses: docker/metadata-action@v5
with:
images: |
apache/eventmesh
images: ${{ matrix.image }}

- name: Build and push
uses: docker/build-push-action@v6
with:
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
file: ./docker/Dockerfile_jdk8
context: ./
file: ${{ matrix.dockerfile }}
context: ./
4 changes: 2 additions & 2 deletions .github/workflows/license.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,11 @@ jobs:
- name: Check license header
uses: apache/skywalking-eyes@main

- name: Set up JDK 11
- name: Set up JDK 21
uses: actions/setup-java@v5
with:
distribution: 'zulu'
java-version: 11
java-version: 21

- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e #v6.1.0
Expand Down
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,15 @@ x86/
[Aa][Rr][Mm]64/
bld/
[Bb]in/
# … but keep source service scripts shipped from module bin/ dirs (dist copies them to dist/bin).
!eventmesh-runtime/bin/
!eventmesh-connector-runtime/bin/
[Oo]bj/
[Ll]og/
[Ll]ogs/
**/org/apache/eventmesh/connector/jdbc/antlr4/autogeneration/*

#rust
Cargo.lock
rocketmq-pull-offsets.properties
kafka-pull-offsets.properties
169 changes: 110 additions & 59 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
<div align="center">

<br /><br />
<img src="resources/logo.png" width="256">
<img src="resources/logo.svg" width="256">
<br />

[![CI status](https://img.shields.io/github/actions/workflow/status/apache/eventmesh/ci.yml?logo=github&style=for-the-badge)](https://github.com/apache/eventmesh/actions/workflows/ci.yml)
[![CodeCov](https://img.shields.io/codecov/c/gh/apache/eventmesh/master?logo=codecov&style=for-the-badge)](https://codecov.io/gh/apache/eventmesh)
[![Code Quality: Java](https://img.shields.io/lgtm/grade/java/g/apache/eventmesh.svg?logo=lgtm&logoWidth=18&style=for-the-badge)](https://lgtm.com/projects/g/apache/eventmesh/context:java)
[![Total Alerts](https://img.shields.io/lgtm/alerts/g/apache/eventmesh.svg?logo=lgtm&logoWidth=18&style=for-the-badge)](https://lgtm.com/projects/g/apache/eventmesh/alerts/)
[![Code Scanning](https://img.shields.io/github/actions/workflow/status/apache/eventmesh/code-scanning.yml?label=code%20scanning&logo=github&style=for-the-badge)](https://github.com/apache/eventmesh/actions/workflows/code-scanning.yml)

[![License](https://img.shields.io/github/license/apache/eventmesh?style=for-the-badge)](https://www.apache.org/licenses/LICENSE-2.0.html)
[![GitHub Release](https://img.shields.io/github/v/release/apache/eventmesh?style=for-the-badge)](https://github.com/apache/eventmesh/releases)
Expand All @@ -27,27 +26,33 @@

### EventMesh Architecture

![EventMesh Architecture](resources/eventmesh-architecture-6.png)
![EventMesh Architecture](resources/eventmesh-architecture-7.png)

EventMesh adopts a unified **CloudEvents-over-MQ** architecture. The message queue (MQ) acts as a pure write-ahead log (WAL) for durable storage only — there are no consumer groups, no tags, and no broker-side subscription semantics. Instead, the stateless EventMesh Runtime owns all delivery logic: its **SubscriptionManager** maintains the subscription registry and offset tracking, and dispatches events with load-balance, broadcast, and multicast semantics. Applications interact through a lightweight **HTTP + CloudEvents 1.0** SDK (`publish` / `subscribe` / `unsubscribe`), while integration with external systems runs in a standalone Connector Runtime via the connector SPI.

## Features

Apache EventMesh has a vast amount of features to help users achieve their goals. Let us share with you some of the key features EventMesh has to offer:
Apache EventMesh is packed with features that help users build event-driven applications with ease. Here are the highlights that set EventMesh apart:

**Core architecture**

- Built around the [CloudEvents](https://cloudevents.io) specification.
- Rapidty extendsible interconnector layer [connectors](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors) using [openConnect](https://github.com/apache/eventmesh/tree/master/eventmesh-openconnect) such as the source or sink of Saas, CloudService, and Database etc.
- Rapidty extendsible storage layer such as [Apache RocketMQ](https://rocketmq.apache.org), [Apache Kafka](https://kafka.apache.org), [Apache Pulsar](https://pulsar.apache.org), [RabbitMQ](https://rabbitmq.com), [Redis](https://redis.io).
- Rapidty extendsible meta such as [Consul](https://consulproject.org/en/), [Nacos](https://nacos.io), [ETCD](https://etcd.io) and [Zookeeper](https://zookeeper.apache.org/).
- Guaranteed at-least-once delivery.
- Deliver events between multiple EventMesh deployments.
- Event schema management by catalog service.
- Powerful event orchestration by [Serverless workflow](https://serverlessworkflow.io/) engine.
- Powerful event filtering and transformation.
- Rapid, seamless scalability.
- Easy Function develop and framework integration.
- **CloudEvents-native, end to end** — built entirely around the [CloudEvents](https://cloudevents.io) 1.0 specification, so events stay vendor-neutral and portable.
- **Lightweight, language-agnostic SDK** — just three operations over plain HTTP (`publish`, `subscribe`, `unsubscribe`); no heavyweight client, no vendor lock-in.
- **Runtime-owned subscription & dispatch** — subscription state and delivery semantics (load-balance / broadcast / multicast) are managed by EventMesh itself, not the underlying MQ, giving you consistent behavior across any storage backend.
- **MQ as a pure write-ahead log (WAL)** — append-only, no consumer groups, no tags; the broker is reduced to durable storage, dramatically simplifying operations.
- **Guaranteed at-least-once delivery** — EventMesh owns reliability through self-managed offsets and explicit ACK.
- **Multiple delivery transports** — subscribers choose HTTP long-polling, Server-Sent Events (SSE), or WebSocket push, with request-reply support.
- **Effortless horizontal scaling** — stateless Runtime instances scale out seamlessly with no rebalancing cost.

## Roadmap
**Extensibility & ecosystem**

Please go to the [roadmap](https://eventmesh.apache.org/docs/roadmap) to get the release history and new features of Apache EventMesh.
- **Agent-to-Agent (A2A) collaboration** — a built-in [A2A protocol](docs/a2a-protocol/README_EN.md) turns EventMesh into an agent collaboration bus, bridging synchronous MCP / JSON-RPC 2.0 tool calls and asynchronous event-driven pub/sub for LLM and multi-agent systems.
- **Pluggable storage layer** — [Apache RocketMQ](https://rocketmq.apache.org), [Apache Kafka](https://kafka.apache.org), [Apache Pulsar](https://pulsar.apache.org), [RabbitMQ](https://rabbitmq.com), [Redis](https://redis.io), and more.
- **Pluggable interconnector layer** — [connectors](https://github.com/apache/eventmesh/tree/master/eventmesh-connectors) run as standalone processes acting as the source or sink of SaaS, CloudService, Database, etc.
- **Pluggable meta service** — [Consul](https://consulproject.org/en/), [Nacos](https://nacos.io), [ETCD](https://etcd.io), and [Zookeeper](https://zookeeper.apache.org/).
- **Event schema management** via catalog service.
- **Powerful event orchestration** through the [Serverless workflow](https://serverlessworkflow.io/) engine.
- **Powerful event filtering and transformation.**

## Subprojects

Expand Down Expand Up @@ -76,60 +81,106 @@ sudo docker pull apache/eventmesh:latest
Use the following command to start the EventMesh container:

```shell
sudo docker run -d --name eventmesh -p 10000:10000 -p 10105:10105 -p 10205:10205 -p 10106:10106 -t apache/eventmesh:latest
sudo docker run -d --name eventmesh -p 8080:8080 -p 8081:8081 -p 8082:8082 -t apache/eventmesh:latest
```
#### 3. Creating Topics
Creating a topic is the first step in using EventMesh. You need to send an HTTP POST request to create a topic.
Example Request

> Ports: `8080` = traffic HTTP (`/events/*` CloudEvents API), `8081` = admin HTTP (`/admin/*`), `8082` = WebSocket push (opt-in). The connector runtime image exposes `8083` for its optional admin port.
#### 3. Publishing a CloudEvent

Applications talk to EventMesh over the `/events/*` HTTP endpoints using standard [CloudEvents](https://cloudevents.io) 1.0. Publish an event with an HTTP POST (`202 Accepted` means the event was written to the WAL):

```shell
POST /eventmesh/topic/create HTTP/1.1
Host: localhost:10105
Content-Type: application/json

{
"topic": "example-topic"
POST /events/publish HTTP/1.1
Host: localhost:8080
Content-Type: application/cloudevents+json

{
"specversion": "1.0",
"id": "89010a5a-3c6f-4a1e-9b2d-0f7c1f2e3a4b",
"source": "/example/producer",
"type": "com.example.order.created",
"subject": "orders",
"datacontenttype": "application/json",
"data": {
"content": "Hello, EventMesh!"
}
}
```
#### 4. Subscribing to Topics
After creating a topic, you can subscribe to it to receive messages. EventMesh provides two subscription methods: local subscription and remote subscription.

#### 4. Subscribing to a Topic

Subscriptions are registered with EventMesh (there are no consumer groups or tags). Provide a `clientId`, the topic, and a distribution `mode` (`LOAD_BALANCE`, `BROADCAST`, `MULTICAST`, or `LOAD_BALANCE_STICKY`). The response returns a `subscriptionId`:

```shell
POST /eventmesh/subscribe/local HTTP/1.1
Host: localhost:10105
Content-Type: application/json
{
"url": "http://localhost:8080/callback",
"consumerGroup": "example-consumer-group",
"topic": [
{
"topic": "example-topic",
"mode": "CLUSTERING",
"type": "SYNC"
}
]
POST /events/subscribe HTTP/1.1
Host: localhost:8080
Content-Type: application/json

{
"clientId": "order-svc",
"topic": "orders",
"mode": "LOAD_BALANCE"
}
```
#### 5. Sending Messages
EventMesh provides multiple message sending methods, including asynchronous sending, synchronous sending, and batch sending.

#### 5. Receiving Events

Subscribers can receive dispatched events through three transports. Pick whichever fits your workload — all of them deliver the same CloudEvents and honor EventMesh's ACK / at-least-once semantics.

**a) HTTP Long-Polling** — pull events; the request blocks until events arrive or the timeout elapses:

```shell
POST /eventmesh/publish HTTP/1.1
Host: localhost:10105
Content-Type: application/json
eventmesh-message-topic: example-topic
{
"content": "Hello, EventMesh!"
GET /events/poll?clientId=order-svc&topics=orders&timeout=30000 HTTP/1.1
Host: localhost:8080
```

After processing the delivered events, acknowledge them so the offset advances (at-least-once delivery):

```shell
POST /events/ack HTTP/1.1
Host: localhost:8080
Content-Type: application/json

{
"subId": "sub-123",
"clientId": "order-svc",
"topic": "orders",
"partition": 0,
"offset": 42
}
```

**b) Server-Sent Events (SSE)** — server push over a long-lived HTTP connection; the client does not poll:

```shell
GET /events/stream?clientId=order-svc&topics=orders HTTP/1.1
Host: localhost:8080
Accept: text/event-stream
```

**c) WebSocket** — full-duplex server push over a dedicated WebSocket port:

```shell
GET /events/stream HTTP/1.1
Host: localhost:8082
Upgrade: websocket
Connection: Upgrade
```

> Long-polling, SSE, and WebSocket are interchangeable delivery transports — a subscriber chooses one. SSE and WebSocket are pushed by the server (no polling loop), while long-polling is client-driven. Request-reply (`POST /events/request` + `POST /events/reply`) is also supported. The `CloudEventsClient` Java SDK wraps all of these (`subscribe` / `subscribeSse` / `subscribeWs`); see the [CloudEvents client guide](docs/eventmesh-cloudevents-client-guide.md).

#### 6. Unsubscribing
When you no longer need to receive messages for a topic, you can unsubscribe.

When you no longer need to receive events for a topic, unsubscribe by `clientId` (optionally with the `subscriptionId`):

```shell
POST /eventmesh/unsubscribe/local HTTP/1.1
Host: localhost:10105
Content-Type: application/json
{
"url": "http://localhost:8080/callback",
"consumerGroup": "example-consumer-group",
"topics": ["example-topic"]
POST /events/unsubscribe HTTP/1.1
Host: localhost:8080
Content-Type: application/json

{
"clientId": "order-svc",
"topic": "orders"
}
```
## Contributing
Expand Down
Loading
Loading