Skip to content

Commit 593c08e

Browse files
Roman StepaniukCroway
Roman Stepaniuk
authored and
Croway
committed
Add HTTP Streaming example
1 parent 52b8d96 commit 593c08e

File tree

24 files changed

+872
-4
lines changed

24 files changed

+872
-4
lines changed

‎README.adoc‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ readme's instructions.
2727
=== Examples
2828

2929
// examples: START
30-
Number of Examples: 62 (0 deprecated)
30+
Number of Examples: 63 (0 deprecated)
3131

3232
[width="100%",cols="4,2,4",options="header"]
3333
|===
@@ -96,7 +96,6 @@ Number of Examples: 62 (0 deprecated)
9696
| link:fhir/readme.adoc[Fhir] (fhir) | Health Care | An example showing how to work with Camel, FHIR and Spring Boot
9797

9898
| link:fhir-auth-tx/readme.adoc[Fhir Auth Tx] (fhir-auth-tx) | Health Care | An example showing how to work with Camel, FHIR Authorization, FHIR Transaction and Spring Boot
99-
10099

101100
| link:validator/readme.adoc[Validator Spring Boot] (validator) | Input/Output Type Contract | An example showing how to work with declarative validation and Spring Boot
102101

@@ -112,7 +111,6 @@ Number of Examples: 62 (0 deprecated)
112111
| link:metrics/README.adoc[Metrics] (metrics) | Management and Monitoring | An example showing how to work with Camel and Spring Boot and report metrics to Graphite
113112

114113
| link:observation/README.adoc[Micrometer Observation] (observation) | Management and Monitoring | An example showing how to trace incoming and outgoing messages from Camel with Micrometer Observation
115-
116114

117115
| link:opentelemetry/README.adoc[OpenTelemetry] (opentelemetry) | Management and Monitoring | An example showing how to use Camel with OpenTelemetry
118116

@@ -139,14 +137,15 @@ Number of Examples: 62 (0 deprecated)
139137
| link:widget-gadget/README.adoc[Widget Gadget] (widget-gadget) | Messaging | The widget and gadget example from EIP book, running on Spring Boot
140138

141139
| link:reactive-streams/readme.adoc[Reactive Streams] (reactive-streams) | Reactive | An example that shows how Camel can exchange data using reactive streams with Spring Boot reactor
142-
143140

144141
| link:http-ssl/README.adoc[Http Ssl] (http-ssl) | Rest | An example showing the Camel HTTP component with Spring Boot and SSL
145142

146143
| link:openapi-contract-first/readme.adoc[Openapi Contract First] (openapi-contract-first) | Rest | Contract First OpenAPI example
147144

148145
| link:platform-http/README.adoc[Platform Http] (platform-http) | Rest | An example showing Camel REST DSL with platform HTTP
149146

147+
| link:http-streaming/README.adoc[Http Streaming] (http-streaming) | Rest | This example shows how to stream large files using platform HTTP component
148+
150149
| link:rest-cxf/README.adoc[Rest Cxf] (rest-cxf) | Rest | An example showing Camel REST using CXF with Spring Boot
151150

152151
| link:rest-openapi/README.adoc[Rest Openapi] (rest-openapi) | Rest | An example showing Camel REST DSL and OpenApi with Spring Boot

‎http-streaming/README.adoc‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
== Camel Spring Boot Example: HTTP Large File Streaming
2+
3+
4+
=== Introduction
5+
6+
This project contains logic to handle large HTTP data streams in download, upload, and proxy scenarios. The aim is to show Camel's ability to process streams without exhausting JVM memory.
7+
8+
=== Use cases
9+
10+
[NOTE]
11+
====
12+
Streaming mode opens up many interesting use cases, for example, data transformation on very large data structures, applying selective/discarding rules, split/merging of data, multiplexing data streams, and others.
13+
====
14+
15+
This example only covers the basics of enabling streaming mode. The two implemented scenarios using Camel in streaming mode are:
16+
17+
- HTTP data downloads
18+
- HTTP data uploads
19+
20+
Camel may act as the end system responsible to locally/remotely store the data stream. Or it may also act as a proxy system, passing the responsibility downstream.
21+
22+
The critical data handling happens in the numbered 1) and 2) positions illustrated below.
23+
24+
=== Download scenario
25+
26+
image::docs/images/uc-download.png[]
27+
28+
=== Upload scenario
29+
30+
image::docs/images/uc-upload.png[]
31+
32+
33+
=== How to run
34+
35+
To demonstrate Camel can handle larger data streams than memory allocated to the JVM, we need to start it with low memory settings.
36+
37+
To run it follow the commands below:
38+
39+
Follow these steps to run the example:
40+
41+
1. Decide which scenario you want to run: `download` or `upload`.
42+
43+
2. Navigate to the corresponding folder and run both the backend and proxy servers with low memory settings (make sure ports 8080 and 9000 are available):
44+
45+
mvn spring-boot:run -Dspring-boot.run.jvmArguments="-Xms200m -Xmx200m"
46+
47+
3. From the `client` directory, send a request using a large data stream.
48+
(See detailed instructions in `client/Readme.adoc`.)
49+
50+
4. Stop Camel Spring Boot and inspect the result file.
51+
52+
Camel Spring Boot should process the HTTP byte stream and dump it in a file in the 'client' directory.
53+
54+
=== Help and contributions
55+
56+
If you hit any problem using Camel or have some feedback, then please
57+
https://camel.apache.org/community/support/[let us know].
58+
59+
We also love contributors, so
60+
https://camel.apache.org/community/contributing/[get involved] :-)
61+
62+
The Camel riders!

‎http-streaming/docs/images/uc-download.png‎

207 KB
Loading

‎http-streaming/docs/images/uc-upload.png‎

208 KB
Loading

‎http-streaming/download/backend-server/pom.xml‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<!--
3+
4+
Licensed to the Apache Software Foundation (ASF) under one or more
5+
contributor license agreements. See the NOTICE file distributed with
6+
this work for additional information regarding copyright ownership.
7+
The ASF licenses this file to You under the Apache License, Version 2.0
8+
(the "License"); you may not use this file except in compliance with
9+
the License. You may obtain a copy of the License at
10+
11+
http://www.apache.org/licenses/LICENSE-2.0
12+
13+
Unless required by applicable law or agreed to in writing, software
14+
distributed under the License is distributed on an "AS IS" BASIS,
15+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16+
See the License for the specific language governing permissions and
17+
limitations under the License.
18+
19+
-->
20+
<project xmlns="http://maven.apache.org/POM/4.0.0"
21+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
22+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
23+
<modelVersion>4.0.0</modelVersion>
24+
25+
<parent>
26+
<groupId>org.apache.camel.springboot.example</groupId>
27+
<artifactId>camel-example-spring-boot-http-streaming</artifactId>
28+
<version>4.14.0-SNAPSHOT</version>
29+
<relativePath>../../pom.xml</relativePath>
30+
</parent>
31+
32+
<artifactId>camel-example-spring-boot-http-streaming-download-server</artifactId>
33+
<name>Camel SB Examples :: HTTP Streaming :: Spring Boot server</name>
34+
<description>Backend Server using platform-http component</description>
35+
36+
<properties>
37+
<category>Rest</category>
38+
39+
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
40+
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
41+
</properties>
42+
43+
<dependencies>
44+
<!-- Camel -->
45+
<dependency>
46+
<groupId>org.apache.camel.springboot</groupId>
47+
<artifactId>camel-spring-boot-starter</artifactId>
48+
</dependency>
49+
<dependency>
50+
<groupId>org.apache.camel.springboot</groupId>
51+
<artifactId>camel-platform-http-starter</artifactId>
52+
</dependency>
53+
</dependencies>
54+
55+
<build>
56+
<plugins>
57+
<plugin>
58+
<groupId>org.springframework.boot</groupId>
59+
<artifactId>spring-boot-maven-plugin</artifactId>
60+
<version>${spring-boot-version}</version>
61+
<executions>
62+
<execution>
63+
<goals>
64+
<goal>repackage</goal>
65+
</goals>
66+
</execution>
67+
</executions>
68+
</plugin>
69+
</plugins>
70+
</build>
71+
</project>

‎http-streaming/download/backend-server/src/main/java/org/apache/camel/springboot/example/http/streaming/HttpStreamingCamelServerApplication.java‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
package org.apache.camel.springboot.example.http.streaming;
18+
19+
import org.springframework.boot.SpringApplication;
20+
import org.springframework.boot.autoconfigure.SpringBootApplication;
21+
22+
@SpringBootApplication
23+
public class HttpStreamingCamelServerApplication {
24+
25+
public static void main(String[] args) {
26+
SpringApplication.run(HttpStreamingCamelServerApplication.class, args);
27+
}
28+
}

‎http-streaming/download/backend-server/src/main/java/org/apache/camel/springboot/example/http/streaming/HttpStreamingCamelServerRouter.java‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
package org.apache.camel.springboot.example.http.streaming;
18+
19+
import org.apache.camel.builder.RouteBuilder;
20+
import org.springframework.stereotype.Component;
21+
22+
import java.io.File;
23+
24+
@Component
25+
public class HttpStreamingCamelServerRouter extends RouteBuilder {
26+
@Override
27+
public void configure() throws Exception {
28+
restConfiguration()
29+
.component("platform-http");
30+
rest()
31+
.get("/test")
32+
.to("direct:getRequest");
33+
34+
from("direct:getRequest")
35+
.process(exchange -> {
36+
File file = new File("../client/input");
37+
exchange.getMessage().setBody(file);
38+
})
39+
.log("responding with content.");
40+
}
41+
}
42+

‎http-streaming/download/backend-server/src/main/resources/application.properties‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
#
2+
# Licensed to the Apache Software Foundation (ASF) under one or more
3+
# contributor license agreements. See the NOTICE file distributed with
4+
# this work for additional information regarding copyright ownership.
5+
# The ASF licenses this file to You under the Apache License, Version 2.0
6+
# (the "License"); you may not use this file except in compliance with
7+
# the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
#
17+
18+
camel.main.name=backend-camel-server
19+
20+
# Backend port
21+
server.port=8080

‎http-streaming/download/client/Readme.adoc‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
== Introduction
2+
3+
This client sends a download request via HTTP (empty request). The server is expected to send back a large data stream. We want to test the server's ability to handle large data responses.
4+
5+
6+
== Prepare the response data
7+
8+
The server will read and stream back the response data.
9+
In the client folder create a large file by running the following command:
10+
11+
dd if=/dev/zero of=input bs=4M count=1024
12+
13+
14+
The command above creates a 4 Gigabytes (input) file in the `client` directory.
15+
You can change the parameters `bs` and `count` to configure the data size.
16+
17+
18+
== Send a request
19+
20+
Use `curl` to send a request:
21+
22+
curl -v -o ./output http://localhost:8080/test
23+
24+
If you're using the SpringBoot proxy, use the proxy port:
25+
26+
curl -v -o ./output http://localhost:9000/test
27+
28+
29+
== Inspect the response
30+
31+
If the request was successful, curl should have written the response data locally.
32+
Check the `output` file size equals the `input` file
33+
34+
35+
== Clean the data
36+
37+
When you're done, make sure you perform the following actions:
38+
39+
1) Delete the large sample data file to clean your filesystem.
40+
41+
2) Delete the output file generated by Camel.

‎http-streaming/download/proxy-server/pom.xml‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<!--
3+
4+
Licensed to the Apache Software Foundation (ASF) under one or more
5+
contributor license agreements. See the NOTICE file distributed with
6+
this work for additional information regarding copyright ownership.
7+
The ASF licenses this file to You under the Apache License, Version 2.0
8+
(the "License"); you may not use this file except in compliance with
9+
the License. You may obtain a copy of the License at
10+
11+
http://www.apache.org/licenses/LICENSE-2.0
12+
13+
Unless required by applicable law or agreed to in writing, software
14+
distributed under the License is distributed on an "AS IS" BASIS,
15+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16+
See the License for the specific language governing permissions and
17+
limitations under the License.
18+
19+
-->
20+
<project xmlns="http://maven.apache.org/POM/4.0.0"
21+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
22+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
23+
<modelVersion>4.0.0</modelVersion>
24+
25+
<parent>
26+
<groupId>org.apache.camel.springboot.example</groupId>
27+
<artifactId>camel-example-spring-boot-http-streaming</artifactId>
28+
<version>4.14.0-SNAPSHOT</version>
29+
<relativePath>../../pom.xml</relativePath>
30+
</parent>
31+
32+
<artifactId>camel-example-spring-boot-http-streaming-download-proxy-server</artifactId>
33+
<name>Camel SB Examples :: HTTP Streaming :: Spring Boot Proxy server</name>
34+
<description>Proxy Server using platform-http component</description>
35+
36+
<properties>
37+
<category>Rest</category>
38+
39+
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
40+
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
41+
</properties>
42+
43+
<dependencies>
44+
<!-- Camel -->
45+
<dependency>
46+
<groupId>org.apache.camel.springboot</groupId>
47+
<artifactId>camel-spring-boot-starter</artifactId>
48+
</dependency>
49+
<dependency>
50+
<groupId>org.apache.camel.springboot</groupId>
51+
<artifactId>camel-platform-http-starter</artifactId>
52+
</dependency>
53+
<dependency>
54+
<groupId>org.apache.camel.springboot</groupId>
55+
<artifactId>camel-http-starter</artifactId>
56+
</dependency>
57+
</dependencies>
58+
59+
<build>
60+
<plugins>
61+
<plugin>
62+
<groupId>org.springframework.boot</groupId>
63+
<artifactId>spring-boot-maven-plugin</artifactId>
64+
<version>${spring-boot-version}</version>
65+
<executions>
66+
<execution>
67+
<goals>
68+
<goal>repackage</goal>
69+
</goals>
70+
</execution>
71+
</executions>
72+
</plugin>
73+
</plugins>
74+
</build>
75+
</project>

0 commit comments

Comments
 (0)