|
1 | | -## My Project |
| 1 | +# Migration Assistant for Amazon OpenSearch |
2 | 2 |
|
3 | | -TODO: Fill this README out! |
| 3 | +The Migration Assistant for Amazon OpenSearch Service solution is a toolkit designed to ease the transition to OpenSearch, facilitate upgrades to the latest OpenSearch versions, and refine cluster configurations based on observed traffic patterns. Whether you're looking to set up a proof-of-concept in AWS, transition production workloads with confidence, or enhance your current OpenSearch clusters, this guide provides references to step-by-step instructions, best practices, and insights to leverage the full potential of the OpenSearch migrations package. |
4 | 4 |
|
5 | | -Be sure to: |
| 5 | +## Table of content |
6 | 6 |
|
7 | | -* Change the title in this README |
8 | | -* Edit your repository description on GitHub |
| 7 | +- [Migration Assistant for Amazon OpenSearch](#migration-assistant-for-amazon-opensearch) |
| 8 | + - [Table of content](#table-of-content) |
| 9 | + - [Use Cases](#use-cases) |
| 10 | + - [Solution Overview](#solution-overview) |
| 11 | + - [Architecture](#architecture) |
| 12 | + - [Deployment](#deployment) |
| 13 | + - [OpenSearch Migrations Repo](#opensearch-migrations-repo) |
| 14 | + - [Collection of Operational Metrics](#collection-of-operational-metrics) |
| 15 | + - [License](#license) |
9 | 16 |
|
10 | | -## Security |
11 | 17 |
|
12 | | -See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. |
| 18 | +## Solution Overview |
| 19 | + |
| 20 | +The solution has the following features: |
| 21 | + |
| 22 | +### Historical backfill with capture and restore |
| 23 | +This solution guides users through the process of transferring data from an originating (source) cluster to a designated (target) cluster. |
| 24 | + |
| 25 | +### Live traffic capture and replay |
| 26 | +The solution offers guidance and tools to intercept traffic intended for an original cluster and archive it for future replay on a destination cluster. Typically, the replay occurs at the same rate and concurrency as the original traffic to precisely mimic the workload experienced by the source cluster. Users can choose to replay the recorded traffic subsequently or adjust the replay speed. This flexibility enables users to fine-tune the target cluster, enhancing its performance to suit their requirements. |
| 27 | + |
| 28 | +### Traffic verification |
| 29 | +The solution records requests and responses between the source and destination clusters for comparison. It then forwards the latency metrics and response codes to an analytics platform, enabling users to analyze the data essential for transitioning their traffic from a legacy system to a new Amazon OpenSearch destination. |
| 30 | + |
| 31 | +## Use Cases |
| 32 | +### Migrating historical data |
| 33 | +Migration Assistant for Amazon OpenSearch Service offers various options for migrating historical data, including detailed guidance on running a historical migration applicable across all supported migration routes, such as from Elasticsearch 7.10.2 to OpenSearch 1.0. |
| 34 | + |
| 35 | +### Near real-time migration of HTTP traffic between clusters |
| 36 | +The solution offers you the option to capture data destined for a source cluster and store this data for reuse. A user can replay this data to a target cluster in near real-time to migrate as soon as possible, or replay at a later time. |
| 37 | + |
| 38 | +### Replay traffic to multiple targets |
| 39 | +The solution allows you to capture traffic for replay through multiple instances or in sequential runs, facilitating the validation of diverse cluster workloads and configurations. |
| 40 | + |
| 41 | +### Precise simulation of your cluster workloads |
| 42 | +The solution allows users to capture and replay traffic either simultaneously with multiple instances, or in separate sequential runs. This feature aids in validating different cluster workloads and configurations. By default, the Replayer preserves the original concurrency and request rate to accurately simulate production loads, ensuring a fair like-for-like comparison. |
| 43 | + |
| 44 | +### Verify target cluster results |
| 45 | +The solution facilitates user comparisons of source and target traffic in terms of accuracy and performance. It captures metrics and logs for analysis, providing users with the necessary confidence to migrate their production traffic to a new target. |
| 46 | + |
13 | 47 |
|
14 | | -## License |
15 | 48 |
|
16 | | -This project is licensed under the Apache-2.0 License. |
| 49 | +## Architecture |
17 | 50 |
|
| 51 | +Deploying this solution with the default parameters builds the following environment in the AWS Cloud. |
| 52 | + |
| 53 | + |
| 54 | + |
| 55 | +The high-level process flow for the solution components deployed with the AWS CloudFormation template is as follows: |
| 56 | + |
| 57 | +1. Traffic is directed to the existing cluster, reaching each coordinator node. |
| 58 | +2. A Capture Proxy is added before each coordinator node in the cluster, allowing for traffic capture and storage in Amazon MSK. |
| 59 | +3. With continuous traffic capture in place, the user initiates a historical backfill. |
| 60 | +4. Following the backfill, the user replays the captured traffic using a Replayer. |
| 61 | +5. The user evaluates the outcomes from routing traffic to both the original and the new cluster. |
| 62 | +6. After confirming the new cluster’s functionality meets expectations, the user dismantles all related stacks, retaining only the new cluster’s setup. Additionally, the user can retire and discard the old cluster’s legacy infrastructure and all Migration Assistant for Amazon OpenSearch Service stacks, keeping only the new cluster. |
| 63 | + |
| 64 | + |
| 65 | +## Deployment |
| 66 | + |
| 67 | +Please follow the [Implementation Guide](https://docs.aws.amazon.com/solutions/latest/migration-assistant-for-amazon-opensearch/) to deploy the solution in your AWS account. |
| 68 | + |
| 69 | +## OpenSearch Migrations Repo |
| 70 | +This solution is open-source. The tools developed in this solution can be found in the [OpenSearch Project Migration Repo](https://github.com/opensearch-project/opensearch-migrations). |
| 71 | + |
| 72 | +Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. |
| 73 | + |
| 74 | +## Collection of Operational Metrics |
| 75 | +This solution collects anonymous operational metrics to help AWS improve the quality and features of the solution. For more information, including how to disable this capability, please see the [implementation guide](https://docs.aws.amazon.com/solutions/latest/migration-assistant-for-amazon-opensearch-service/). |
| 76 | + |
| 77 | +## License |
| 78 | +Licensed under the Apache License Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at |
| 79 | + [https://www.apache.org/licenses/](https://www.apache.org/licenses/) |
| 80 | +or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions and limitations under the License. |
0 commit comments