Skip to content

Commit 1d70e58

Browse files
committed
Added v1.0.0 Artifacts
Signed-off-by: Brian Presley <[email protected]>
1 parent 7025c8e commit 1d70e58

21 files changed

+733
-21
lines changed

CHANGELOG.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# Change Log
2+
3+
All notable changes to this project will be documented in this file.
4+
5+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7+
8+
## [1.0.0] - 2023-11-17
9+
10+
### Added
11+
12+
- All files, initial release

CODE_OF_CONDUCT.md

100644100755
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
## Code of Conduct
22
This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
33
For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
4-
[email protected] with any additional questions or comments.
4+
[email protected] with any additional questions or comments.

CONTRIBUTING.md

100644100755
Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ information to effectively respond to your bug report or contribution.
1111

1212
We welcome you to use the GitHub issue tracker to report bugs or suggest features.
1313

14-
When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already
14+
When filing an issue, please check [existing open](https://github.com/opensearch-project/opensearch-migrations), or [recently closed](https://github.com/opensearch-project/opensearch-migrations/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already
1515
reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
1616

1717
* A reproducible test case or series of steps
@@ -23,25 +23,26 @@ reported the issue. Please try to include as much information as you can. Detail
2323
## Contributing via Pull Requests
2424
Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
2525

26-
1. You are working against the latest source on the *main* branch.
26+
1. You are working against the latest source on the *master* branch.
2727
2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
2828
3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
2929

3030
To send us a pull request, please:
3131

3232
1. Fork the repository.
3333
2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
34-
3. Ensure local tests pass.
35-
4. Commit to your fork using clear commit messages.
36-
5. Send us a pull request, answering any default questions in the pull request interface.
37-
6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
34+
3. Ensure all build processes execute successfully (see README.md for additional guidance).
35+
4. Ensure all unit, integration, and/or snapshot tests pass, as applicable.
36+
5. Commit to your fork using clear commit messages.
37+
6. Send us a pull request, answering any default questions in the pull request interface.
38+
7. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
3839

3940
GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
4041
[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
4142

4243

4344
## Finding contributions to work on
44-
Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start.
45+
Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/awslabs/opensearch-migrations/labels/help%20wanted) issues is a great place to start.
4546

4647

4748
## Code of Conduct
@@ -51,9 +52,11 @@ [email protected] with any additional questions or comments.
5152

5253

5354
## Security issue notifications
54-
If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
55+
If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public GitHub issue.
5556

5657

5758
## Licensing
5859

59-
See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
60+
See the [LICENSE](https://github.com/aws-solutions/migration-assistant-for-amazon-opensearch/blob/main/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
61+
62+
We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.

LICENSE renamed to LICENSE.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -172,4 +172,4 @@
172172
of any other Contributor, and only if You agree to indemnify,
173173
defend, and hold each Contributor harmless for any liability
174174
incurred by, or claims asserted against, such Contributor by reason
175-
of your accepting any such warranty or additional liability.
175+
of your accepting any such warranty or additional liability.

NOTICE

Lines changed: 0 additions & 1 deletion
This file was deleted.

NOTICE.txt

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
Migration Assistant for Amazon OpenSearch Service
2+
3+
Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
4+
Licensed under the Apache License Version 2.0 (the "License"). You may not use this file except
5+
in compliance with the License. A copy of the License is located at http://www.apache.org/licenses/
6+
or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS,
7+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied. See the License for the
8+
specific language governing permissions and limitations under the License.
9+
10+
**********************
11+
THIRD PARTY COMPONENTS
12+
**********************
13+
This software includes third party software subject to the following copyrights:
14+
15+
AWS SDK under the Apache License Version 2.0
16+
source-map-support under the MIT License
17+
attrs under the MIT License
18+
boolean.py under the BSD-2-Clause
19+
coverage under the Apache License Version 2.0
20+
iniconfig under the MIT License
21+
jsondiff under the MIT License
22+
license-expression under the Apache License Version 2.0
23+
pluggy under the MIT License
24+
prometheus-client under the Apache License Version 2.0
25+
pytest under the MIT License
26+
responses under the Apache License Version 2.0
27+
types-PyYAML under the Apache License Version 2.0
28+
@aws-cdk/aws-servicecatalogappregistry-alpha under the Apache License Version 2.0
29+
Jinja2 under the BSD License (BSD-3-Clause)
30+
MarkupSafe under the BSD License (BSD-3-Clause)
31+
Werkzeug under the BSD License
32+
awscli under the Apache License Version 2.0
33+
boto3 under the Apache License Version 2.0
34+
botocore under the Apache License Version 2.0
35+
cffi under the MIT License
36+
click under the BSD 3-Clause "New"
37+
colorama under the BSD License
38+
cryptography under the BSD License
39+
docutils under the BSD License
40+
jmespath under MIT License (MIT)
41+
jsonpath-ng under Apache Software License Apache 2.0
42+
moto under Apache Software License (Apache License 2.0)
43+
ply under the BSD License
44+
pur under the BSD License
45+
pyasn1 under the BSD License (BSD-2-Clause)
46+
pycparser under the BSD License (BSD)
47+
python-dateutil under the Apache Software License, BSD License (Dual License)
48+
requests-aws under the BSD License
49+
requests-aws4auth is under MIT License (MIT License)
50+
rsa under the Apache Software License (Apache-2.0)
51+
s3transfer under the Apache Software License (Apache License 2.0)
52+
xmltodict under the MIT License (MIT)

README.md

100644100755
Lines changed: 72 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,80 @@
1-
## My Project
1+
# Migration Assistant for Amazon OpenSearch
22

3-
TODO: Fill this README out!
3+
The Migration Assistant for Amazon OpenSearch Service solution is a toolkit designed to ease the transition to OpenSearch, facilitate upgrades to the latest OpenSearch versions, and refine cluster configurations based on observed traffic patterns. Whether you're looking to set up a proof-of-concept in AWS, transition production workloads with confidence, or enhance your current OpenSearch clusters, this guide provides references to step-by-step instructions, best practices, and insights to leverage the full potential of the OpenSearch migrations package.
44

5-
Be sure to:
5+
## Table of content
66

7-
* Change the title in this README
8-
* Edit your repository description on GitHub
7+
- [Migration Assistant for Amazon OpenSearch](#migration-assistant-for-amazon-opensearch)
8+
- [Table of content](#table-of-content)
9+
- [Use Cases](#use-cases)
10+
- [Solution Overview](#solution-overview)
11+
- [Architecture](#architecture)
12+
- [Deployment](#deployment)
13+
- [OpenSearch Migrations Repo](#opensearch-migrations-repo)
14+
- [Collection of Operational Metrics](#collection-of-operational-metrics)
15+
- [License](#license)
916

10-
## Security
1117

12-
See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
18+
## Solution Overview
19+
20+
The solution has the following features:
21+
22+
### Historical backfill with capture and restore
23+
This solution guides users through the process of transferring data from an originating (source) cluster to a designated (target) cluster.
24+
25+
### Live traffic capture and replay
26+
The solution offers guidance and tools to intercept traffic intended for an original cluster and archive it for future replay on a destination cluster. Typically, the replay occurs at the same rate and concurrency as the original traffic to precisely mimic the workload experienced by the source cluster. Users can choose to replay the recorded traffic subsequently or adjust the replay speed. This flexibility enables users to fine-tune the target cluster, enhancing its performance to suit their requirements.
27+
28+
### Traffic verification
29+
The solution records requests and responses between the source and destination clusters for comparison. It then forwards the latency metrics and response codes to an analytics platform, enabling users to analyze the data essential for transitioning their traffic from a legacy system to a new Amazon OpenSearch destination.
30+
31+
## Use Cases
32+
### Migrating historical data
33+
Migration Assistant for Amazon OpenSearch Service offers various options for migrating historical data, including detailed guidance on running a historical migration applicable across all supported migration routes, such as from Elasticsearch 7.10.2 to OpenSearch 1.0.
34+
35+
### Near real-time migration of HTTP traffic between clusters
36+
The solution offers you the option to capture data destined for a source cluster and store this data for reuse. A user can replay this data to a target cluster in near real-time to migrate as soon as possible, or replay at a later time.
37+
38+
### Replay traffic to multiple targets
39+
The solution allows you to capture traffic for replay through multiple instances or in sequential runs, facilitating the validation of diverse cluster workloads and configurations.
40+
41+
### Precise simulation of your cluster workloads
42+
The solution allows users to capture and replay traffic either simultaneously with multiple instances, or in separate sequential runs. This feature aids in validating different cluster workloads and configurations. By default, the Replayer preserves the original concurrency and request rate to accurately simulate production loads, ensuring a fair like-for-like comparison.
43+
44+
### Verify target cluster results
45+
The solution facilitates user comparisons of source and target traffic in terms of accuracy and performance. It captures metrics and logs for analysis, providing users with the necessary confidence to migrate their production traffic to a new target.
46+
1347

14-
## License
1548

16-
This project is licensed under the Apache-2.0 License.
49+
## Architecture
1750

51+
Deploying this solution with the default parameters builds the following environment in the AWS Cloud.
52+
53+
![Architecture](AWSSolutionsArchitecture.png)
54+
55+
The high-level process flow for the solution components deployed with the AWS CloudFormation template is as follows:
56+
57+
1. Traffic is directed to the existing cluster, reaching each coordinator node.
58+
2. A Capture Proxy is added before each coordinator node in the cluster, allowing for traffic capture and storage in Amazon MSK.
59+
3. With continuous traffic capture in place, the user initiates a historical backfill.
60+
4. Following the backfill, the user replays the captured traffic using a Replayer.
61+
5. The user evaluates the outcomes from routing traffic to both the original and the new cluster.
62+
6. After confirming the new cluster’s functionality meets expectations, the user dismantles all related stacks, retaining only the new cluster’s setup. Additionally, the user can retire and discard the old cluster’s legacy infrastructure and all Migration Assistant for Amazon OpenSearch Service stacks, keeping only the new cluster.
63+
64+
65+
## Deployment
66+
67+
Please follow the [Implementation Guide](https://docs.aws.amazon.com/solutions/latest/migration-assistant-for-amazon-opensearch/) to deploy the solution in your AWS account.
68+
69+
## OpenSearch Migrations Repo
70+
This solution is open-source. The tools developed in this solution can be found in the [OpenSearch Project Migration Repo](https://github.com/opensearch-project/opensearch-migrations).
71+
72+
Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
73+
74+
## Collection of Operational Metrics
75+
This solution collects anonymous operational metrics to help AWS improve the quality and features of the solution. For more information, including how to disable this capability, please see the [implementation guide](https://docs.aws.amazon.com/solutions/latest/migration-assistant-for-amazon-opensearch-service/).
76+
77+
## License
78+
Licensed under the Apache License Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at
79+
[https://www.apache.org/licenses/](https://www.apache.org/licenses/)
80+
or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions and limitations under the License.

deployment/build-s3-dist.sh

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
#!/usr/bin/env bash
2+
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
3+
# SPDX-License-Identifier: Apache-2.0
4+
5+
# Check to see if input has been provided:
6+
if [ -z "$1" ] || [ -z "$2" ] || [ -z "$3" ]; then
7+
echo "Please provide the base source bucket name, trademark approved solution name and version where the lambda code will eventually reside."
8+
echo "For example: ./build-s3-dist.sh solutions trademarked-solution-name v1.0.0"
9+
exit 1
10+
fi
11+
12+
set -e
13+
14+
# Get reference for all important folders
15+
template_dir="$PWD"
16+
template_dist_dir="${template_dir}/global-s3-assets"
17+
build_dist_dir="${template_dir}/regional-s3-assets"
18+
source_dir="${template_dir}/../source"
19+
20+
echo "------------------------------------------------------------------------------"
21+
echo "Rebuild distribution"
22+
echo "------------------------------------------------------------------------------"
23+
rm -rf "${template_dist_dir}"
24+
mkdir -p "${template_dist_dir}"
25+
rm -rf "${build_dist_dir}"
26+
mkdir -p "${build_dist_dir}"
27+
28+
[ -e "${template_dist_dir}" ] && rm -r "${template_dist_dir}"
29+
[ -e "${build_dist_dir}" ] && rm -r "${build_dist_dir}"
30+
mkdir -p "${template_dist_dir}" "${build_dist_dir}"
31+
touch "$build_dist_dir"/test.txt
32+
33+
echo "--------------------------------------------------------------------------------------"
34+
echo "CloudFormation Template generation"
35+
echo "--------------------------------------------------------------------------------------"
36+
export CODE_BUCKET=$1
37+
export SOLUTION_NAME=$2
38+
export CODE_VERSION=$3
39+
40+
cd "$source_dir"/solutions-infrastructure
41+
npm install
42+
node_modules/aws-cdk/bin/cdk synth --asset-metadata false --path-metadata false > "$template_dist_dir"/"$SOLUTION_NAME".template
43+
44+
if [ $? -eq 0 ]
45+
then
46+
echo "Build for $SOLUTION_NAME.template succeeded"
47+
else
48+
echo "******************************************************************************"
49+
echo "Build FAILED for $SOLUTION_NAME.template"
50+
echo "******************************************************************************"
51+
exit 1
52+
fi

0 commit comments

Comments
 (0)