[Deliverable] Create Segmentation Library

[fryorcraken]

Owner: App/Chat Dev

Feature: Segmentation

FURPS:

• F1. Outbound messages larger than the maximum Waku message size are partitioned in several messages to fit in Waku messages.

• F2. Inbound partitioned messages are reconstructed in a whole message.

• F3. A capping limit is applied to pre-segmented messages (e.g. 100MB).

• F4. Messages under the maximum message size are not modified.

• U1. Only takes a maximum message size as a parameter.

• R1. Reconstruction can be performed even when parts are received out or order.

• R2. Reconstruction can be performed as long as 87.5% of the segments is received.

• R3. If too many parts missing to reconstruct an informative error should be logged.

• P1. The payload overhead does not exceed 12.5% overall, and 100 bytes per segment.

• S1. Nim library.

• +1. Segmentation metadata should not reveal information about the original message content

• +2. Relevant for all Waku nodes

• +3. Nimble package manager is used to build.

Checklist:

• Specs: link to specs
  • feat: add segmentation spec specs#91
• Code: link to GitHub issues/PRs/Epic
• Dogfood: link to dogfooding session/artefact
  • N/A for now up until chat-sdk poc or mvp ready
• Docs: links to README.md or docs.waku.org
  • https://github.com/waku-org/nim-chat-sdk/blob/main/README.segmentation.md

Tasks

• feat: sqlite migration nim-chat-sdk#4
• feat: message segmentation nim-chat-sdk#1

[fryorcraken]

@plopezlpz as dicussed:

• can we have specS? don't we already have some form of specs from Status?
• docs add a README file to explain how to use segmentation