Cherry-pick "add Forward mode #3697" to 7.9.x

[kcorman0]

What

Cherry-picking these PRs to support forward mode on 7.9.x
#3697
#3705
#3809

Checklist

Please answer the questions with Y, N or N/A if not applicable.

• [Y] Contains customer facing changes? Including API/behavior changes
• [N] Is this change gated behind config(s)?
  • List the config(s) needed to be set to enable this change: Kinda, gated behind setting mode to "FORWARD"
• [Y] Did you add sufficient unit test and/or integration test coverage for this PR?
  • If not, please explain why it is not required
• [N] Does this change require modifying existing system tests or adding new system tests?
  • If so, include tracking information for the system test changes
• [N] Must this be released together with other change(s), either in this repo or another one?
  • If so, please include the link(s) to the changes that must be released together

References

JIRA:

Test & Review

Open questions / Follow-ups

[confluent-cla-assistant]

🎉 All Contributor License Agreements have been signed. Ready to merge.
_{Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.}

[sonarqube-confluent]

Quality Gate passed

Issues
2 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
89.1% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube