[automatic-failover] Draft implementation for automatic-failover

[atakavci]

Draft: Automatic Failover Feature for Lettuce

Overview

Initial implementation of automatic failover capabilities for Lettuce Redis client with a built in circuit breaker and multi-database support.

---

Key Components

Circuit Breaker Implementation

• CircuitBreaker - Core circuit breaker with configurable failure thresholds and tracked exceptions
• CircuitBreakerMetrics - Metrics tracking for success/failure rates
• State management (CLOSED/OPEN) with listener support for state transitions
• Default tracked exceptions: connection failures, I/O errors, timeouts

Multi-Database Client

• MultiDbClient - Failover-aware client managing multiple Redis endpoints
• StatefulRedisMultiDbConnection - Connection wrapper with automatic endpoint switching
• StatefulRedisMultiDbPubSubConnection - PubSub support with failover capabilities
• Database configuration with weight-based load balancing

Endpoint Tracking

• DatabaseEndpoint - Custom endpoints, DatabaseEndpointImpl/DatabasePubSubEndpointImpl extending DefaultEndpoint and PubSubEndpoint
• Command-level success/failure tracking via completion callbacks
• Automatic circuit breaker evaluation on each command completion

Failover Logic

• Automatic failover triggered when circuit breaker opens
• Weight-based selection of healthy databases
• Command queue handover during database switches
• Listener migration between endpoints

---

Configuration

DatabaseConfig - Per-database configuration:

• Redis URI
• Weight for load balancing
• Client options
• Circuit breaker config

Default Circuit Breaker Settings:

• Failure rate threshold: 10%
• Minimum failure count: 1000

---

Notes

• Standalone Redis only (not for Sentinel/Cluster)
• Draft/POC implementation
• Sync/async/reactive API support maintained across failovers (Notable limitation: Sync commands API needs major improvement on handling timeout exceptions and recording in circuit breaker.)