documentation: updating core docs

CLAUDE.md

···
 # QuickDID - Development Guide for Claude
 
 ## Overview
-QuickDID is a high-performance AT Protocol identity resolution service written in Rust. It provides handle-to-DID resolution with Redis-backed caching and queue processing.
+QuickDID is a high-performance AT Protocol identity resolution service written in Rust. It provides handle-to-DID resolution with multi-layer caching (Redis, SQLite, in-memory), queue processing, metrics support, and proactive cache refreshing.
 
 ## Configuration
 
···
    - `CachingHandleResolver`: In-memory caching layer
    - `RedisHandleResolver`: Redis-backed persistent caching
    - `SqliteHandleResolver`: SQLite-backed persistent caching
+   - `ProactiveRefreshResolver`: Automatically refreshes cache entries before expiration
    - Uses binary serialization via `HandleResolutionResult` for space efficiency
-   - Resolution stack: Cache → RateLimited (optional) → Base → DNS/HTTP
+   - Resolution stack: Cache → ProactiveRefresh (optional) → RateLimited (optional) → Base → DNS/HTTP
    - Includes resolution timing measurements for metrics
 
 2. **Binary Serialization** (`src/handle_resolution_result.rs`)
···
 - `METRICS_STATSD_BIND`: Bind address for StatsD UDP socket (default: [::]:0 for IPv6, can use 0.0.0.0:0 for IPv4)
 - `METRICS_PREFIX`: Prefix for all metrics (default: quickdid)
 - `METRICS_TAGS`: Comma-separated tags (e.g., env:prod,service:quickdid)
+
+### Optional - Proactive Refresh
+- `PROACTIVE_REFRESH_ENABLED`: Enable proactive cache refreshing (default: false)
+- `PROACTIVE_REFRESH_THRESHOLD`: Refresh when TTL remaining is below this threshold (0.0-1.0, default: 0.8)
 
 ## Error Handling
 

README.md

···
 # QuickDID
 
-QuickDID is a high-performance AT Protocol identity resolution service written in Rust. It provides blazing-fast handle-to-DID resolution with intelligent caching strategies, supporting in-memory, Redis-backed, and SQLite-backed persistent caching with binary serialization for optimal storage efficiency.
+QuickDID is a high-performance AT Protocol identity resolution service written in Rust. It provides blazing-fast handle-to-DID resolution with intelligent caching strategies, supporting in-memory, Redis-backed, and SQLite-backed persistent caching with binary serialization for optimal storage efficiency. The service includes proactive cache refreshing to maintain optimal performance and comprehensive metrics support for production monitoring.
 
 Built following the 12-factor app methodology with minimal dependencies and optimized for production use, QuickDID delivers exceptional performance while maintaining a lean footprint. Configuration is handled exclusively through environment variables, with only `--version` and `--help` command-line arguments supported.
 
···
   - Resolution timing measurements
   - Configurable tags for environment/service identification
   - Integration guides for Telegraf and TimescaleDB
+  - Configurable bind address for StatsD UDP socket (IPv4/IPv6)
+- **Proactive Cache Refresh**: 
+  - Automatically refreshes cache entries before expiration
+  - Configurable refresh threshold
+  - Prevents cache misses for frequently accessed handles
+  - Metrics tracking for refresh operations
+- **Queue Deduplication**: 
+  - Redis-based deduplication for queue items
+  - Prevents duplicate handle resolution work
+  - Configurable TTL for deduplication keys
 - **AT Protocol Compatible**: Implements XRPC endpoints for seamless integration with AT Protocol infrastructure
 - **Comprehensive Error Handling**: Structured errors with unique identifiers (e.g., `error-quickdid-config-1`), health checks, and graceful shutdown
 - **12-Factor App**: Environment-based configuration following cloud-native best practices
···
 - `METRICS_PREFIX`: Prefix for all metrics (default: quickdid)
 - `METRICS_TAGS`: Comma-separated tags (e.g., env:prod,service:quickdid)
 
+#### Proactive Refresh
+- `PROACTIVE_REFRESH_ENABLED`: Enable proactive cache refreshing (default: false)
+- `PROACTIVE_REFRESH_THRESHOLD`: Refresh when TTL remaining is below this threshold (0.0-1.0, default: 0.8)
+
 #### Logging
 - `RUST_LOG`: Logging level (e.g., debug, info, warn, error)
 
···
 QuickDID uses a layered architecture for optimal performance:
 
 ```
-Request → Cache Layer → Rate Limiter → Base Resolver → DNS/HTTP
-           ↓               ↓                ↓
-        Memory/Redis/   Semaphore      AT Protocol
-          SQLite        (optional)     Infrastructure
+Request → Cache Layer → Proactive Refresh → Rate Limiter → Base Resolver → DNS/HTTP
+           ↓                  ↓                 ↓               ↓
+        Memory/Redis/    Background         Semaphore      AT Protocol
+          SQLite         Refresher         (optional)     Infrastructure
 ```
 
 ### Cache Priority

docs/configuration-reference.md

···
 QUEUE_REDIS_TIMEOUT=30   # Minimal polling, slow shutdown
 ```
 
+### `QUEUE_REDIS_DEDUP_ENABLED`
+
+**Required**: No  
+**Type**: Boolean  
+**Default**: `false`  
+
+Enable deduplication for Redis queue to prevent duplicate handles from being queued multiple times within the TTL window. When enabled, uses Redis SET with TTL to track handles currently being processed.
+
+**Examples**:
+```bash
+# Enable deduplication (recommended for production)
+QUEUE_REDIS_DEDUP_ENABLED=true
+
+# Disable deduplication (default)
+QUEUE_REDIS_DEDUP_ENABLED=false
+```
+
+**Use cases**:
+- **Production**: Enable to prevent duplicate work and reduce load
+- **High-traffic**: Essential to avoid processing the same handle multiple times
+- **Development**: Can be disabled for simpler debugging
+
+### `QUEUE_REDIS_DEDUP_TTL`
+
+**Required**: No  
+**Type**: Integer (seconds)  
+**Default**: `60`  
+**Range**: 10-300 (recommended)  
+**Constraints**: Must be > 0 when deduplication is enabled  
+
+TTL for Redis queue deduplication keys in seconds. Determines how long to prevent duplicate handle resolution requests.
+
+**Examples**:
+```bash
+# Quick deduplication window (10 seconds)
+QUEUE_REDIS_DEDUP_TTL=10
+
+# Default (1 minute)
+QUEUE_REDIS_DEDUP_TTL=60
+
+# Extended deduplication (5 minutes)
+QUEUE_REDIS_DEDUP_TTL=300
+```
+
+**Recommendations**:
+- **Fast processing**: 10-30 seconds
+- **Normal processing**: 60 seconds (default)
+- **Slow processing or high load**: 120-300 seconds
+
 ### `QUEUE_WORKER_ID`
 
 **Required**: No  
···
 Rate limit permit acquisition timed out after {timeout}ms
 ```
 
+## Metrics Configuration
+
+### `METRICS_ADAPTER`
+
+**Required**: No  
+**Type**: String  
+**Default**: `noop`  
+**Values**: `noop`, `statsd`  
+
+Metrics adapter type for collecting and publishing metrics.
+
+**Options**:
+- `noop`: No metrics collection (default)
+- `statsd`: Send metrics to StatsD server
+
+**Examples**:
+```bash
+# No metrics (default)
+METRICS_ADAPTER=noop
+
+# Enable StatsD metrics
+METRICS_ADAPTER=statsd
+```
+
+### `METRICS_STATSD_HOST`
+
+**Required**: Yes (when METRICS_ADAPTER=statsd)  
+**Type**: String  
+**Format**: hostname:port  
+
+StatsD server host and port for metrics collection.
+
+**Examples**:
+```bash
+# Local StatsD
+METRICS_STATSD_HOST=localhost:8125
+
+# Remote StatsD
+METRICS_STATSD_HOST=statsd.example.com:8125
+
+# Docker network
+METRICS_STATSD_HOST=statsd:8125
+```
+
+### `METRICS_STATSD_BIND`
+
+**Required**: No  
+**Type**: String  
+**Default**: `[::]:0`  
+
+Bind address for StatsD UDP socket. Controls which local address to bind for sending UDP packets.
+
+**Examples**:
+```bash
+# IPv6 any address, random port (default)
+METRICS_STATSD_BIND=[::]:0
+
+# IPv4 any address, random port
+METRICS_STATSD_BIND=0.0.0.0:0
+
+# Specific interface
+METRICS_STATSD_BIND=192.168.1.100:0
+
+# Specific port
+METRICS_STATSD_BIND=[::]:8126
+```
+
+### `METRICS_PREFIX`
+
+**Required**: No  
+**Type**: String  
+**Default**: `quickdid`  
+
+Prefix for all metrics. Used to namespace metrics in your monitoring system.
+
+**Examples**:
+```bash
+# Default
+METRICS_PREFIX=quickdid
+
+# Environment-specific
+METRICS_PREFIX=prod.quickdid
+METRICS_PREFIX=staging.quickdid
+
+# Region-specific
+METRICS_PREFIX=us-east-1.quickdid
+METRICS_PREFIX=eu-west-1.quickdid
+
+# Service-specific
+METRICS_PREFIX=api.quickdid
+```
+
+### `METRICS_TAGS`
+
+**Required**: No  
+**Type**: String (comma-separated key:value pairs)  
+**Default**: None  
+
+Default tags for all metrics. Added to all metrics for filtering and grouping.
+
+**Examples**:
+```bash
+# Basic tags
+METRICS_TAGS=env:production,service:quickdid
+
+# Detailed tags
+METRICS_TAGS=env:production,service:quickdid,region:us-east-1,version:1.0.0
+
+# Deployment-specific
+METRICS_TAGS=env:staging,cluster:k8s-staging,namespace:quickdid
+```
+
+**Common tag patterns**:
+- `env`: Environment (production, staging, development)
+- `service`: Service name
+- `region`: Geographic region
+- `version`: Application version
+- `cluster`: Kubernetes cluster name
+- `instance`: Instance identifier
+
+## Proactive Refresh Configuration
+
+### `PROACTIVE_REFRESH_ENABLED`
+
+**Required**: No  
+**Type**: Boolean  
+**Default**: `false`  
+
+Enable proactive cache refresh for frequently accessed handles. When enabled, cache entries that have reached the refresh threshold will be queued for background refresh to keep the cache warm.
+
+**Examples**:
+```bash
+# Enable proactive refresh (recommended for production)
+PROACTIVE_REFRESH_ENABLED=true
+
+# Disable proactive refresh (default)
+PROACTIVE_REFRESH_ENABLED=false
+```
+
+**Benefits**:
+- Prevents cache misses for popular handles
+- Maintains consistent response times
+- Reduces latency spikes during cache expiration
+
+**Considerations**:
+- Increases background processing load
+- More DNS/HTTP requests to upstream services
+- Best for high-traffic services with predictable access patterns
+
+### `PROACTIVE_REFRESH_THRESHOLD`
+
+**Required**: No  
+**Type**: Float  
+**Default**: `0.8`  
+**Range**: 0.0-1.0  
+**Constraints**: Must be between 0.0 and 1.0  
+
+Threshold as a percentage (0.0-1.0) of cache TTL when to trigger proactive refresh. For example, 0.8 means refresh when an entry has lived for 80% of its TTL.
+
+**Examples**:
+```bash
+# Very aggressive (refresh at 50% of TTL)
+PROACTIVE_REFRESH_THRESHOLD=0.5
+
+# Moderate (refresh at 70% of TTL)
+PROACTIVE_REFRESH_THRESHOLD=0.7
+
+# Default (refresh at 80% of TTL)
+PROACTIVE_REFRESH_THRESHOLD=0.8
+
+# Conservative (refresh at 90% of TTL)
+PROACTIVE_REFRESH_THRESHOLD=0.9
+
+# Very conservative (refresh at 95% of TTL)
+PROACTIVE_REFRESH_THRESHOLD=0.95
+```
+
+**Recommendations**:
+- **High-traffic services**: 0.5-0.7 (aggressive refresh)
+- **Normal traffic**: 0.8 (default, balanced)
+- **Low traffic**: 0.9-0.95 (conservative)
+- **Development**: 0.5 (test refresh behavior)
+
+**Impact on different cache TTLs**:
+- TTL=600s (10 min), threshold=0.8: Refresh after 8 minutes
+- TTL=3600s (1 hour), threshold=0.8: Refresh after 48 minutes
+- TTL=86400s (1 day), threshold=0.8: Refresh after 19.2 hours
+
 ## HTTP Caching Configuration
 
 ### `CACHE_MAX_AGE`
···
 QUEUE_ADAPTER=redis
 QUEUE_REDIS_TIMEOUT=5
 QUEUE_BUFFER_SIZE=5000
+QUEUE_REDIS_DEDUP_ENABLED=true  # Prevent duplicate work
+QUEUE_REDIS_DEDUP_TTL=60
 
 # Rate Limiting (optional, recommended for production)
 RESOLVER_MAX_CONCURRENT=100
 RESOLVER_MAX_CONCURRENT_TIMEOUT_MS=5000  # 5 second timeout
+
+# Metrics (optional, recommended for production)
+METRICS_ADAPTER=statsd
+METRICS_STATSD_HOST=localhost:8125
+METRICS_PREFIX=quickdid
+METRICS_TAGS=env:prod,service:quickdid
+
+# Proactive Refresh (optional, recommended for high-traffic)
+PROACTIVE_REFRESH_ENABLED=true
+PROACTIVE_REFRESH_THRESHOLD=0.8
 
 # HTTP Caching (Cache-Control headers)
 CACHE_MAX_AGE=86400  # 24 hours
···
 QUEUE_REDIS_PREFIX=prod:queue:
 QUEUE_WORKER_ID=${HOSTNAME:-worker1}
 QUEUE_REDIS_TIMEOUT=10
+QUEUE_REDIS_DEDUP_ENABLED=true  # Essential for multi-instance
+QUEUE_REDIS_DEDUP_TTL=120  # Longer TTL for HA
 
 # Performance
 QUEUE_BUFFER_SIZE=10000
···
 # Rate Limiting (important for HA deployments)
 RESOLVER_MAX_CONCURRENT=500
 RESOLVER_MAX_CONCURRENT_TIMEOUT_MS=10000  # 10 second timeout for HA
+
+# Metrics (recommended for HA monitoring)
+METRICS_ADAPTER=statsd
+METRICS_STATSD_HOST=statsd:8125
+METRICS_PREFIX=quickdid.prod
+METRICS_TAGS=env:prod,service:quickdid,cluster:ha
+
+# Proactive Refresh (recommended for HA)
+PROACTIVE_REFRESH_ENABLED=true
+PROACTIVE_REFRESH_THRESHOLD=0.7  # More aggressive for HA
 
 # Logging
 RUST_LOG=warn

docs/production-deployment.md

···
 # Higher = less polling overhead, slower shutdown
 QUEUE_REDIS_TIMEOUT=5
 
+# Enable deduplication for Redis queue to prevent duplicate handles (default: false)
+# When enabled, uses Redis SET with TTL to track handles being processed
+# Prevents the same handle from being queued multiple times within the TTL window
+QUEUE_REDIS_DEDUP_ENABLED=false
+
+# TTL for Redis queue deduplication keys in seconds (default: 60)
+# Range: 10-300 recommended
+# Determines how long to prevent duplicate handle resolution requests
+QUEUE_REDIS_DEDUP_TTL=60
+
 # Worker ID for Redis queue (defaults to "worker1")
 # Set this for predictable worker identification in multi-instance deployments
 # Examples: worker-001, prod-us-east-1, $(hostname)
···
 # Minimum time response must remain fresh
 # Clients won't accept responses expiring within this time
 CACHE_MIN_FRESH=3600
+
+# ----------------------------------------------------------------------------
+# METRICS CONFIGURATION
+# ----------------------------------------------------------------------------
+
+# Metrics adapter type: 'noop' or 'statsd' (default: noop)
+# - 'noop': No metrics collection (default)
+# - 'statsd': Send metrics to StatsD server
+METRICS_ADAPTER=statsd
+
+# StatsD host and port (required when METRICS_ADAPTER=statsd)
+# Format: hostname:port
+# Examples:
+#   - localhost:8125 (local StatsD)
+#   - statsd.example.com:8125 (remote StatsD)
+METRICS_STATSD_HOST=localhost:8125
+
+# Bind address for StatsD UDP socket (default: [::]:0)
+# Controls which local address to bind for sending UDP packets
+# Examples:
+#   - [::]:0 (IPv6 any address, random port - default)
+#   - 0.0.0.0:0 (IPv4 any address, random port)
+#   - 192.168.1.100:0 (specific interface)
+METRICS_STATSD_BIND=[::]:0
+
+# Prefix for all metrics (default: quickdid)
+# Used to namespace metrics in your monitoring system
+# Examples:
+#   - quickdid (default)
+#   - prod.quickdid
+#   - us-east-1.quickdid
+METRICS_PREFIX=quickdid
+
+# Tags for all metrics (comma-separated key:value pairs)
+# Added to all metrics for filtering and grouping
+# Examples:
+#   - env:production,service:quickdid
+#   - env:staging,region:us-east-1,version:1.0.0
+METRICS_TAGS=env:production,service:quickdid
+
+# ----------------------------------------------------------------------------
+# PROACTIVE REFRESH CONFIGURATION
+# ----------------------------------------------------------------------------
+
+# Enable proactive cache refresh (default: false)
+# When enabled, cache entries nearing expiration are automatically refreshed
+# in the background to prevent cache misses for frequently accessed handles
+PROACTIVE_REFRESH_ENABLED=false
+
+# Threshold for proactive refresh as percentage of TTL (default: 0.8)
+# Range: 0.0-1.0 (0% to 100% of TTL)
+# Example: 0.8 means refresh when 80% of TTL has elapsed
+# Lower values = more aggressive refreshing, higher load
+# Higher values = less aggressive refreshing, more cache misses
+PROACTIVE_REFRESH_THRESHOLD=0.8
 
 # ----------------------------------------------------------------------------
 # PERFORMANCE TUNING