clean up

.github/SQLX_AND_CROSS_COMPILATION.md

···
-# SQLx Offline and Cross-Compilation Setup
-
-This document explains the configuration changes made to support SQLx offline builds and fix cross-compilation issues.
-
-## Problems Solved
-
-### 1. SQLx Offline Compilation
-When `SQLX_OFFLINE=true`, each Rust project needs access to `.sqlx` query metadata files in their local directory. Previously, these files only existed at the workspace root, causing builds to fail outside Docker.
-
-### 2. Cross-Compilation OpenSSL Issues  
-Cross-compilation was failing due to OpenSSL dependencies being pulled in by various crates, which is notoriously difficult to cross-compile.
-
-## Solutions Implemented
-
-### SQLx Offline Setup
-
-#### Script: `scripts/setup-sqlx-offline.sh`
-- Automatically copies `.sqlx` files from workspace root to each SQLx-dependent project
-- Identifies projects that use SQLx by checking `Cargo.toml` files
-- Projects that receive `.sqlx` files:
-  - `apps/aqua`
-  - `services/cadet` 
-  - `services/satellite`
-
-#### CI Integration
-The script is now called in all CI jobs that build Rust code:
-- `setup-and-build` job
-- `rust-cross-compile` job  
-- `rust-quality` job
-- `security-audit` job
-
-### Cross-Compilation Fixes
-
-#### 1. Replaced OpenSSL with rustls
-Updated workspace dependencies to use rustls instead of OpenSSL:
-
-```toml
-# Root Cargo.toml
-tokio = { version = "1.0", features = [
-    "rt-multi-thread",
-    "macros",
-    "time",        # Required for tokio::time module
-    "net",         # Required for networking
-    "sync",        # Required for synchronization primitives
-] }
-
-sqlx = { version = "0.8", features = [
-    "runtime-tokio",
-    "postgres", 
-    "uuid",
-    "tls-rustls",  # Instead of default OpenSSL
-] }
-
-reqwest = { version = "0.12", default-features = false, features = [
-    "json",
-    "rustls-tls",  # Instead of default native-tls
-    "stream",
-    "gzip",
-] }
-
-tokio-tungstenite = { version = "*", default-features = false, features = [
-    "rustls-tls-webpki-roots",  # Instead of default native-tls
-    "connect",                   # For connect_async function
-    "handshake",                # For accept_async function (tests)
-] }
-```
-
-#### 2. Fixed Workspace Dependency Conflicts
-The `services/Cargo.toml` was overriding workspace dependencies with different configurations. Fixed by:
-- Changing `reqwest = { version = "0.12", features = ["json"] }` to `reqwest.workspace = true`
-- Changing `tokio-tungstenite = "0.24"` to `tokio-tungstenite.workspace = true`
-
-#### 3. Enhanced Cross.toml Configuration
-Created comprehensive `Cross.toml` files for cross-compilation:
-
-```toml
-[build.env]
-passthrough = [
-    "CARGO_HOME",
-    "CARGO_TARGET_DIR", 
-    "SQLX_OFFLINE",
-    "PKG_CONFIG_ALLOW_CROSS",
-]
-
-[target.aarch64-unknown-linux-gnu]
-image = "ghcr.io/cross-rs/aarch64-unknown-linux-gnu:main"
-
-[target.aarch64-unknown-linux-gnu.env]
-passthrough = ["CARGO_HOME", "CARGO_TARGET_DIR", "SQLX_OFFLINE"]
-PKG_CONFIG_ALLOW_CROSS = "1"
-RUSTFLAGS = "-C target-feature=+crt-static -C link-arg=-s"
-CC_aarch64_unknown_linux_gnu = "aarch64-linux-gnu-gcc"
-CXX_aarch64_unknown_linux_gnu = "aarch64-linux-gnu-g++"
-```
-
-#### 4. Improved CI Cross-Compilation
-- Uses latest `cross` from Git: `cargo install cross --git https://github.com/cross-rs/cross`
-- Sets `PKG_CONFIG_ALLOW_CROSS=1` environment variable
-- Copies `.sqlx` files to individual service directories during cross-compilation
-- Improved executable collection with better filtering
-
-### Executable Collection
-
-Enhanced executable collection in CI:
-- Filters out build artifacts (`.d` files and temporary files with `-` in names)
-- Handles missing target directories gracefully
-- Collects executables for both x86_64 and aarch64 targets
-- Provides clear logging of collected executables
-
-## File Changes
-
-### Created
-- `scripts/setup-sqlx-offline.sh` - SQLx offline setup script
-- `Cross.toml` - Root cross-compilation config
-- `services/Cross.toml` - Services cross-compilation config  
-- `apps/aqua/Cross.toml` - Aqua cross-compilation config
-
-### Modified
-- `Cargo.toml` - Updated workspace dependencies to use rustls
-- `services/Cargo.toml` - Fixed workspace dependency usage
-- `.github/workflows/ci.yml` - Added SQLx setup and improved cross-compilation
-
-## Usage
-
-### Running SQLx Setup Locally
-```bash
-./scripts/setup-sqlx-offline.sh
-```
-
-### Cross-Compilation Locally
-```bash
-# Install cross if not already installed
-cargo install cross --git https://github.com/cross-rs/cross
-
-# Add target
-rustup target add aarch64-unknown-linux-gnu
-
-# Set environment
-export PKG_CONFIG_ALLOW_CROSS=1
-export SQLX_OFFLINE=true
-
-# Run SQLx setup
-./scripts/setup-sqlx-offline.sh
-
-# Cross-compile services
-cd services
-cross build --release --target aarch64-unknown-linux-gnu
-
-# Cross-compile apps
-cd ../apps/aqua  
-cross build --release --target aarch64-unknown-linux-gnu
-```
-
-### Updating SQLx Queries
-When you add or modify SQL queries:
-
-1. Generate new query metadata from the services directory:
-   ```bash
-   cd services
-   cargo sqlx prepare
-   ```
-
-2. Update all project copies:
-   ```bash
-   ./scripts/setup-sqlx-offline.sh
-   ```
-
-## Troubleshooting
-
-### Cross-Compilation Still Fails
-- Ensure you're using the latest `cross` from Git
-- Check that `PKG_CONFIG_ALLOW_CROSS=1` is set
-- Verify no dependencies are pulling in OpenSSL (use `cargo tree | grep openssl`)
-
-### tokio-tungstenite Import Errors
-If you see "unresolved import `tokio_tungstenite::connect_async`" errors:
-- Ensure the `connect` feature is enabled for client functionality
-- Add the `handshake` feature if using `accept_async` for server functionality
-- Check that `default-features = false` is set to avoid OpenSSL dependencies
-
-### tokio Module Errors
-If you see "could not find `time` in `tokio`" or similar errors:
-- Ensure tokio workspace dependency includes required features: `time`, `net`, `sync`
-- These features are often included in default features but must be explicit when using `default-features = false`
-
-### SQLx Offline Errors
-- Run `./scripts/setup-sqlx-offline.sh` after any SQL query changes
-- Ensure `.sqlx` directory exists in workspace root
-- Check that `SQLX_OFFLINE=true` is set in CI environment
-
-### Missing Executables
-- Check that build succeeded without errors
-- Verify executable names match expected service/app names
-- Look for executables in `artifacts/` directory structure
-
-## Dependencies Avoided
-To maintain cross-compilation compatibility, avoid dependencies that:
-- Default to OpenSSL (use rustls variants)
-- Require system libraries not available in cross containers
-- Have complex native build requirements
-
-### Common Feature Configuration Issues
-When disabling default features to avoid OpenSSL:
-- `tokio`: Must explicitly enable `time`, `net`, `sync` features for common functionality
-- `tokio-tungstenite`: Must explicitly enable `connect` and `handshake` features
-- `reqwest`: Must explicitly enable required features like `json`, `stream`, `gzip`
-- `sqlx`: Use `tls-rustls` instead of default OpenSSL TLS
-
-Always test cross-compilation locally before pushing changes.

docs/migration-troubleshooting.md

···
-# Migration Troubleshooting Guide
-
-## Common Migration Issues and Solutions
-
-### Issue: "cannot drop function because other objects depend on it"
-
-**Error Message:**
-```
-error: while executing migration 20241220000008: error returned from database: cannot drop function extract_discriminant(text) because other objects depend on it
-```
-
-**Cause:**
-This error occurs when trying to drop database functions that have dependent objects (views, other functions, triggers, etc.) without properly handling the dependencies.
-
-**Solution:**
-
-#### Option 1: Fix the Migration (Recommended)
-Update the problematic migration to handle dependencies properly:
-
-1. **Edit the migration file** (e.g., `20241220000008_fix_discriminant_case_sensitivity.sql`):
-
-```sql
--- Drop dependent views first, then functions, then recreate everything
-DROP VIEW IF EXISTS discriminant_analysis CASCADE;
-DROP VIEW IF EXISTS discriminant_stats CASCADE;
-
--- Drop existing functions with CASCADE to handle dependencies
-DROP FUNCTION IF EXISTS extract_discriminant(TEXT) CASCADE;
-DROP FUNCTION IF EXISTS get_base_name(TEXT) CASCADE;
-DROP FUNCTION IF EXISTS extract_edition_discriminant(TEXT) CASCADE;
-
--- Then recreate functions and views...
-```
-
-2. **Reset the migration state** if the migration was partially applied:
-
-```bash
-# Connect to your database and reset the specific migration
-psql $DATABASE_URL -c "DELETE FROM _sqlx_migrations WHERE version = '20241220000008';"
-
-# Or reset all migrations and start fresh (WARNING: This drops all data)
-psql $DATABASE_URL -c "DROP SCHEMA public CASCADE; CREATE SCHEMA public;"
-```
-
-3. **Run migrations again**:
-```bash
-cd services
-DATABASE_URL="your_database_url" sqlx migrate run
-```
-
-#### Option 2: Manual Dependency Cleanup
-If you can't modify the migration file:
-
-1. **Identify dependencies**:
-```sql
--- Find objects that depend on the function
-SELECT 
-    p.proname as function_name,
-    d.objid,
-    d.classid::regclass as object_type,
-    d.refobjid
-FROM pg_depend d
-JOIN pg_proc p ON d.refobjid = p.oid
-WHERE p.proname = 'extract_discriminant';
-```
-
-2. **Drop dependencies manually**:
-```sql
--- Drop dependent views
-DROP VIEW IF EXISTS discriminant_analysis CASCADE;
-DROP VIEW IF EXISTS discriminant_stats CASCADE;
-DROP VIEW IF EXISTS track_variants CASCADE;
-DROP VIEW IF EXISTS release_variants CASCADE;
-
--- Drop the functions
-DROP FUNCTION IF EXISTS extract_discriminant(TEXT) CASCADE;
-DROP FUNCTION IF EXISTS get_base_name(TEXT) CASCADE;
-DROP FUNCTION IF EXISTS extract_edition_discriminant(TEXT) CASCADE;
-```
-
-3. **Continue with migration**:
-```bash
-DATABASE_URL="your_database_url" sqlx migrate run
-```
-
-### Issue: "migration was previously applied but has been modified"
-
-**Error Message:**
-```
-error: migration 20241220000008 was previously applied but has been modified
-```
-
-**Cause:**
-The migration file has been changed after it was already applied to the database.
-
-**Solutions:**
-
-#### Option 1: Reset Migration State
-```bash
-# Remove the specific migration from tracking
-psql $DATABASE_URL -c "DELETE FROM _sqlx_migrations WHERE version = '20241220000008';"
-
-# Run migrations again
-DATABASE_URL="your_database_url" sqlx migrate run
-```
-
-#### Option 2: Create a New Migration
-```bash
-# Create a new migration with your changes
-sqlx migrate add fix_discriminant_case_sensitivity_v2
-
-# Copy your changes to the new migration file
-# Run the new migration
-DATABASE_URL="your_database_url" sqlx migrate run
-```
-
-#### Option 3: Full Reset (WARNING: Destroys all data)
-```bash
-# Connect to database and reset everything
-psql $DATABASE_URL -c "DROP SCHEMA public CASCADE; CREATE SCHEMA public;"
-
-# Run all migrations from scratch
-DATABASE_URL="your_database_url" sqlx migrate run
-```
-
-### Issue: "No such file or directory" when running migrations
-
-**Error Message:**
-```
-error: while resolving migrations: No such file or directory (os error 2)
-```
-
-**Cause:**
-The migration directory is not found in the expected location.
-
-**Solutions:**
-
-#### Option 1: Check Migration Directory Location
-```bash
-# Check where sqlx expects migrations
-cat services/.sqlx/.sqlxrc
-
-# Ensure migrations exist in the correct location
-ls -la services/migrations/
-```
-
-#### Option 2: Copy Migrations to Correct Location
-```bash
-# If migrations are in wrong location, copy them
-cp migrations/*.sql services/migrations/
-
-# Or create symlink
-ln -s ../migrations services/migrations
-```
-
-#### Option 3: Update sqlx Configuration
-Edit `services/.sqlx/.sqlxrc`:
-```toml
-[database]
-url = "postgres://localhost/teal"
-migrations = "../migrations"  # Update path as needed
-```
-
-### Issue: Database Connection Problems
-
-**Error Messages:**
-- `Connection refused (os error 61)`
-- `password authentication failed`
-- `database "teal_test" does not exist`
-
-**Solutions:**
-
-#### Connection Refused
-```bash
-# Check if database is running
-docker ps | grep postgres
-
-# Start database if needed
-docker-compose -f compose.db-test.yml up -d
-
-# Wait for database to start
-sleep 5
-```
-
-#### Authentication Issues
-```bash
-# Check connection string format
-DATABASE_URL="postgres://username:password@host:port/database"
-
-# Example for test database
-DATABASE_URL="postgres://postgres:testpass123@localhost:5433/teal_test"
-```
-
-#### Database Doesn't Exist
-```bash
-# Create database
-docker exec postgres_container psql -U postgres -c "CREATE DATABASE teal_test;"
-
-# Or recreate test environment
-docker-compose -f compose.db-test.yml down
-docker-compose -f compose.db-test.yml up -d
-```
-
-## Migration Best Practices
-
-### 1. Handle Dependencies Properly
-Always use `CASCADE` when dropping objects with dependencies:
-```sql
-DROP FUNCTION function_name(args) CASCADE;
-DROP VIEW view_name CASCADE;
-```
-
-### 2. Test Migrations Locally
-```bash
-# Use test database for migration testing
-DATABASE_URL="postgres://localhost:5433/teal_test" sqlx migrate run
-
-# Verify results
-psql "postgres://localhost:5433/teal_test" -c "SELECT extract_discriminant('Test (Example)');"
-```
-
-### 3. Backup Before Major Migrations
-```bash
-# Create backup
-pg_dump $DATABASE_URL > backup_before_migration.sql
-
-# Apply migrations
-sqlx migrate run
-
-# Restore if needed
-psql $DATABASE_URL < backup_before_migration.sql
-```
-
-### 4. Version Control Migration Files
-- Never modify applied migrations
-- Create new migrations for changes
-- Use descriptive migration names
-- Include rollback instructions in comments
-
-### 5. Migration File Structure
-```sql
--- Migration: descriptive_name
--- Purpose: Brief description of what this migration does
--- Dependencies: List any required prior migrations
--- Rollback: Instructions for manual rollback if needed
-
--- Drop dependencies first
-DROP VIEW IF EXISTS dependent_view CASCADE;
-
--- Make changes
-CREATE OR REPLACE FUNCTION new_function() ...;
-
--- Recreate dependencies
-CREATE VIEW dependent_view AS ...;
-
--- Update existing data if needed
-UPDATE table_name SET column = new_value WHERE condition;
-
--- Add comments
-COMMENT ON FUNCTION new_function IS 'Description of function purpose';
-```
-
-## Emergency Recovery
-
-### Complete Database Reset
-If migrations are completely broken:
-
-```bash
-# 1. Stop all services
-docker-compose down
-
-# 2. Remove database volume (WARNING: Destroys all data)
-docker volume rm teal_postgres_data
-
-# 3. Start fresh
-docker-compose up -d postgres
-
-# 4. Wait for database to initialize
-sleep 10
-
-# 5. Run all migrations from scratch
-DATABASE_URL="your_database_url" sqlx migrate run
-```
-
-### Partial Recovery
-If only discriminant system is broken:
-
-```sql
--- Remove discriminant-related objects
-DROP VIEW IF EXISTS discriminant_analysis CASCADE;
-DROP VIEW IF EXISTS discriminant_stats CASCADE;
-DROP VIEW IF EXISTS track_variants CASCADE;
-DROP VIEW IF EXISTS release_variants CASCADE;
-DROP FUNCTION IF EXISTS extract_discriminant(TEXT) CASCADE;
-DROP FUNCTION IF EXISTS get_base_name(TEXT) CASCADE;
-DROP FUNCTION IF EXISTS extract_edition_discriminant(TEXT) CASCADE;
-
--- Remove discriminant columns
-ALTER TABLE plays DROP COLUMN IF EXISTS track_discriminant;
-ALTER TABLE plays DROP COLUMN IF EXISTS release_discriminant;
-ALTER TABLE recordings DROP COLUMN IF EXISTS discriminant;
-ALTER TABLE releases DROP COLUMN IF EXISTS discriminant;
-
--- Mark discriminant migrations as not applied
-DELETE FROM _sqlx_migrations WHERE version >= '20241220000006';
-
--- Re-run discriminant migrations
-```
-
-## Getting Help
-
-### Debug Information to Collect
-When reporting migration issues, include:
-
-1. **Error message** (full stack trace)
-2. **Migration file content** that's causing issues
-3. **Database state**:
-   ```sql
-   SELECT version FROM _sqlx_migrations ORDER BY version;
-   \df extract_discriminant
-   \dv discriminant_*
-   ```
-4. **Environment details**:
-   - Database version: `SELECT version();`
-   - Operating system
-   - sqlx version: `cargo sqlx --version`
-
-### Useful Debugging Commands
-```sql
--- Check applied migrations
-SELECT * FROM _sqlx_migrations ORDER BY version;
-
--- Check function definitions
-\df+ extract_discriminant
-
--- Check view definitions
-\d+ discriminant_analysis
-
--- Check table schemas
-\d+ plays
-\d+ recordings
-\d+ releases
-
--- Test function directly
-SELECT extract_discriminant('Test (Example)');
-```
-
-## Contact and Support
-
-For persistent migration issues:
-1. Check this troubleshooting guide first
-2. Review the specific migration file causing issues
-3. Try solutions in order of preference (fix migration → manual cleanup → reset)
-4. Create minimal reproduction case for complex issues
-5. Document exact steps that led to the error for support requests