tinsnip / docs / service_setup_guide.md

# Service Setup Script Guide

This guide covers creating `setup.sh` scripts for tinsnip services. Setup scripts prepare the service environment after machine infrastructure is created but before the service containers start.

## Overview

Service setup scripts run once during service deployment to:
- Create required directories and files
- Set up configuration templates
- Initialize service-specific data structures
- Create convenience utilities for the service user

## Script Structure

### Basic Template

```bash
#!/bin/bash
# [Service Name] Service Setup
# Brief description of what this service does

set -euo pipefail

# Source service environment variables
if [[ -f "/mnt/${USER}/.env" ]]; then
    source "/mnt/${USER}/.env"
elif [[ -f "../../../.env" ]]; then
    source "../../../.env"
else
    echo "Error: Could not find service .env file" >&2
    exit 1
fi

# Simple logging function (self-contained)
log_with_prefix() {
    local prefix="$1"
    local message="$2"
    echo "[$prefix] $message"
}

log_with_prefix "ServiceName" "Setting up service environment"

# Your setup logic here...

log_with_prefix "ServiceName" "Setup complete!"
```

## Available Environment Variables

The service `.env` file provides these standard variables:

```bash
TIN_SERVICE_NAME        # Service name (e.g., "nrfconnect")
TIN_SERVICE_ENVIRONMENT # Environment (e.g., "prod", "test")
TIN_SERVICE_UID         # Service user UID (e.g., 50300)
TIN_SHEET              # Sheet name (e.g., "topsheet")

# Ports (if service uses them)
TIN_PORT_0             # Base port (usually same as UID)
TIN_PORT_1             # Additional ports as needed
TIN_PORT_2             # ...

# XDG directories (NFS-backed)
XDG_DATA_HOME          # /mnt/service-env/data
XDG_CONFIG_HOME        # /mnt/service-env/config  
XDG_STATE_HOME         # /mnt/service-env/state

# Docker environment
DOCKER_HOST            # unix:///run/user/UID/docker.sock
XDG_RUNTIME_DIR        # /run/user/UID
```

## Directory Structure

Services follow the XDG Base Directory specification with NFS backing:

```
/mnt/service-environment/
├── data/          # Persistent application data (XDG_DATA_HOME)
├── config/        # Configuration files (XDG_CONFIG_HOME)
├── state/         # Logs, runtime state (XDG_STATE_HOME)
└── service/       # Service definitions (docker-compose.yml, etc.)
```

### Directory Usage Patterns

- **data/** - Database files, user content, application-specific data
- **config/** - Configuration templates, settings files  
- **state/** - Log files, session data

## Common Setup Tasks

### 1. Create Required Directories

```bash
# Define service-specific directory structure
MOUNT_BASE="/mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}"
REQUIRED_DIRS=(
    "$MOUNT_BASE/data/app-specific"
    "$MOUNT_BASE/config/templates"
    "$MOUNT_BASE/state/logs"
)

log_with_prefix "ServiceName" "Creating required directories"
for dir in "${REQUIRED_DIRS[@]}"; do
    if [[ ! -d "$dir" ]]; then
        mkdir -p "$dir"
        log_with_prefix "ServiceName" "Created: $dir"
    fi
done
```

### 2. Create Configuration Templates

```bash
# Create default configuration if it doesn't exist
CONFIG_DIR="$MOUNT_BASE/config"
if [[ ! -f "$CONFIG_DIR/app.conf" ]]; then
    log_with_prefix "ServiceName" "Creating default configuration"
    cat > "$CONFIG_DIR/app.conf" << 'EOF'
# Default configuration for ServiceName
port=${TIN_PORT_0}
data_dir=/workdir/data
log_level=info
EOF
fi
```

### 3. Create User Convenience Tools

```bash
# Create aliases for easy service management
SERVICE_USER="${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}"
SERVICE_HOME="/home/$SERVICE_USER"

if [[ -d "$SERVICE_HOME" ]]; then
    log_with_prefix "ServiceName" "Creating convenience aliases"
    cat > "$SERVICE_HOME/.service_aliases" << EOF
# ServiceName convenience aliases
# IMPORTANT: Always include --env-file flag for tinsnip services
alias service-logs='docker compose --env-file /mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}/.env logs -f servicename'
alias service-shell='docker compose --env-file /mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}/.env exec servicename bash'
alias service-restart='docker compose --env-file /mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}/.env restart servicename'

echo "ServiceName aliases loaded. Available commands:"
echo "  service-logs    - View service logs"
echo "  service-shell   - Interactive shell"  
echo "  service-restart - Restart service"
EOF

    # Add to .bashrc if not already there
    if ! grep -q ".service_aliases" "$SERVICE_HOME/.bashrc" 2>/dev/null; then
        echo "" >> "$SERVICE_HOME/.bashrc"
        echo "# ServiceName aliases" >> "$SERVICE_HOME/.bashrc"
        echo "source ~/.service_aliases" >> "$SERVICE_HOME/.bashrc"
        log_with_prefix "ServiceName" "Added aliases to .bashrc"
    fi
fi
```

## Best Practices

### Do:
- ✅ **Source .env file first** - Before using any TIN_* variables
- ✅ **Use self-contained functions** - Don't depend on tinsnip lib.sh
- ✅ **Check if files/dirs exist** - Make script idempotent
- ✅ **Use absolute paths** - Relative paths can be unreliable
- ✅ **Log all actions** - Help with debugging
- ✅ **Use heredocs for templates** - Cleaner than echo statements
- ✅ **Follow XDG structure** - data/, config/, state/ organization
- ✅ **Let containers run as root** - Most images expect root for initialization

### Don't:
- ❌ **Don't use sudo** - Service user doesn't have sudo access
- ❌ **Don't use chown** - NFS all_squash handles ownership
- ❌ **Don't depend on external tools** - Keep scripts portable
- ❌ **Don't hardcode usernames/paths** - Use environment variables
- ❌ **Don't assume directories exist** - Always check first
- ❌ **Don't modify system files** - Stay within service boundaries

### Script Execution Context

Setup scripts run as the service user in this context:
```bash
# Working directory: /mnt/service-env/service/servicename
# User: service-env (e.g., nrfconnect-prod)
# UID: Service-specific UID (e.g., 50300)
# Environment: Variables from /mnt/service-env/.env
```

## Testing Setup Scripts

Test your setup script manually:

```bash
# Switch to service user
sudo -u service-environment -i

# Navigate to service directory  
cd /mnt/service-environment/service/servicename

# Run setup script
./setup.sh

# Verify results
ls -la /mnt/service-environment/
cat ~/.bashrc
source ~/.service_aliases  # Test aliases
```

## Examples

### Simple Service (No Special Setup)
```bash
#!/bin/bash
set -euo pipefail
source "/mnt/${USER}/.env"
log_with_prefix() { echo "[$1] $2"; }
log_with_prefix "SimpleService" "No additional setup required"
```

### Database Service
```bash
#!/bin/bash
set -euo pipefail
source "/mnt/${USER}/.env"
log_with_prefix() { echo "[$1] $2"; }

MOUNT_BASE="/mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}"

# Create database directory
mkdir -p "$MOUNT_BASE/data/database"
mkdir -p "$MOUNT_BASE/config"

# Create default database config
if [[ ! -f "$MOUNT_BASE/config/db.conf" ]]; then
    cat > "$MOUNT_BASE/config/db.conf" << EOF
port=${TIN_PORT_0}
data_directory=/workdir/data/database
log_directory=/workdir/state/logs
EOF
fi

log_with_prefix "Database" "Setup complete"
```

### Development Environment
```bash
#!/bin/bash
set -euo pipefail
source "/mnt/${USER}/.env"
log_with_prefix() { echo "[$1] $2"; }

MOUNT_BASE="/mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}"

# Create development workspace
mkdir -p "$MOUNT_BASE/data/projects"
mkdir -p "$MOUNT_BASE/data/builds"
mkdir -p "$MOUNT_BASE/state/.config"

# Create development aliases
SERVICE_HOME="/home/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}"
if [[ -d "$SERVICE_HOME" ]]; then
    cat > "$SERVICE_HOME/.dev_aliases" << 'EOF'
alias dev-build='docker compose exec dev make build'
alias dev-test='docker compose exec dev make test'
alias dev-shell='docker compose exec dev bash'
EOF
    
    if ! grep -q ".dev_aliases" "$SERVICE_HOME/.bashrc" 2>/dev/null; then
        echo "source ~/.dev_aliases" >> "$SERVICE_HOME/.bashrc"
    fi
fi

log_with_prefix "DevEnv" "Development environment ready"
```

## Integration with Docker Compose

Setup scripts run before `docker compose up`, so they can create files that containers will mount:

```yaml
# docker-compose.yml
volumes:
  - /mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}/config/app.conf:/app/config.conf:ro
  - /mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}/data:/app/data
```

The setup script ensures `config/app.conf` exists before the container starts.

### Volume Mount Alignment

**Critical**: Ensure volume mounts in docker-compose.yml match directories created by setup.sh:

```bash
# In setup.sh
mkdir -p "$MOUNT_BASE/data/west-workspace"

# In docker-compose.yml  
volumes:
  - /mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}/data/west-workspace:/workdir/west
```

Mismatched directory names will cause mount failures.

### Container User Management

**Recommended Approach**: Let containers run as root initially, then drop to service user.

Most Docker images are designed to:
1. Start as root to handle initialization (create dirs, set permissions, install deps)  
2. Drop to non-root user for normal operation

```yaml
# Recommended: Let container handle user management
# user: "${TIN_SERVICE_UID}:${TIN_SERVICE_UID}"  # Comment out

# Alternative: Force specific UID (may cause issues with some images)
user: "${TIN_SERVICE_UID}:${TIN_SERVICE_UID}"
```

**Why this works with tinsnip:**
- NFS `all_squash` handles file ownership on the host side
- Container can start as root, do setup, then switch users internally
- More compatible with existing Docker ecosystem

**Home Directory:**
```yaml
# Container expects writable home directory
environment:
  - HOME=/workdir/home
volumes:
  - /mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}/state:/workdir/home
```

**Environment File Requirement:**

Tinsnip places the service `.env` file at `/mnt/service-environment/.env`. When running `docker compose` manually, always specify:

```bash
# Required pattern for manual docker compose commands
docker compose --env-file /mnt/service-environment/.env [command]

# Examples
docker compose --env-file /mnt/nrfconnect-prod/.env ps
docker compose --env-file /mnt/nrfconnect-prod/.env exec service bash
docker compose --env-file /mnt/nrfconnect-prod/.env logs -f
```

**Common Issues:**
- Permission denied errors → Remove `user:` constraint, let container run as root initially
- Logger/config failures → Ensure HOME directory is mounted and writable  
- Missing directories → Check setup.sh creates all directories referenced in docker-compose.yml
- Image won't start → Many images expect root access for initialization
- "Variable not set" warnings → Always use `--env-file` flag when running docker compose manually

## Error Handling

```bash
# Good error handling pattern
if ! mkdir -p "$REQUIRED_DIR" 2>/dev/null; then
    log_with_prefix "ServiceName" "ERROR: Could not create $REQUIRED_DIR"
    exit 1
fi

# Check for required tools
if ! command -v some-tool >/dev/null 2>&1; then
    log_with_prefix "ServiceName" "WARNING: some-tool not found, skipping optional setup"
fi
```

## Working with Third-Party Containers

Many third-party containers require special handling beyond basic Docker Compose configuration.

### Container Initialization Requirements

**Development toolchain containers** (like Nordic nRF Connect SDK, Xilinx Vivado, etc.) often need initialization:

```bash
# Common pattern: containers expect toolchains/SDKs to be installed on first run
# Inside container (during debugging):
container-tool install --version latest
container-tool setup --workspace /workdir

# This may take 30-60 minutes for large toolchain downloads
```

**Key strategies:**
- **Use persistent storage** - Install to NFS-mounted directories so installation survives restarts
- **Expect long initialization** - Large toolchain downloads can take hours
- **Test interactively first** - Use `docker compose run --rm --entrypoint="" service bash` to debug

### Container Command Patterns

**Don't override entrypoints unless necessary:**
```yaml
# Bad: Overrides container's initialization logic
command: ["tail", "-f", "/dev/null"]

# Good: Let container handle its own setup
# command: # commented out - use container default

# Better: Use container's recommended shell launcher
command: ["container-tool", "launch", "--shell"]
```

### Debugging Container Issues

**Step 1: Get a working shell**
```bash
# Override command to get basic shell access
docker compose run --rm --entrypoint="" service bash

# Inside container, explore what's available:
ls /usr/local/bin/    # Check installed tools
env                   # Check environment variables
service-tool --help   # Test main application
```

**Step 2: Understand initialization requirements**
```bash
# Common checks inside container:
which main-tool                    # Is primary tool available?
main-tool list-installed          # What components are installed?  
main-tool search-available        # What needs to be installed?
ls -la /opt/ /usr/local/share/     # Check installation directories
```

**Step 3: Install missing components**
```bash
# Install to persistent storage (NFS-mounted)
main-tool install --install-dir /workdir/home/tools
main-tool setup --workspace /workdir
```

**Step 4: Update service configuration**
```yaml
# Once you understand requirements, update docker-compose.yml:
volumes:
  # Persist large installations
  - /mnt/service-env/state:/workdir/home
environment:
  # Set tool-specific environment variables
  - TOOL_HOME=/workdir/home/tools
command: ["main-tool", "launch", "--shell"]  # Use proper launcher
```

### Common Third-Party Container Patterns

**SDK/Toolchain containers** (nRF Connect SDK, ESP-IDF, etc.):
- Expect user to install specific SDK versions
- May need 1-10GB of downloads and storage  
- Usually provide launcher commands for proper environment

**Database containers** (PostgreSQL, MongoDB, etc.):
- May need initialization scripts run on first start
- Often create users/databases from environment variables
- Check logs for initialization completion

**IDE/Development containers** (VSCode, Jupyter, etc.):
- May need extensions or plugins installed
- Often expose web interfaces on specific ports
- May need authentication setup

### Performance Considerations

**Large downloads:**
- SDK installations can take 30-60+ minutes
- Use persistent storage to avoid re-downloading  
- Consider running installation during setup.sh instead of container startup

**Resource usage:**
- Development containers often need significant RAM/CPU
- Monitor with `docker stats` during heavy operations
- Consider resource limits in docker-compose.yml

## Debugging

If setup scripts fail:

1. **Check the logs** - Setup output is shown during deployment
2. **Run manually** - Switch to service user and run `./setup.sh`
3. **Check permissions** - Ensure service user can write to directories
4. **Verify .env file** - `cat /mnt/service-env/.env`
5. **Test environment loading** - `source /mnt/service-env/.env; env | grep TIN_`

**For container issues:**
6. **Get interactive shell** - `docker compose --env-file /mnt/service-env/.env run --rm --entrypoint="" service bash`
7. **Check container tools** - Verify expected applications are installed and working
8. **Review container logs** - `docker compose --env-file /mnt/service-env/.env logs service`
9. **Test default command** - Remove custom `command:` to see container's normal behavior

## See Also

- [CLAUDE.md](CLAUDE.md) - Project architecture overview
- [DEPLOYMENT_STRATEGY.md](DEPLOYMENT_STRATEGY.md) - Complete deployment workflow
- [SHEET_CONFIGURATION.md](SHEET_CONFIGURATION.md) - Multi-tenant configuration
- Service examples in `./service/` directory