Service Setup Script Guide#

This guide covers creating setup.sh scripts for tinsnip services. Setup scripts prepare the service environment after machine infrastructure is created but before the service containers start.

Overview#

Service setup scripts run once during service deployment to:

Create required directories and files
Set up configuration templates
Initialize service-specific data structures
Create convenience utilities for the service user

Script Structure#

Basic Template#

#!/bin/bash
# [Service Name] Service Setup
# Brief description of what this service does

set -euo pipefail

# Source service environment variables
if [[ -f "/mnt/${USER}/.env" ]]; then
    source "/mnt/${USER}/.env"
elif [[ -f "../../../.env" ]]; then
    source "../../../.env"
else
    echo "Error: Could not find service .env file" >&2
    exit 1
fi

# Simple logging function (self-contained)
log_with_prefix() {
    local prefix="$1"
    local message="$2"
    echo "[$prefix] $message"
}

log_with_prefix "ServiceName" "Setting up service environment"

# Your setup logic here...

log_with_prefix "ServiceName" "Setup complete!"

Available Environment Variables#

The service .env file provides these standard variables:

TIN_SERVICE_NAME        # Service name (e.g., "nrfconnect")
TIN_SERVICE_ENVIRONMENT # Environment (e.g., "prod", "test")
TIN_SERVICE_UID         # Service user UID (e.g., 50300)
TIN_SHEET              # Sheet name (e.g., "topsheet")

# Ports (if service uses them)
TIN_PORT_0             # Base port (usually same as UID)
TIN_PORT_1             # Additional ports as needed
TIN_PORT_2             # ...

# XDG directories (NFS-backed)
XDG_DATA_HOME          # /mnt/service-env/data
XDG_CONFIG_HOME        # /mnt/service-env/config  
XDG_STATE_HOME         # /mnt/service-env/state

# Docker environment
DOCKER_HOST            # unix:///run/user/UID/docker.sock
XDG_RUNTIME_DIR        # /run/user/UID

Directory Structure#

Services follow the XDG Base Directory specification with NFS backing:

/mnt/service-environment/
├── data/          # Persistent application data (XDG_DATA_HOME)
├── config/        # Configuration files (XDG_CONFIG_HOME)
├── state/         # Logs, runtime state (XDG_STATE_HOME)
└── service/       # Service definitions (docker-compose.yml, etc.)

Directory Usage Patterns#

data/ - Database files, user content, application-specific data
config/ - Configuration templates, settings files
state/ - Log files, session data

Common Setup Tasks#

1. Create Required Directories#

# Define service-specific directory structure
MOUNT_BASE="/mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}"
REQUIRED_DIRS=(
    "$MOUNT_BASE/data/app-specific"
    "$MOUNT_BASE/config/templates"
    "$MOUNT_BASE/state/logs"
)

log_with_prefix "ServiceName" "Creating required directories"
for dir in "${REQUIRED_DIRS[@]}"; do
    if [[ ! -d "$dir" ]]; then
        mkdir -p "$dir"
        log_with_prefix "ServiceName" "Created: $dir"
    fi
done

2. Create Configuration Templates#

# Create default configuration if it doesn't exist
CONFIG_DIR="$MOUNT_BASE/config"
if [[ ! -f "$CONFIG_DIR/app.conf" ]]; then
    log_with_prefix "ServiceName" "Creating default configuration"
    cat > "$CONFIG_DIR/app.conf" << 'EOF'
# Default configuration for ServiceName
port=${TIN_PORT_0}
data_dir=/workdir/data
log_level=info
EOF
fi

3. Create User Convenience Tools#

# Create aliases for easy service management
SERVICE_USER="${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}"
SERVICE_HOME="/home/$SERVICE_USER"

if [[ -d "$SERVICE_HOME" ]]; then
    log_with_prefix "ServiceName" "Creating convenience aliases"
    cat > "$SERVICE_HOME/.service_aliases" << EOF
# ServiceName convenience aliases
# IMPORTANT: Always include --env-file flag for tinsnip services
alias service-logs='docker compose --env-file /mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}/.env logs -f servicename'
alias service-shell='docker compose --env-file /mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}/.env exec servicename bash'
alias service-restart='docker compose --env-file /mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}/.env restart servicename'

echo "ServiceName aliases loaded. Available commands:"
echo "  service-logs    - View service logs"
echo "  service-shell   - Interactive shell"  
echo "  service-restart - Restart service"
EOF

    # Add to .bashrc if not already there
    if ! grep -q ".service_aliases" "$SERVICE_HOME/.bashrc" 2>/dev/null; then
        echo "" >> "$SERVICE_HOME/.bashrc"
        echo "# ServiceName aliases" >> "$SERVICE_HOME/.bashrc"
        echo "source ~/.service_aliases" >> "$SERVICE_HOME/.bashrc"
        log_with_prefix "ServiceName" "Added aliases to .bashrc"
    fi
fi

Best Practices#

Do:#

✅ Source .env file first - Before using any TIN_* variables
✅ Use self-contained functions - Don't depend on tinsnip lib.sh
✅ Check if files/dirs exist - Make script idempotent
✅ Use absolute paths - Relative paths can be unreliable
✅ Log all actions - Help with debugging
✅ Use heredocs for templates - Cleaner than echo statements
✅ Follow XDG structure - data/, config/, state/ organization
✅ Let containers run as root - Most images expect root for initialization

Don't:#

❌ Don't use sudo - Service user doesn't have sudo access
❌ Don't use chown - NFS all_squash handles ownership
❌ Don't depend on external tools - Keep scripts portable
❌ Don't hardcode usernames/paths - Use environment variables
❌ Don't assume directories exist - Always check first
❌ Don't modify system files - Stay within service boundaries

Script Execution Context#

Setup scripts run as the service user in this context:

# Working directory: /mnt/service-env/service/servicename
# User: service-env (e.g., nrfconnect-prod)
# UID: Service-specific UID (e.g., 50300)
# Environment: Variables from /mnt/service-env/.env

Testing Setup Scripts#

Test your setup script manually:

# Switch to service user
sudo -u service-environment -i

# Navigate to service directory  
cd /mnt/service-environment/service/servicename

# Run setup script
./setup.sh

# Verify results
ls -la /mnt/service-environment/
cat ~/.bashrc
source ~/.service_aliases  # Test aliases

Examples#

Simple Service (No Special Setup)#

#!/bin/bash
set -euo pipefail
source "/mnt/${USER}/.env"
log_with_prefix() { echo "[$1] $2"; }
log_with_prefix "SimpleService" "No additional setup required"

Database Service#

#!/bin/bash
set -euo pipefail
source "/mnt/${USER}/.env"
log_with_prefix() { echo "[$1] $2"; }

MOUNT_BASE="/mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}"

# Create database directory
mkdir -p "$MOUNT_BASE/data/database"
mkdir -p "$MOUNT_BASE/config"

# Create default database config
if [[ ! -f "$MOUNT_BASE/config/db.conf" ]]; then
    cat > "$MOUNT_BASE/config/db.conf" << EOF
port=${TIN_PORT_0}
data_directory=/workdir/data/database
log_directory=/workdir/state/logs
EOF
fi

log_with_prefix "Database" "Setup complete"

Development Environment#

#!/bin/bash
set -euo pipefail
source "/mnt/${USER}/.env"
log_with_prefix() { echo "[$1] $2"; }

MOUNT_BASE="/mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}"

# Create development workspace
mkdir -p "$MOUNT_BASE/data/projects"
mkdir -p "$MOUNT_BASE/data/builds"
mkdir -p "$MOUNT_BASE/state/.config"

# Create development aliases
SERVICE_HOME="/home/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}"
if [[ -d "$SERVICE_HOME" ]]; then
    cat > "$SERVICE_HOME/.dev_aliases" << 'EOF'
alias dev-build='docker compose exec dev make build'
alias dev-test='docker compose exec dev make test'
alias dev-shell='docker compose exec dev bash'
EOF
    
    if ! grep -q ".dev_aliases" "$SERVICE_HOME/.bashrc" 2>/dev/null; then
        echo "source ~/.dev_aliases" >> "$SERVICE_HOME/.bashrc"
    fi
fi

log_with_prefix "DevEnv" "Development environment ready"

Integration with Docker Compose#

Setup scripts run before docker compose up, so they can create files that containers will mount:

# docker-compose.yml
volumes:
  - /mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}/config/app.conf:/app/config.conf:ro
  - /mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}/data:/app/data

The setup script ensures config/app.conf exists before the container starts.

Volume Mount Alignment#

Critical: Ensure volume mounts in docker-compose.yml match directories created by setup.sh:

# In setup.sh
mkdir -p "$MOUNT_BASE/data/west-workspace"

# In docker-compose.yml  
volumes:
  - /mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}/data/west-workspace:/workdir/west

Mismatched directory names will cause mount failures.

Container User Management#

Recommended Approach: Let containers run as root initially, then drop to service user.

Most Docker images are designed to:

Start as root to handle initialization (create dirs, set permissions, install deps)
Drop to non-root user for normal operation

# Recommended: Let container handle user management
# user: "${TIN_SERVICE_UID}:${TIN_SERVICE_UID}"  # Comment out

# Alternative: Force specific UID (may cause issues with some images)
user: "${TIN_SERVICE_UID}:${TIN_SERVICE_UID}"

Why this works with tinsnip:

NFS all_squash handles file ownership on the host side
Container can start as root, do setup, then switch users internally
More compatible with existing Docker ecosystem

Home Directory:

# Container expects writable home directory
environment:
  - HOME=/workdir/home
volumes:
  - /mnt/${TIN_SERVICE_NAME}-${TIN_SERVICE_ENVIRONMENT}/state:/workdir/home

Environment File Requirement:

Tinsnip places the service .env file at /mnt/service-environment/.env. When running docker compose manually, always specify:

# Required pattern for manual docker compose commands
docker compose --env-file /mnt/service-environment/.env [command]

# Examples
docker compose --env-file /mnt/nrfconnect-prod/.env ps
docker compose --env-file /mnt/nrfconnect-prod/.env exec service bash
docker compose --env-file /mnt/nrfconnect-prod/.env logs -f

Common Issues:

Permission denied errors → Remove user: constraint, let container run as root initially
Logger/config failures → Ensure HOME directory is mounted and writable
Missing directories → Check setup.sh creates all directories referenced in docker-compose.yml
Image won't start → Many images expect root access for initialization
"Variable not set" warnings → Always use --env-file flag when running docker compose manually

Error Handling#

# Good error handling pattern
if ! mkdir -p "$REQUIRED_DIR" 2>/dev/null; then
    log_with_prefix "ServiceName" "ERROR: Could not create $REQUIRED_DIR"
    exit 1
fi

# Check for required tools
if ! command -v some-tool >/dev/null 2>&1; then
    log_with_prefix "ServiceName" "WARNING: some-tool not found, skipping optional setup"
fi

Working with Third-Party Containers#

Many third-party containers require special handling beyond basic Docker Compose configuration.

Container Initialization Requirements#

Development toolchain containers (like Nordic nRF Connect SDK, Xilinx Vivado, etc.) often need initialization:

# Common pattern: containers expect toolchains/SDKs to be installed on first run
# Inside container (during debugging):
container-tool install --version latest
container-tool setup --workspace /workdir

# This may take 30-60 minutes for large toolchain downloads

Key strategies:

Use persistent storage - Install to NFS-mounted directories so installation survives restarts
Expect long initialization - Large toolchain downloads can take hours
Test interactively first - Use docker compose run --rm --entrypoint="" service bash to debug

Container Command Patterns#

Don't override entrypoints unless necessary:

# Bad: Overrides container's initialization logic
command: ["tail", "-f", "/dev/null"]

# Good: Let container handle its own setup
# command: # commented out - use container default

# Better: Use container's recommended shell launcher
command: ["container-tool", "launch", "--shell"]

Debugging Container Issues#

Step 1: Get a working shell

# Override command to get basic shell access
docker compose run --rm --entrypoint="" service bash

# Inside container, explore what's available:
ls /usr/local/bin/    # Check installed tools
env                   # Check environment variables
service-tool --help   # Test main application

Step 2: Understand initialization requirements

# Common checks inside container:
which main-tool                    # Is primary tool available?
main-tool list-installed          # What components are installed?  
main-tool search-available        # What needs to be installed?
ls -la /opt/ /usr/local/share/     # Check installation directories

Step 3: Install missing components

# Install to persistent storage (NFS-mounted)
main-tool install --install-dir /workdir/home/tools
main-tool setup --workspace /workdir

Step 4: Update service configuration

# Once you understand requirements, update docker-compose.yml:
volumes:
  # Persist large installations
  - /mnt/service-env/state:/workdir/home
environment:
  # Set tool-specific environment variables
  - TOOL_HOME=/workdir/home/tools
command: ["main-tool", "launch", "--shell"]  # Use proper launcher

Common Third-Party Container Patterns#

SDK/Toolchain containers (nRF Connect SDK, ESP-IDF, etc.):

Expect user to install specific SDK versions
May need 1-10GB of downloads and storage
Usually provide launcher commands for proper environment

Database containers (PostgreSQL, MongoDB, etc.):

May need initialization scripts run on first start
Often create users/databases from environment variables
Check logs for initialization completion

IDE/Development containers (VSCode, Jupyter, etc.):

May need extensions or plugins installed
Often expose web interfaces on specific ports
May need authentication setup

Performance Considerations#

Large downloads:

SDK installations can take 30-60+ minutes
Use persistent storage to avoid re-downloading
Consider running installation during setup.sh instead of container startup

Resource usage:

Development containers often need significant RAM/CPU
Monitor with docker stats during heavy operations
Consider resource limits in docker-compose.yml

Debugging#

If setup scripts fail:

Check the logs - Setup output is shown during deployment
Run manually - Switch to service user and run ./setup.sh
Check permissions - Ensure service user can write to directories
Verify .env file - cat /mnt/service-env/.env
Test environment loading - source /mnt/service-env/.env; env | grep TIN_

For container issues: 6. Get interactive shell - docker compose --env-file /mnt/service-env/.env run --rm --entrypoint="" service bash 7. Check container tools - Verify expected applications are installed and working 8. Review container logs - docker compose --env-file /mnt/service-env/.env logs service 9. Test default command - Remove custom command: to see container's normal behavior