tinsnip / docs / create_machine.md

# Creating a New Machine

This guide covers setting up a new machine in the tinsnip infrastructure, including prerequisites, NFS setup, machine configuration, and testing.

## Decisions to Make First

Before starting, you need to decide on these key inputs:

### 1. Service Name
Give the thing you are making a name - this might be the product name (lldap or caddy) or a name which wraps a product of your own like gazette or daffodyl.
- Currently configured: `tinsnip`, `gazette`
- You can add more in the future

### 2. Environment
Decide if this is for production use or testing. This affects which UID and ports get assigned.
- `prod` - Production environment (UID: xxx00)
- `test` - Test environment (UID: xxx10)

### 3. NAS Server
Provide the hostname or IP address of your NAS where data will be stored.
- Hostname example: `DS412plus`, `nas.local`
- IP example: `192.168.1.100`

### 4. Sheet (Optional)
If you're running services for multiple organizations or want to separate your services from the default, set a custom sheet.
- Default: `dynamicalsystem`
- Custom: Your organization name (e.g., `mycompany`)

## Prerequisites

### Machine Requirements
- **OS**: Ubuntu 20.04 or later (tested on 20.04 and 22.04)
- **User**: Account with sudo privileges
- **Network**: Access to NAS and internet for Docker installation
- **Resources**: At least 2GB RAM, 10GB disk space

### NAS Requirements
- **Synology NAS** with DSM 6.x or later
- **SSH access** to NAS with administrative privileges
- **Available storage** for service data

### Verify Prerequisites

```bash
# Check Ubuntu version
lsb_release -a

# Check sudo access
sudo -v && echo "✓ sudo access confirmed" || echo "✗ Need sudo access"

# Test NAS connectivity
ping -c 1 your-nas-hostname
```

## Overview

Creating a new machine involves:
1. **Sheet Configuration**: Set the organizational sheet (defaults to `dynamicalsystem`)
2. **NAS Setup**: Create NFS exports with proper UID mapping
3. **Machine Setup**: Install and configure the machine environment
4. **Service Deployment**: Deploy services with proper isolation
5. **Testing**: Validate the complete setup


## Step 1: Tinsnip Sheet Configuration

The system supports configurable sheets for multi-tenant deployments. The sheet affects NFS paths and XDG directory organization.

### Setting the Sheet

**Option 1: Environment Variable**
```bash
export TIN_SHEET=mycompany
./machine/setup.sh tinsnip test DS412plus
```

**Option 2: System-wide Configuration**
```bash
echo "mycompany" | sudo tee /etc/tinsnip-sheet
./machine/setup.sh tinsnip test DS412plus
```

**Default**: If no sheet is configured, `dynamicalsystem` is used.

For detailed sheet configuration options, see [SHEET_CONFIGURATION.md](SHEET_CONFIGURATION.md).

## Step 2: NAS Setup (Critical)

### Create Directory Structure

On your Synology NAS, create the directory structure for your sheet:

```bash
# For topsheet (default sheet, most users deploy here)
/volume1/topsheet/
├── station/
│   ├── prod/  (will be owned by UID 50000)
│   └── test/  (will be owned by UID 50010)
└── gazette/
    ├── prod/  (will be owned by UID 50100)
    └── test/  (will be owned by UID 50110)

# For custom sheet (infrastructure)
/volume1/tinsnip/infrastructure/
├── station/
│   ├── prod/  (will be owned by UID 10000)
│   └── test/  (will be owned by UID 10010)
└── gazette/
    ├── prod/  (will be owned by UID 11000)
    └── test/  (will be owned by UID 11110)
```

**Create directories and set ownership:**
```bash
# SSH into NAS
ssh admin@synology-ip

# Create directory structure (replace {sheet} with your sheet)
sudo mkdir -p /volume1/{sheet}/{tinsnip,gazette}/{prod,test}

# Set ownership to match service UIDs
sudo chown 11000:11000 /volume1/{sheet}/tinsnip/prod
sudo chown 11010:11010 /volume1/{sheet}/tinsnip/test
sudo chown 11100:11100 /volume1/{sheet}/gazette/prod
sudo chown 11110:11110 /volume1/{sheet}/gazette/test

# Example for custom sheet:
sudo mkdir -p /volume1/mycompany/{tinsnip,gazette}/{prod,test}
sudo chown 11000:11000 /volume1/mycompany/tinsnip/prod
sudo chown 11010:11010 /volume1/mycompany/tinsnip/test
sudo chown 11100:11100 /volume1/mycompany/gazette/prod
sudo chown 11110:11110 /volume1/mycompany/gazette/test
```

**Important**: The `chown` commands ensure the directories are owned by the correct UIDs before NFS export. This is critical for proper permissions.

### Configure NFS Exports

**IMPORTANT**: The Synology DSM GUI cannot map to specific UIDs (11000+), so you must configure exports via SSH.

#### Option 1: Generate Export Rules Automatically (Recommended)

Use the NFS export generator to create the exact rules you need:

```bash
# Interactive mode - prompts for all inputs
cd machine
./generate_nfs_exports.sh

# Direct mode - provide all parameters
./generate_nfs_exports.sh tinsnip test dynamicalsystem ubuntu-machine.local

# Example output shows:
# - Directory creation commands
# - Ownership commands  
# - Export rules to add to /etc/exports
# - Test commands
```

#### Option 2: Manual Configuration

1. **SSH into Synology NAS:**
```bash
ssh admin@synology-ip
```

2. **Edit /etc/exports file:**
```bash
sudo vi /etc/exports
```

3. **Add export rules (CRITICAL: Use TAB characters between path and options):**

Use the output from the generator above, or manually create rules like:
```
/volume1/{sheet}/{service}/{env}	{client-host}(rw,async,no_subtree_check,all_squash,anonuid={uid},anongid={uid})
```

4. **Reload NFS exports:**
```bash
sudo exportfs -ra
```

5. **Verify exports are active:**
```bash
sudo exportfs -v
```

## Step 3: Machine Setup

### Install Machine Setup Tools

The machine setup tools provide automated scripts for creating service users, mounting NFS, and installing rootless Docker.

```bash
# One-line installation - downloads scripts to ~/.local/opt/dynamicalsystem.machine
curl -fsSL "https://tangled.sh/dynamicalsystem.com/machine/raw/main/install.sh?$(date +%s)" | bash

# Navigate to installation directory
cd ~/.local/opt/dynamicalsystem.machine
```

**What this installs:**
- `setup.sh` - Main orchestration script for service environment creation
- `scripts/` - Helper scripts for NFS mounting, Docker installation, etc.
- `machine/` - Service-specific setup scripts with sheet support

### Run Validations (Optional)

Before making any system changes, you can optionally validate the setup logic:

```bash
cd machine
./validate.sh
```

Running validations is recommended but not required.

### Create Service Environment

Run the setup script from the machine directory:

**Interactive setup (recommended for first time):**
```bash
cd machine
./setup.sh
# Will prompt for:
# - Service name (tinsnip or gazette)
# - Environment (test or prod)
# - NAS server hostname/IP
```

**Direct setup (if you know the parameters):**
```bash
cd machine
./setup.sh tinsnip test DS412plus    # Creates tinsnip-test environment
./setup.sh gazette prod 192.168.1.100  # Creates gazette-prod environment

# With custom sheet:
TIN_SHEET=mycompany ./setup.sh tinsnip prod nas.mycompany.local
```

### What Gets Created

The setup creates:

1. **Service User**: `tinsnip-test` (UID 11010) with dedicated home directory
2. **NFS Mount**: `/mnt/tinsnip-test` → `DS412plus:/volume1/{sheet}/tinsnip/test`
3. **XDG Integration**: Symlinks in `~/.local/{state,share,config}/{sheet}/@tinsnip`
4. **Rootless Docker**: Installed for service user with privileged port support
5. **Directory Structure**:
   ```
   /mnt/tinsnip-test/
   ├── state/     # Service state (logs, history)
   ├── data/      # Service data files
   └── config/    # Service configuration
   ```

## Step 4: Service Deployment

### Switch to Service User

```bash
# Switch to service user context
sudo -u tinsnip-test -i
```

### Deploy Service

Create service configuration in the mounted NFS directory:

```bash
cd /mnt/tinsnip-test
mkdir -p service/lldap

# Create docker-compose.yml for the service
cat > service/lldap/docker-compose.yml << 'EOF'
services:
  lldap:
    image: lldap/lldap:stable
    ports:
      - "11010:3890"    # LDAP port (UID-based)
      - "11011:17170"   # Web UI port (UID-based)
    volumes:
      - ../data:/data
    user: "11010:11010"  # Run as service UID
    environment:
      - LLDAP_JWT_SECRET_FILE=/data/jwt_secret
      - LLDAP_KEY_SEED_FILE=/data/key_seed
      - LLDAP_BASE_DN=dc=home,dc=local
EOF

# Deploy the service
cd service/lldap
docker compose up -d
```

## Step 5: Testing

### Basic Validation

```bash
# Check service user
id tinsnip-test  # Should show UID 11010

# Verify NFS mount
mount | grep /mnt/tinsnip-test
ls -la /mnt/tinsnip-test/  # Should show correct ownership

# Check XDG symlinks
ls -la ~/.local/state/dynamicalsystem/@tinsnip  # Should point to /mnt/tinsnip-test/state

# Test Docker
sudo -u tinsnip-test docker --version
sudo -u tinsnip-test docker run hello-world
```

### Service Testing

```bash
# Check service status
sudo -u tinsnip-test -i
cd /mnt/tinsnip-test/service/lldap
docker compose ps

# Test service connectivity
curl -I http://localhost:11011  # Web UI
ldapsearch -H ldap://localhost:11010 -b "dc=home,dc=local"  # LDAP
```

### VM Testing Plan

For comprehensive testing, follow the [VM Test Plan](machine/VM_TEST_PLAN.md):

1. **Function Tests**: Validate UID calculation and port allocation
2. **Partial Tests**: Test user creation and NFS connectivity 
3. **Full Setup**: Complete service environment creation
4. **Service Deployment**: Deploy and test actual services

## Troubleshooting

### NFS Mount Issues

```bash
# Test NFS connectivity manually (adjust path for your sheet)
sudo mount -t nfs DS412plus:/volume1/{sheet}/tinsnip/test /tmp/test-mount
ls -la /tmp/test-mount  # Should show UID 11010 ownership
sudo umount /tmp/test-mount

# Check NFS exports on server
ssh admin@DS412plus sudo exportfs -v
```

### Permission Issues

```bash
# Verify UID mapping
sudo -u tinsnip-test touch /mnt/tinsnip-test/test-file
ls -la /mnt/tinsnip-test/test-file  # Should show UID 11010

# Check from NAS side (adjust path for your sheet)
ssh admin@DS412plus ls -la /volume1/{sheet}/tinsnip/test/
```

### Docker Issues

```bash
# Check rootless Docker status
sudo -u tinsnip-test systemctl --user status docker

# Test privileged ports
sudo -u tinsnip-test docker run -p 80:80 nginx:alpine
```

## Rollback Plan

If setup fails or needs to be undone:

```bash
# Stop containers
sudo -u tinsnip-test -i bash -c 'docker stop $(docker ps -q)' || true

# Remove service user
sudo pkill -u tinsnip-test || true
sudo userdel -r tinsnip-test

# Unmount NFS
sudo umount /mnt/tinsnip-test || true
sudo rm -rf /mnt/tinsnip-test

# Clean up XDG symlinks (adjust paths for your sheet)
rm -rf ~/.local/state/{sheet}/@tinsnip
rm -rf ~/.local/share/{sheet}/@tinsnip  
rm -rf ~/.config/{sheet}/@tinsnip
```

## Security Considerations

- Each service runs under dedicated UID for complete isolation
- NFS `all_squash` ensures consistent file ownership
- Rootless Docker provides container isolation without root privileges
- Services cannot access each other's data due to UID separation

## Next Steps

After successful machine creation:

1. **Service Configuration**: Configure service-specific settings
2. **Integration Testing**: Test with other tinsnip services
3. **Monitoring Setup**: Add service monitoring and logging
4. **Backup Validation**: Ensure data backup processes work

For service-specific deployment patterns and UID conventions, see [DEPLOYMENT_STRATEGY.md](DEPLOYMENT_STRATEGY.md).