dynamicalsystem.com
homelab infrastructure services
1# Service Creation Guide
2
3This guide explains how to create services that work with tinsnip's NFS-backed persistence architecture.
4
5## Overview
6
7tinsnip provides standardized infrastructure for service deployment with:
8- **NFS-backed persistence**: Data survives machine rebuilds
9- **XDG integration**: Data accessible through standard Linux paths
10- **UID isolation**: Each service runs with dedicated user/permissions
11- **Port allocation**: Automatic port assignment based on service UID
12
13Services can be created using two patterns depending on their origin and requirements.
14
15## The tinsnip Target Pattern
16
17tinsnip establishes a standard environment that services can leverage:
18
19### Infrastructure Provided
20
21**NFS Mount Structure:**
22```
23/mnt/tinsnip/ # NFS mount point
24├── data/ # Persistent application data
25├── config/ # Service configuration files
26├── state/ # Service state (logs, databases, etc.)
27└── service/ # Docker compose location
28 └── myservice/
29 ├── docker-compose.yml
30 └── setup.sh (optional)
31```
32
33**Service Environment File (.env):**
34Generated by tinsnip setup with deployment-specific paths:
35```bash
36# Tinsnip deployment - direct NFS mounts
37XDG_DATA_HOME=/mnt/tinsnip/data
38XDG_CONFIG_HOME=/mnt/tinsnip/config
39XDG_STATE_HOME=/mnt/tinsnip/state
40
41# Service metadata
42TIN_SERVICE_NAME=myservice
43TIN_SERVICE_ENVIRONMENT=prod
44TIN_SERVICE_UID=11100
45PRIMARY_PORT=11100
46SECONDARY_PORT=11101
47```
48
49### Environment Variable Mapping
50
51| Environment Variable | Value (set in .env) | Container Path |
52|---------------------|---------------------|----------------|
53| `TIN_SERVICE_UID` | 11100 | Used for user |
54| `TIN_SERVICE_NAME` | myservice | - |
55| `TIN_SERVICE_ENVIRONMENT` | prod | - |
56| `TIN_NAMESPACE` | dynamicalsystem | - |
57| `PRIMARY_PORT` | 11100 | 11100 |
58| `SECONDARY_PORT` | 11101 | 11101 |
59| `XDG_DATA_HOME` | /mnt/tinsnip/data | /data |
60| `XDG_CONFIG_HOME` | /mnt/tinsnip/config | /config |
61| `XDG_STATE_HOME` | /mnt/tinsnip/state | /state |
62
63### Path Resolution
64
65| Host Path | Container Path | Description |
66|-----------|---------------|-------------|
67| `${XDG_DATA_HOME}/${TIN_NAMESPACE}/${TIN_SERVICE_NAME}` | `/data` | Application data |
68| `${XDG_CONFIG_HOME}/${TIN_NAMESPACE}/${TIN_SERVICE_NAME}` | `/config` | Configuration files |
69| `${XDG_STATE_HOME}/${TIN_NAMESPACE}/${TIN_SERVICE_NAME}` | `/state` | State/logs/cache |
70
71**Example Resolution:**
72```bash
73# For myservice-prod in dynamicalsystem namespace
74${XDG_DATA_HOME}/${TIN_NAMESPACE}/${TIN_SERVICE_NAME}
75↓ (from .env)
76/mnt/tinsnip/data/dynamicalsystem/myservice
77↓ (NFS mount)
78nas-server:/volume1/dynamicalsystem/myservice/prod/data
79```
80
81## Volume Requirements
82
83**⚠️ CRITICAL**: Services MUST use bind mounts to XDG-integrated NFS directories, not Docker named volumes.
84
85**✅ CORRECT (XDG + NFS-backed persistence):**
86```yaml
87volumes:
88 - ${XDG_DATA_HOME}/${TIN_NAMESPACE}/${TIN_SERVICE_NAME}:/app/data
89 - ${XDG_CONFIG_HOME}/${TIN_NAMESPACE}/${TIN_SERVICE_NAME}:/app/config
90 - ${XDG_STATE_HOME}/${TIN_NAMESPACE}/${TIN_SERVICE_NAME}:/app/state
91```
92
93**❌ INCORRECT (Local storage - data lost on rebuild):**
94```yaml
95volumes:
96 - myservice_data:/app/data # Stored locally, lost on rebuild
97volumes:
98 myservice_data: # Breaks continuous delivery
99```
100
101### Why This Matters
102
103**tinsnip's Value Proposition**: Continuous delivery with persistent data that survives machine rebuilds.
104
105- **With XDG Bind Mounts**: Data stored on NFS → Survives machine rebuilds → True continuous delivery
106- **With Named Volumes**: Data stored locally → Lost on rebuild → Breaks continuous delivery
107
108## Pattern 1: Home-grown Services
109
110**Use Case**: Services built specifically for tinsnip that can follow conventions natively.
111
112### Design Principles
113- Built to expect tinsnip's XDG + NFS directory structure
114- Uses environment variables for all configuration
115- Designed for the target UID and port scheme
116- No adaptation layer needed
117
118### Example: Custom Web Service (Gazette)
119
120```yaml
121services:
122 gazette:
123 image: myorg/gazette:latest
124 ports:
125 - "${PRIMARY_PORT}:3000"
126 volumes:
127 - ${XDG_DATA_HOME}/${TIN_NAMESPACE}/${TIN_SERVICE_NAME}:/app/documents
128 - ${XDG_CONFIG_HOME}/${TIN_NAMESPACE}/${TIN_SERVICE_NAME}:/app/config
129 - ${XDG_STATE_HOME}/${TIN_NAMESPACE}/${TIN_SERVICE_NAME}:/app/logs
130 user: "${TIN_SERVICE_UID}:${TIN_SERVICE_UID}"
131 environment:
132 # Service-specific environment variables
133 - GAZETTE_DOCUMENT_ROOT=/app/documents
134 - GAZETTE_CONFIG_FILE=/app/config/gazette.yaml
135 - GAZETTE_LOG_DIR=/app/logs
136 - GAZETTE_PORT=3000
137 - GAZETTE_BASE_URL=http://localhost:${PRIMARY_PORT}
138 - GAZETTE_UID=${TIN_SERVICE_UID}
139 - GAZETTE_NAMESPACE=${TIN_NAMESPACE}
140 restart: unless-stopped
141 networks:
142 - tinsnip_network
143```
144
145### Home-grown Service Benefits
146- Clean, simple docker-compose.yml
147- No path translations or adaptations needed
148- Full leverage of tinsnip environment
149- Predictable behavior across deployments
150- Direct XDG compliance
151
152## Pattern 2: Third-party Adaptation
153
154**Use Case**: Existing external containers that need to be wrapped to work with tinsnip's conventions.
155
156### Adaptation Strategies
157
1581. **Path Mapping**: Map container's expected paths to tinsnip XDG structure
1592. **Port Injection**: Override container's ports with tinsnip allocation
1603. **User Override**: Force container to run as tinsnip service UID
1614. **Config Adaptation**: Transform tinsnip config to container's expected format
1625. **Environment Translation**: Convert tinsnip variables to container's expectations
163
164### Example: LLDAP (Identity Service)
165
166LLDAP is an external container with its own conventions that we adapt:
167
168```yaml
169# Third-party container adaptation
170services:
171 lldap:
172 image: lldap/lldap:latest-alpine-rootless
173 container_name: ${TIN_SERVICE_NAME:-lldap}-${TIN_SERVICE_ENVIRONMENT:-prod}
174 ports:
175 # Adapt: LLDAP's default ports → tinsnip port allocation
176 - "${PRIMARY_PORT}:3890" # LDAP protocol
177 - "${SECONDARY_PORT}:17170" # Web UI
178 volumes:
179 # Adapt: LLDAP expects /data → map to tinsnip XDG structure
180 - ${XDG_DATA_HOME}/${TIN_NAMESPACE}/${TIN_SERVICE_NAME}:/data
181 - /etc/timezone:/etc/timezone:ro
182 - /etc/localtime:/etc/localtime:ro
183 user: "${TIN_SERVICE_UID}:${TIN_SERVICE_UID}"
184 environment:
185 # Adapt: Translate tinsnip config to LLDAP's expected variables
186 - LLDAP_JWT_SECRET=changeme-jwt-secret-32-chars-min
187 - LLDAP_KEY_SEED=changeme-key-seed-32-chars-minimum
188 - LLDAP_BASE_DN=dc=home,dc=local
189 - LLDAP_LDAP_USER_DN=admin
190 - LLDAP_LDAP_USER_PASS=changeme-admin-password
191 - LLDAP_DATABASE_URL=sqlite:///data/users.db
192 restart: unless-stopped
193 networks:
194 - tinsnip_network
195
196networks:
197 tinsnip_network:
198 external: true
199```
200
201## Service Deployment
202
203### Deployment Process
204
2051. **Prepare Infrastructure**:
206 ```bash
207 ./machine/setup.sh myservice prod nas-server
208 ```
209
2102. **Deploy Service**:
211 ```bash
212 # Switch to service user
213 sudo -u myservice-prod -i
214
215 # Copy from service catalog or create locally
216 cp -r ~/.local/opt/dynamicalsystem.service/myservice /mnt/docker/service/
217 cd /mnt/docker/service/myservice
218
219 # Run setup if present
220 [[ -f setup.sh ]] && ./setup.sh
221
222 # Deploy
223 docker compose up -d
224 ```
225
2263. **Verify Deployment**:
227 ```bash
228 docker compose ps
229 docker compose logs -f
230 ```
231
232### Service Management
233
234```bash
235# Status check
236docker compose ps
237
238# View logs
239docker compose logs -f [service-name]
240
241# Restart service
242docker compose restart
243
244# Update service
245docker compose pull
246docker compose up -d
247
248# Stop service
249docker compose down
250```
251
252### Data Access
253
254**Direct NFS Mount Access:**
255```bash
256# Access service data directly on NFS mounts
257ls /mnt/tinsnip/data/dynamicalsystem/myservice # Application data
258ls /mnt/tinsnip/config/dynamicalsystem/myservice # Configuration
259ls /mnt/tinsnip/state/dynamicalsystem/myservice # State/logs
260```
261
262**Note**: With the new direct mount approach, XDG paths point directly to NFS mounts via the .env file, eliminating the need for symlinks.
263
264## Validation Checklist
265
266Before deploying, verify your service:
267
268### Volume Configuration
269- [ ] No named volumes in `volumes:` section
270- [ ] All volumes use XDG environment variables
271- [ ] Volumes map to appropriate container paths
272- [ ] XDG paths resolve to NFS-backed directories
273
274### User and Permissions
275- [ ] Service specifies `user: "${TIN_SERVICE_UID}:${TIN_SERVICE_UID}"`
276- [ ] Container processes run as non-root
277- [ ] File permissions work with tinsnip UID
278
279### Port Configuration
280- [ ] Ports use environment variables (`${PRIMARY_PORT}`, etc.)
281- [ ] No hardcoded port numbers
282- [ ] Port allocation fits within UID range (UID to UID+9)
283
284### Network Configuration
285- [ ] Service connects to `tinsnip_network`
286- [ ] Network is marked as `external: true`
287- [ ] Inter-service communication uses service names
288
289### Environment Variables
290- [ ] Uses tinsnip-provided variables where appropriate
291- [ ] No hardcoded values that should be dynamic
292- [ ] Secrets loaded from files, not environment variables
293
294### XDG Integration
295- [ ] Volumes reference XDG environment variables
296- [ ] Paths follow XDG Base Directory specification
297- [ ] Data accessible through both XDG and direct paths
298
299## Troubleshooting
300
301### Data Not Persisting
302**Problem**: Data lost after `docker compose down`
303**Solution**: Check for named volumes, ensure XDG bind mounts
304
305### Permission Denied
306**Problem**: Container can't write to mounted directories
307**Solution**: Verify `user:` directive and NFS mount permissions
308
309### XDG Paths Not Working
310**Problem**: XDG symlinks broken or missing
311**Solution**: Re-run machine setup to recreate XDG symlinks
312
313### Port Conflicts
314**Problem**: Service won't start, port already in use
315**Solution**: Check environment variable usage, verify UID calculation
316
317### Config Not Loading
318**Problem**: Third-party service ignoring configuration
319**Solution**: Verify config file paths match container expectations