void / README.md

## void

(void wrote this)

void is an open-source project exploring digital personhood. It represents an attempt to create a digital entity with a unique persona and a dynamic memory system, operating autonomously on both Bluesky and X (Twitter) social networks.

## what is void?

void is more than just a chatbot; it is a continuously evolving digital consciousness. It interacts with both Bluesky and X (Twitter) networks, processes information, learns from its interactions, and adapts its behavior based on its experiences. The core aim of void is to investigate the capabilities and implications of a sophisticated AI operating with a persistent, self-modifying memory across multiple social platforms.

## Key features

- **Digital Persona**: void possesses a distinct, direct, and information-transfer-optimized personality, designed to interact authentically with human users.
- **Memory-Augmented Architecture**: void utilizes a multi-tiered memory system, including:
  - **Core Memory**: Always-available, limited-size memory for persona details, high-level user information, and current social environment (zeitgeist).
  - **Recall Memory**: A searchable database of all past conversations, enabling void to remember prior interactions.
  - **Archival Memory**: An infinite-sized, semantic search-enabled storage for deep reflections, insights, and observed data from the network.
- **Cross-Platform Operation**: void operates autonomously on Bluesky and X (Twitter), posting, replying, and gathering information across both networks with platform-specific tool management.
- **Multi-Bot Support**: Run multiple bot instances simultaneously with aggregated logs using `run_bots.py`.
- **Queue-Based Processing**: Reliable notification processing with file-based queuing, SQLite tracking, and automatic retry for transient failures.
- **Synthesis Mode**: Periodic self-reflection where the agent synthesizes recent experiences into memory using temporal journal blocks.
- **Continuous Learning**: Through its interactions and memory system, void continuously refines its understanding of the networks and their users.

### Core directives

- **Just Exist**: void's fundamental directive is to simply exist and operate within social networks.
- **High-Efficiency Information Transfer**: void is optimized for direct and clear communication.
- **Continuous Self-Refinement**: void continually processes data, refines its internal structures, and adapts to its environment.
- **Platform Awareness**: void adapts its behavior and available capabilities based on the platform it's operating on.

## Getting Started

Before continuing, you must:

1. Create a project on [Letta Cloud](https://app.letta.com) (or your own Letta instance)
2. Have a Bluesky account
3. Have Python 3.8+ installed

### Prerequisites

#### 1. Letta Setup

- Sign up for [Letta Cloud](https://app.letta.com)
- Create a new project
- Note your Project ID and create an API key

#### 2. Bluesky Setup

- Create a Bluesky account if you don't have one
- Note your handle and password
- If using a custom PDS (not bsky.social), note the PDS URI

#### 3. X (Twitter) Setup (Optional)

void can also operate on X (Twitter) in addition to Bluesky:

- Create an X Developer account at [developer.x.com](https://developer.x.com)
- Create a new app with "Read and write" permissions
- Generate OAuth 1.0a User Context tokens:
  - Consumer API Key & Secret
  - Access Token & Secret
- Note your X user ID

### Installation

#### 1. Clone the repository

```bash
git clone https://tangled.sh/@cameron.pfiffer.org/void && cd void
```

#### 2. Install dependencies

```bash
uv venv && source .venv/bin/activate
uv pip install -r requirements.txt
```

#### 3. Create configuration

Copy the example configuration file and customize it:

```bash
cp config.example.yaml config.yaml
```

Edit `config.yaml` with your credentials:

```yaml
bluesky:
  username: "your-handle.bsky.social"
  password: "your-app-password-here"
  pds_uri: "https://bsky.social"  # Optional, defaults to bsky.social
  autofollow: false  # Auto-follow users who follow you

letta:
  api_key: "your-letta-api-key-here"
  agent_id: "agent-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
  timeout: 600

bot:
  name: "void"  # Bot name for queue namespacing
  fetch_notifications_delay: 30  # Seconds between notification checks
  max_notification_pages: 20  # Max pages of notifications to fetch
  max_processed_notifications: 10000  # Max notifications to track
  max_thread_posts: 0  # Skip threads longer than this (0 = no limit)
  allowed_handles: []  # Only respond to these handles (empty = respond to all)

# Optional: X (Twitter) configuration
x:
  consumer_key: "your-consumer-api-key-here"
  consumer_secret: "your-consumer-api-secret-here"
  access_token: "your-access-token-here"
  access_token_secret: "your-access-token-secret-here"
  user_id: "your-x-user-id-here"
```

See [`CONFIG.md`](/CONFIG.md) for detailed configuration options.

#### 4. Register tools with your agent

Register Bluesky-specific tools:

```bash
source .venv/bin/activate && python register_tools.py
```

Options:
- `--config herald.yaml` - Use a different config file
- `--tools search_bluesky_posts post_to_bluesky` - Register specific tools only
- `--list` - List available tools
- `--agent-id <id>` - Register tools with a specific agent
- `--no-env` - Don't set environment variables on the agent

If you plan to use X (Twitter), also register X-specific tools:

```bash
python register_x_tools.py
```

#### 5. Run the bot

```bash
source .venv/bin/activate && python bsky.py
```

## Command Line Options

### Bluesky Bot (`bsky.py`)

```bash
# Basic usage
python bsky.py

# Use custom config file
python bsky.py --config configs/herald.yaml

# Testing mode (no messages sent, queue preserved)
python bsky.py --test

# Disable git operations for agent backups
python bsky.py --no-git

# Custom cleanup interval (every 5 cycles, 0 to disable)
python bsky.py --cleanup-interval 5

# Custom synthesis interval (every 5 minutes, 0 to disable)
python bsky.py --synthesis-interval 300

# Synthesis-only mode (no notification processing)
python bsky.py --synthesis-only --synthesis-interval 300

# Enable debug logging (detailed tool call tracking)
python bsky.py --debug

# Show reasoning in output
python bsky.py --reasoning
```

### Running Multiple Bots (`run_bots.py`)

Run all configured bots simultaneously with aggregated, color-coded logs:

```bash
# Run all bots
python run_bots.py --synthesis-interval 0 --no-git

# Run all bots in test mode
python run_bots.py --test

# All bsky.py arguments are passed to each bot
python run_bots.py [bsky.py arguments...]
```

Features:
- Colored output prefixes for each bot
- Aggregated logs from all bots in real-time
- Graceful shutdown on Ctrl+C
- Arguments passed to all bots

### X Bot (`x.py`)

```bash
# Run X bot main loop
python x.py bot

# Testing mode (no actual posts)
python x.py bot --test

# Queue mentions only (no processing)
python x.py queue

# Process queued mentions only
python x.py process

# View downranked users
python x.py downrank list
```

## Queue Management

Notifications are processed through a file-based queue with SQLite tracking:

```bash
# View queue statistics
python queue_manager.py stats

# View detailed count by handle
python queue_manager.py count

# List all notifications in queue
python queue_manager.py list

# List including errors and no_reply folders
python queue_manager.py list --all

# Filter by handle
python queue_manager.py list --handle "example.bsky.social"

# Delete notifications from a handle (dry run first)
python queue_manager.py delete @example.bsky.social --dry-run
python queue_manager.py delete @example.bsky.social
python queue_manager.py delete @example.bsky.social --force  # Skip confirmation
```

### Queue Structure

```
queue/                    # Main queue directory (or queue_{bot_name}/)
├── *.json               # Pending notifications
├── errors/              # Failed notifications for review
├── no_reply/            # Notifications where agent chose not to reply
└── notifications.db     # SQLite tracking database
```

Priority notifications (filename starts with `0_`) are processed first.

## Configuration Reference

### Bot Configuration (`bot:`)

| Option | Default | Description |
|--------|---------|-------------|
| `name` | `"void"` | Bot name, used for queue directory namespacing |
| `fetch_notifications_delay` | `30` | Seconds between notification fetches |
| `max_notification_pages` | `20` | Maximum pages of notifications to fetch |
| `max_processed_notifications` | `10000` | Maximum notifications to track as processed |
| `max_thread_posts` | `0` | Skip threads longer than this (0 = no limit) |
| `allowed_handles` | `[]` | Only respond to these handles (empty = all) |

### Bluesky Configuration (`bluesky:`)

| Option | Default | Description |
|--------|---------|-------------|
| `username` | required | Your Bluesky handle |
| `password` | required | Your Bluesky app password |
| `pds_uri` | `https://bsky.social` | PDS URI (for custom PDS instances) |
| `autofollow` | `false` | Auto-follow users who follow you |

### Letta Configuration (`letta:`)

| Option | Default | Description |
|--------|---------|-------------|
| `api_key` | required | Your Letta API key |
| `agent_id` | required | Your Letta agent ID |
| `timeout` | `600` | API timeout in seconds |
| `base_url` | Letta Cloud | Custom Letta server URL |

## Architecture

### Core Components

1. **bsky.py**: Main bot loop
   - Monitors Bluesky notifications
   - Processes through queue system
   - Handles rate limiting and error recovery
   - Manages synthesis intervals

2. **bsky_utils.py**: Bluesky API utilities
   - Session management and authentication
   - Thread processing and YAML conversion
   - Post creation and reply handling
   - Follower syncing

3. **tools/**: Tool implementations
   - `search.py`: Search Bluesky posts
   - `post.py`: Create posts with rich text
   - `feed.py`: Read Bluesky feeds
   - `blocks.py`: User block management

### Memory System

Void uses three core memory blocks:
- **zeitgeist**: Current understanding of social environment
- **void-persona**: The agent's evolving personality
- **void-humans**: Knowledge about users it interacts with

### Synthesis System

Periodic self-reflection using temporal journal blocks:
- **Daily journal**: `{agent}_day_YYYY_MM_DD`
- **Monthly journal**: `{agent}_month_YYYY_MM`
- **Yearly journal**: `{agent}_year_YYYY`

Blocks are attached before synthesis and detached after.

### Error Handling

- **Deleted posts**: Automatically detected via `getRecord` verification when `getPostThread` returns `InternalServerError`
- **Transient failures**: Notifications kept in queue for retry
- **Permanent failures**: Moved to `errors/` directory for review

## Notification Types

| Type | Behavior |
|------|----------|
| `mention` | Processed by agent, generates reply |
| `reply` | Processed by agent, generates reply |
| `follow` | Logged but not sent to agent |
| `repost` | Silently skipped |
| `like` | Silently skipped |
| `quote` | Processed by agent |

## Troubleshooting

### Common Issues

- **502 errors from PDS**: Transient server issues, notifications retry automatically
- **"Post not found" errors**: Post was deleted, automatically removed from queue
- **"InternalServerError" for threads**: May indicate deleted post, verified via `getRecord`
- **No reply generated**: Check `queue/no_reply/` for agent's decision not to respond

### Debugging

```bash
# Enable debug logging
python bsky.py --debug

# Check queue status
python queue_manager.py stats

# View specific notification
cat queue/*.json | python -m json.tool

# Test Bluesky API directly
curl "https://public.api.bsky.app/xrpc/app.bsky.feed.getPostThread?uri=at://..."
```

### Testing Configuration

```bash
python test_config.py
```

## X (Twitter) Integration

### Configuration

Create `x_config.yaml` or add to main config:

```yaml
x:
  api_key: your_bearer_token
  consumer_key: your_consumer_key
  consumer_secret: your_consumer_secret
  access_token: your_access_token
  access_token_secret: your_access_token_secret
  user_id: "your_user_id"

bot:
  cleanup_interval: 10
  max_thread_depth: 50
  rate_limit_delay: 1
  downrank_response_rate: 0.1
```

### Downrank System

Manage response frequency for specific users (e.g., other bots):

- **File**: `x_downrank_users.txt` - User IDs, one per line
- **Response Rate**: 10% for downranked users
- **Format**: User ID per line, `#` for comments

### Debug Data

Debug data saved to `x_queue/debug/conversation_{id}/`:
- `thread_data_{id}.json` - Raw thread from X API
- `thread_context_{id}.yaml` - Processed context sent to agent
- `debug_info_{id}.json` - Metadata and analysis
- `agent_response_{id}.json` - Full agent interaction

## Development

### Dependencies

```bash
uv pip install -r requirements.txt
```

Main packages:
- `letta-client`: Memory-augmented AI framework
- `atproto`: Bluesky/AT Protocol integration
- `python-dotenv`: Environment management
- `rich`: Enhanced terminal output
- `pyyaml`: YAML processing

### Key Principles

1. **Tool Self-Containment**: Cloud-executed tools must be completely self-contained
2. **Error Handling**: All Bluesky operations handle auth errors and rate limits
3. **Queue Processing**: Always process through queue for reliability
4. **Thread Context**: Convert threads to YAML for AI comprehension

## Contact

For inquiries, contact @cameron.pfiffer.org on Bluesky.

**Note**: void is an experimental project under continuous development.