Add facet extraction and embed handling. Enables void to properly read links, link previews, quotes, and see images. Also includes consecutive chain processing to enable passing the last post in multi-part replies.

Port features from umbra to void:

extract_links_from_facets: Extract link URLs with text from facets

extract_images_from_embed: Extract images from all embed types

extract_images_from_thread: Collect images from thread chronologically

extract_external_link_from_embed: Extract link card data

extract_quote_post_from_embed: Extract quoted post with status handling

extract_embed_data: Main entry point for embed extraction

Enhanced thread processing:

flatten_thread_structure now properly accesses AT Protocol properties

Extracts links from facets, embed data, and parent_uri

Thread visualization:

compute_tree_prefixes: Generate tree-style prefixes

build_tree_view: Build text visualization of thread structure

thread_to_yaml_string: Add include_tree_view parameter

Consecutive chain processing:

find_last_consecutive_post_in_chain: Traverse down to find last post

find_consecutive_parent_posts_by_author: Traverse up parent chain

- Add comprehensive configuration reference tables
- Document all CLI options for bsky.py, run_bots.py, x.py
- Add queue management section with commands
- Document notification types and their behaviors
- Add architecture overview (memory, synthesis, error handling)
- Include troubleshooting and debugging sections
- Document X integration and downrank system
- Skip sending follow notifications to agent (commented out)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Print entire error chunk plus model_dump()/vars() to capture
all fields regardless of what Letta returns.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Extract useful attributes from error_message chunks instead of
logging the unhelpful default string representation. Dumps all
non-None attributes if standard fields are empty.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

When bot.allowed_handles is configured, only mentions from those
handles will be processed. Empty list means respond to all (default).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Bluesky's getPostThread API sometimes returns InternalServerError
instead of NotFound for deleted posts. Now we verify with getRecord
to confirm the post is actually deleted before removing from queue.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

- Move all config files (*.yaml) to configs/ directory
- Update .gitignore to ignore configs/ instead of individual files
- Remove archivist.yaml and grunk.yaml from git history (contained credentials)
- Update all code references to use configs/ directory:
- config_loader.py: configs/config.yaml default path
- bsky.py: configs/config.yaml default path
- register_tools.py: configs/config.yaml default path
- run_bots.py: all bot configs now in configs/
- x.py: configs/x_config.yaml default path

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Removed import of blocks module from tools/__init__.py which was
causing "No module named 'tools.blocks'" error.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Removes the user block management system entirely:
- Deleted tools/blocks.py containing all user block tools
- Removed attach/detach user blocks from bsky.py
- Removed attach/detach X user blocks from x.py
- Removed periodic_user_block_cleanup functions
- Updated register_x_tools.py to remove X user block tools
- Deleted test_x_blocks.py test file
- Updated CLAUDE.md documentation

User-specific memory blocks are no longer used. The agents will
rely solely on their core memory blocks (zeitgeist, persona, etc.)
and archival memory.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Includes the main void bot (config.yaml) in run_bots.py, bringing
total to 6 bots. Void displays with red color prefix.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Add documentation for run_bots.py script showing usage examples
and features.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Creates run_bots.py to launch all 5 bot configs simultaneously with
docker-compose-style aggregated output. Features colored prefixes for
each bot, graceful shutdown handling, and argument passthrough.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

- Add bot name to log format (shows 'void', 'herald', 'archivist', etc.)
- Update gitignore to exclude all queue_* directories
- Update gitignore to exclude all bot-specific config files
- Remove queue databases from git tracking
- Update void agent state backup

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Adds a configurable max_thread_posts setting that allows skipping
notifications when the thread is too long. This helps avoid:
- Context window exhaustion
- Processing extremely long conversations
- High token costs for very deep threads

Configuration:
- bot.max_thread_posts: Maximum number of posts allowed in a thread
- Set to 0 to disable the gate (default)
- When exceeded, notification is removed from queue with info log

Implementation:
- count_thread_posts() helper function in bsky_utils.py
- Thread length check happens immediately after fetching the thread
- Logged as: "Thread too long (N posts > M max), skipping this mention"

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Streamlined the prompt agents receive when mentioned:
- Removed prescriptive guidance about when to use threads
- Removed redundant explanation of YAML structure
- Made replying feel optional ("If you choose to reply...")
- Kept essential info: who mentioned them, the post, thread context, and tool usage

This gives agents more autonomy to decide whether and how to respond,
rather than implying they must reply to every mention.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Reduced the synthesis prompt from a detailed numbered list of instructions
to a simpler, clearer message that:
- Explains what synthesis is for (periodic reflection and memory update)
- Lists available temporal journal blocks
- Lets the agent decide how to use them

This removes potentially misleading prescriptive guidance and gives agents
more autonomy in their synthesis process.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

The create_memory_record call was using undefined 'client' variable
instead of the function parameter 'atproto_client', causing
'name 'client' is not defined' error.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Implemented stream.thought.memory record type to store archival_memory_insert
tool calls on the AT Protocol. This preserves important memories and context
in a queryable format.

Changes:
- Added create_memory_record() function in bsky_utils.py
- Records contain content (required) and tags (optional) fields
- Integrated into both notification processing and synthesis workflows
- Automatically detects archival_memory_insert tool calls
- Parses tags from string or array format
- Includes proper logging with emoji indicators (📝)

Tested with sample data - successfully creates records on the PDS.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Added hashtag parsing to both reply_to_post() in bsky_utils.py and
create_new_bluesky_post() in tools/post.py. Hashtags are now properly
parsed and converted to app.bsky.richtext.facet#tag facets with correct
byte positions.

- Parse hashtags using regex pattern that matches #word patterns
- Create facets with proper byte slice positioning
- Include hashtag tracking in debug logging
- Tested regex with various hashtag patterns

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

This commit adds an autofollow system that automatically follows users who follow the bot:

- Add `bluesky.autofollow` config option (default: false)
- Implement `sync_followers()` function in bsky_utils.py
- Uses Bluesky graph API to compare followers and following lists
- Creates app.bsky.graph.follow records for new followers
- Rate limited to 2 seconds between follows to avoid server spam
- Integrate autofollow check into main bot loop cleanup cycle
- Update config.example.yaml and archivist.yaml with autofollow option
- Document autofollow system in CLAUDE.md

The feature runs periodically (every N cycles, default 10) and logs
all follow actions for tracking and debugging.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

The first pass was incorrectly checking for a 'name' attribute to identify
tool return messages, but Letta's tool return messages don't have this attribute
(or it's None). Instead, tool return messages are identified by having:
- tool_return attribute (the actual return value)
- tool_call_id (to match with the call)
- status (success/error)

This fix:
- Changes first pass to detect tool returns using has_tool_return instead of has_name
- Stores ALL tool return statuses by tool_call_id (not just specific tools)
- Simplifies ignore_notification handling to check the tool_return string directly
- Moves deprecated tool check to tool_call messages where it belongs

This resolves the "unknown status" warnings that occurred because tool returns
weren't being properly collected.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Adds --debug command-line flag to bsky.py to enable DEBUG level logging.
This makes the detailed tool call tracking logs visible.

Usage: ac && python bsky.py --debug

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Adds comprehensive debug logging to track tool call/return message processing:
- Logs all message attributes (type, tool_call_id, status, name)
- Shows which tool results are collected in first pass
- Displays available tool_call_results when processing add_post_to_bluesky_reply_thread calls
- Helps diagnose "unknown status" warnings when tool returns are missing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

These tools are managed by bsky.py automatically and should not be exposed to the agent to prevent it from managing its own memory blocks.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

- Automatically set BSKY_USERNAME, BSKY_PASSWORD, and PDS_URI as tool execution environment variables on the agent
- Add --config flag to register_tools.py for multi-agent support
- Add --no-env flag to skip environment variable configuration
- Simplify config handling by removing global variables
- Update documentation with new usage examples

This allows tools to access Bluesky credentials when running on Letta's cloud servers, and enables registering tools for different agents with their respective configs.

Usage:
python register_tools.py --config herald.yaml

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

- Add --config command-line argument to specify custom config file
- Implement bot.name-based queue directory namespacing
- bot.name: void → queue/ (backward compatible)
- bot.name: herald → queue_herald/
- Other names → queue_{bot_name}/
- Each bot now has isolated queues, databases, and error directories
- Move config/client initialization into main() after argument parsing
- Update config_loader.py to build queue paths from bot.name

This allows multiple bots to run independently without queue conflicts.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Ensures flag_archival_memory_for_deletion requires a non-empty reason
field before processing deletions. Logs warning if reason is missing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Adds important note that all archival memories with identical text
will be deleted when using flag_archival_memory_for_deletion.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Enhances the flag_archival_memory_for_deletion tool with safety guards:
- Added 'reason' field (required, first argument) to force reasoning
- Added 'confirm' boolean field (required, last argument) for explicit confirmation
- Tool only processes deletions when confirm=true
- Deletion handler extracts and logs the reason with each deletion

This pattern ensures the agent must:
1. Explain why it's deleting (reason)
2. Specify what it's deleting (memory_text)
3. Explicitly confirm the deletion (confirm)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Implements a new tool that allows the agent to flag archival memories for
deletion based on exact text matching. The tool works as a no-op that signals
the bot loop to perform deletions at the end of the turn.

Changes:
- Created tools/flag_memory_deletion.py with the tool definition
- Updated register_tools.py to include the new tool
- Modified bsky.py to handle flagged memory deletion:
- Tracks flagged memories during message processing
- Searches for exact text matches using passages.list()
- Deletes all matching passages using passages.delete()
- Executes after message processing but before reply handling
- Automatically skipped if halt_activity is called (due to immediate exit)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

- Agent state updated with latest tool and message configurations
- Removed config.yaml.bkp (no longer needed)
- Updated X bot queue tracking files

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

- Remove unused project_id (not needed for agent retrieval)
- Add base_url configuration for self-hosted servers
- Update all Letta client instantiations to use base_url when configured
- Default behavior unchanged (uses cloud API when base_url not set)
- Tested with localhost:8283 self-hosted server

Added comprehensive X bot configuration section with:
- x_config.yaml structure and required fields
- API credentials setup for X integration
- Bot behavior configuration options
- Logging configuration for debug data

This complements the existing Bluesky bot documentation.

## Rate Limit Improvements:
- Add specific endpoint logging to identify rate limit sources
- Implement user info caching to reduce /users/me API calls from ~720/day to 1/day
- Add get_username() and get_user_info() methods with 24-hour caching
- Update all /users/me calls to use cached methods

## Enhanced Logging:
- Add endpoint-specific rate limit error messages
- Add configurable logging level from x_config.yaml
- Add debug logging for API operations
- Show exact API endpoints when rate limits occur

## Benefits:
- Prevents rate limiting on bot startup and routine operations
- Better debugging visibility for API issues
- Improved bot reliability and uptime

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

- Switch from config_loader.get_letta_config() to x.get_x_letta_config()
- Remove x_user_note_* tools from registration (attach/detach remain)
- Fixes X bot tool registration to use correct configuration file

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

- Split X bot configuration from main config.yaml into x_config.yaml
- Updated all config loading functions to use new configuration structure
- Added x_config.yaml to .gitignore for security
- Updated documentation with new configuration format and structure
- Improved error handling and validation for configuration loading

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

- Fixed 'NoneType' object has no attribute 'get' errors in notification_db.py
- Added proper null checking for record fields and reply information
- Fixed logic in save_notification_to_queue to prevent queueing when DB add fails
- This resolves notification duplication where failed DB adds led to reprocessing

The bot was reprocessing the same notifications because:
1. Notification gets fetched and queued
2. Database add fails due to None field access
3. Notification gets processed but never marked as processed in DB
4. Next fetch cycle finds it again and re-queues it

Major improvements:
- Added SQLite database for reliable notification tracking
- Removed problematic is_read filter that was blocking all notifications
- Implemented timestamp-based filtering for continuity between runs
- Added comprehensive debug logging to understand notification flow
- Created recovery tools for missed notifications
- Added queue health monitoring and alerts
- Auto-migration from old JSON format to database
- Periodic cleanup of old notification records

This should significantly improve notification reliability and provide better visibility into what's being processed.

This commit addresses multiple issues preventing the bot from responding
to non-priority users and improves visibility into outbound message status.

## Key Fixes

**Bot Detection Issue:**
- Temporarily disable bot detection causing 90% skip rate for normal users
- Bot was incorrectly flagging most users as bots, only responding to cameron.pfiffer.org
- Added clear TODO to re-enable after debugging the root cause

**Configuration Priority:**
- Fix config_loader to prioritize config.yaml over environment variables
- Update default_login() to use config-based authentication instead of env vars
- Ensures bot connects to configured PDS (comind.network) consistently

**PDS Server Consistency:**
- Update all bsky_utils functions to use configured PDS URI from config.yaml
- Fixed create_synthesis_ack(), acknowledge_post(), create_tool_call_record(), create_reasoning_record()
- Eliminates "Token could not be verified" errors from posting to wrong server

## Enhanced Logging

**Correlation ID Tracking:**
- Add end-to-end correlation IDs for tracking messages through pipeline
- Generate unique 8-char IDs in process_mention() and pass through all functions
- Enables debugging of dropped messages with structured logging

**Improved Success Visibility:**
- Enhanced reply logging shows response times and post URIs in console output
- Added structured logging with extra fields for detailed analysis
- Better confirmation that posts are being sent successfully

**Facet Parsing Details:**
- Log mention resolution (handles -> DIDs) and URL detection
- Track parsing failures and network issues during rich text processing

## Files Modified

- `bsky.py`: Add correlation IDs, disable bot detection, enhance process_mention logging
- `bsky_utils.py`: Update all functions to use config PDS, add comprehensive logging
- `config_loader.py`: Prioritize config.yaml over environment variables
- `tools/post.py`: Cleaned up (removed enhanced logging for cloud compatibility)

## Testing Notes

Bot should now:
- Respond to ALL users (not just priority ones)
- Use consistent PDS server (comind.network) for all operations
- Provide clear logging confirmation when posts are sent
- Resolve acknowledgment and reasoning record errors

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

- Remove X tools from general register_tools.py
- Add post_to_x tool to register_x_tools.py
- Tool creates standalone posts on X (not replies)
- Properly organized with other X-specific tools

- Add post_to_x and add_post_to_x_thread to register_tools.py
- Both tools now available for void to use
- post_to_x: Create standalone X posts
- add_post_to_x_thread: Add posts to X reply threads

- Add post_tweet() method to XClient for standalone posts
- Create self-contained post_to_x() tool for cloud execution
- Validate 280 character limit
- Successfully tested both implementations

- Change from stream.thought.ack embedded format back to dedicated collections
- Use stream.thought.tool.call for tool call records (dots for valid NSID)
- Use stream.thought.reasoning for reasoning records
- Add better error logging for debugging record creation issues

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

- Added create_tool_call_record() to publish stream.thought.tool_call records
- Added create_reasoning_record() to publish stream.thought.reasoning records
- Updated process_mention() to create records for both reasoning and tool calls
- Updated send_synthesis_message() to create records during synthesis
- Records include timestamp and preserve raw tool arguments as strings

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

- Attach day/month/year journal blocks during synthesis
- Create blocks with labels like void_day_2025_08_09
- Enhanced synthesis prompt explains journal usage
- Blocks detached after synthesis completes
- Allows void to maintain temporal memory across sessions

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

- Fixed user_note_append, user_note_replace, user_note_set, user_note_view functions
- Changed from get_letta_client() to inline Letta client creation for cloud compatibility
- Also updated attach/detach functions to handle both local and cloud execution
- Removed logger.debug call that wouldn't work in cloud environment
- All functions now create Letta client inline using environment variables

This fixes the critical failure void reported where user_note_* tools were non-functional
due to NameError when get_letta_client couldn't import config_loader in cloud.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

Connect to Bluesky even in synthesis-only mode (unless --test is used) to enable posting synthesis acks when the agent calls annotate_ack tool.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>