Manage Atom feeds in a persistent git repository
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

Add comprehensive Zulip bot integration with dynamic features

- Complete Zulip bot implementation with debug mode and stream posting
- Dynamic bot name detection instead of hardcoded "@thicket"
- HTML to Markdown conversion with compact summary formatting
- Zulip user association management (add/remove/list commands)
- Rate limiting (5 messages per batch with 5s pause)
- Catchup mode (posts last 5 entries on first run)
- Smart duplicate author detection to avoid redundancy
- Comprehensive error handling and fallback mechanisms
- Bot configuration via chat commands (stream, topic, interval)
- Clean, non-redundant message formatting for feed updates

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

+2182
+400
SPEC.md
··· 1 + # Thicket Git Store Specification 2 + 3 + This document comprehensively defines the JSON format and structure of the Thicket Git repository, enabling third-party clients to read and write to the store while leveraging Thicket's existing Python classes for data validation and business logic. 4 + 5 + ## Overview 6 + 7 + The Thicket Git store is a structured repository that persists Atom/RSS feed entries in JSON format. The store is designed to be both human-readable and machine-parseable, with a clear directory structure and standardized JSON schemas. 8 + 9 + ## Repository Structure 10 + 11 + ``` 12 + <git_store>/ 13 + ├── index.json # Main index of all users and metadata 14 + ├── duplicates.json # Maps duplicate entry IDs to canonical IDs 15 + ├── index.opml # OPML export of all feeds (generated) 16 + ├── <username1>/ # User directory (sanitized username) 17 + │ ├── <entry_id1>.json # Individual feed entry 18 + │ ├── <entry_id2>.json # Individual feed entry 19 + │ └── ... 20 + ├── <username2>/ 21 + │ ├── <entry_id3>.json 22 + │ └── ... 23 + └── ... 24 + ``` 25 + 26 + ## JSON Schemas 27 + 28 + ### 1. Index File (`index.json`) 29 + 30 + The main index tracks all users, their metadata, and repository statistics. 31 + 32 + **Schema:** 33 + ```json 34 + { 35 + "users": { 36 + "<username>": { 37 + "username": "string", 38 + "display_name": "string | null", 39 + "email": "string | null", 40 + "homepage": "string (URL) | null", 41 + "icon": "string (URL) | null", 42 + "feeds": ["string (URL)", ...], 43 + "zulip_associations": [ 44 + { 45 + "server": "string", 46 + "user_id": "string" 47 + }, 48 + ... 49 + ], 50 + "directory": "string", 51 + "created": "string (ISO 8601 datetime)", 52 + "last_updated": "string (ISO 8601 datetime)", 53 + "entry_count": "integer" 54 + } 55 + }, 56 + "created": "string (ISO 8601 datetime)", 57 + "last_updated": "string (ISO 8601 datetime)", 58 + "total_entries": "integer" 59 + } 60 + ``` 61 + 62 + **Example:** 63 + ```json 64 + { 65 + "users": { 66 + "johndoe": { 67 + "username": "johndoe", 68 + "display_name": "John Doe", 69 + "email": "john@example.com", 70 + "homepage": "https://johndoe.blog", 71 + "icon": "https://johndoe.blog/avatar.png", 72 + "feeds": [ 73 + "https://johndoe.blog/feed.xml", 74 + "https://johndoe.blog/categories/tech/feed.xml" 75 + ], 76 + "zulip_associations": [ 77 + { 78 + "server": "myorg.zulipchat.com", 79 + "user_id": "john.doe" 80 + }, 81 + { 82 + "server": "community.zulipchat.com", 83 + "user_id": "johndoe@example.com" 84 + } 85 + ], 86 + "directory": "johndoe", 87 + "created": "2024-01-15T10:30:00", 88 + "last_updated": "2024-01-20T14:22:00", 89 + "entry_count": 42 90 + } 91 + }, 92 + "created": "2024-01-15T10:30:00", 93 + "last_updated": "2024-01-20T14:22:00", 94 + "total_entries": 42 95 + } 96 + ``` 97 + 98 + ### 2. Duplicates File (`duplicates.json`) 99 + 100 + Maps duplicate entry IDs to their canonical representations to handle feed entries that appear with different IDs but identical content. 101 + 102 + **Schema:** 103 + ```json 104 + { 105 + "duplicates": { 106 + "<duplicate_id>": "<canonical_id>" 107 + }, 108 + "comment": "Entry IDs that map to the same canonical content" 109 + } 110 + ``` 111 + 112 + **Example:** 113 + ```json 114 + { 115 + "duplicates": { 116 + "https://example.com/posts/123?utm_source=rss": "https://example.com/posts/123", 117 + "https://example.com/feed/item-duplicate": "https://example.com/feed/item-original" 118 + }, 119 + "comment": "Entry IDs that map to the same canonical content" 120 + } 121 + ``` 122 + 123 + ### 3. Feed Entry Files (`<username>/<entry_id>.json`) 124 + 125 + Individual feed entries are stored as normalized Atom entries, regardless of their original format (RSS/Atom). 126 + 127 + **Schema:** 128 + ```json 129 + { 130 + "id": "string", 131 + "title": "string", 132 + "link": "string (URL)", 133 + "updated": "string (ISO 8601 datetime)", 134 + "published": "string (ISO 8601 datetime) | null", 135 + "summary": "string | null", 136 + "content": "string | null", 137 + "content_type": "html | text | xhtml", 138 + "author": { 139 + "name": "string | null", 140 + "email": "string | null", 141 + "uri": "string (URL) | null" 142 + } | null, 143 + "categories": ["string", ...], 144 + "rights": "string | null", 145 + "source": "string (URL) | null" 146 + } 147 + ``` 148 + 149 + **Example:** 150 + ```json 151 + { 152 + "id": "https://johndoe.blog/posts/my-first-post", 153 + "title": "My First Blog Post", 154 + "link": "https://johndoe.blog/posts/my-first-post", 155 + "updated": "2024-01-20T14:22:00", 156 + "published": "2024-01-20T09:00:00", 157 + "summary": "This is a summary of my first blog post.", 158 + "content": "<p>This is the full content of my <strong>first</strong> blog post with HTML formatting.</p>", 159 + "content_type": "html", 160 + "author": { 161 + "name": "John Doe", 162 + "email": "john@example.com", 163 + "uri": "https://johndoe.blog" 164 + }, 165 + "categories": ["blogging", "personal"], 166 + "rights": "Copyright 2024 John Doe", 167 + "source": "https://johndoe.blog/feed.xml" 168 + } 169 + ``` 170 + 171 + ## Python Class Integration 172 + 173 + To leverage Thicket's existing validation and business logic, third-party clients should use the following Python classes from the `thicket.models` package: 174 + 175 + ### Core Data Models 176 + 177 + ```python 178 + from thicket.models import ( 179 + AtomEntry, # Feed entry representation 180 + GitStoreIndex, # Repository index 181 + UserMetadata, # User information 182 + DuplicateMap, # Duplicate ID mappings 183 + FeedMetadata, # Feed-level metadata 184 + ThicketConfig, # Configuration 185 + UserConfig, # User configuration 186 + ZulipAssociation # Zulip server/user_id pairs 187 + ) 188 + ``` 189 + 190 + ### Repository Operations 191 + 192 + ```python 193 + from thicket.core.git_store import GitStore 194 + from thicket.core.feed_parser import FeedParser 195 + 196 + # Initialize git store 197 + store = GitStore(Path("/path/to/git/store")) 198 + 199 + # Read data 200 + index = store._load_index() # Load index.json 201 + user = store.get_user("username") # Get user metadata 202 + entries = store.list_entries("username", limit=10) 203 + entry = store.get_entry("username", "entry_id") 204 + duplicates = store.get_duplicates() # Load duplicates.json 205 + 206 + # Write data 207 + store.add_user("username", display_name="Display Name") 208 + store.store_entry("username", atom_entry) 209 + store.add_duplicate("duplicate_id", "canonical_id") 210 + store.commit_changes("Commit message") 211 + 212 + # Zulip associations 213 + store.add_zulip_association("username", "myorg.zulipchat.com", "user@example.com") 214 + store.remove_zulip_association("username", "myorg.zulipchat.com", "user@example.com") 215 + associations = store.get_zulip_associations("username") 216 + 217 + # Search and statistics 218 + results = store.search_entries("query", username="optional") 219 + stats = store.get_stats() 220 + ``` 221 + 222 + ### Feed Processing 223 + 224 + ```python 225 + from thicket.core.feed_parser import FeedParser 226 + from pydantic import HttpUrl 227 + 228 + parser = FeedParser() 229 + 230 + # Fetch and parse feeds 231 + content = await parser.fetch_feed(HttpUrl("https://example.com/feed.xml")) 232 + feed_metadata, entries = parser.parse_feed(content, source_url) 233 + 234 + # Entry ID sanitization for filenames 235 + safe_filename = parser.sanitize_entry_id(entry.id) 236 + ``` 237 + 238 + ## File Naming and ID Sanitization 239 + 240 + Entry IDs from feeds are sanitized to create safe filenames using `FeedParser.sanitize_entry_id()`: 241 + 242 + - URLs are parsed and the path component is used as the base 243 + - Characters are limited to alphanumeric, hyphens, underscores, and periods 244 + - Other characters are replaced with underscores 245 + - Maximum length is 200 characters 246 + - Empty results default to "entry" 247 + 248 + **Examples:** 249 + - `https://example.com/posts/my-post` → `posts_my-post.json` 250 + - `https://blog.com/2024/01/title?utm=source` → `2024_01_title.json` 251 + 252 + ## Data Validation 253 + 254 + All JSON data should be validated using Pydantic models before writing to the store: 255 + 256 + ```python 257 + from thicket.models import AtomEntry 258 + from pydantic import ValidationError 259 + 260 + try: 261 + entry = AtomEntry(**json_data) 262 + # Data is valid, safe to store 263 + store.store_entry(username, entry) 264 + except ValidationError as e: 265 + # Handle validation errors 266 + print(f"Invalid entry data: {e}") 267 + ``` 268 + 269 + ## Timestamps 270 + 271 + All timestamps use ISO 8601 format in UTC: 272 + - `created`: When the record was first created 273 + - `last_updated`: When the record was last modified 274 + - `updated`: When the feed entry was last updated (from feed) 275 + - `published`: When the feed entry was originally published (from feed) 276 + 277 + ## Content Sanitization 278 + 279 + HTML content in entries is sanitized using the `FeedParser._sanitize_html()` method to prevent XSS attacks. Allowed tags and attributes are strictly controlled. 280 + 281 + **Allowed HTML tags:** 282 + `a`, `abbr`, `acronym`, `b`, `blockquote`, `br`, `code`, `em`, `i`, `li`, `ol`, `p`, `pre`, `strong`, `ul`, `h1`-`h6`, `img`, `div`, `span` 283 + 284 + **Allowed attributes:** 285 + - `a`: `href`, `title` 286 + - `img`: `src`, `alt`, `title`, `width`, `height` 287 + - `blockquote`: `cite` 288 + - `abbr`/`acronym`: `title` 289 + 290 + ## Error Handling and Robustness 291 + 292 + The store is designed to be fault-tolerant: 293 + 294 + - Invalid entries are skipped during processing with error logging 295 + - Malformed JSON files are ignored in listings 296 + - Missing files return `None` rather than raising exceptions 297 + - Git operations are atomic where possible 298 + 299 + ## Example Usage 300 + 301 + ### Reading the Store 302 + 303 + ```python 304 + from pathlib import Path 305 + from thicket.core.git_store import GitStore 306 + 307 + # Initialize 308 + store = GitStore(Path("/path/to/thicket/store")) 309 + 310 + # Get all users 311 + index = store._load_index() 312 + for username, user_metadata in index.users.items(): 313 + print(f"User: {user_metadata.display_name} ({username})") 314 + print(f" Feeds: {user_metadata.feeds}") 315 + print(f" Entries: {user_metadata.entry_count}") 316 + 317 + # Get recent entries for a user 318 + entries = store.list_entries("johndoe", limit=5) 319 + for entry in entries: 320 + print(f" - {entry.title} ({entry.updated})") 321 + ``` 322 + 323 + ### Adding Data 324 + 325 + ```python 326 + from thicket.models import AtomEntry 327 + from datetime import datetime 328 + from pydantic import HttpUrl 329 + 330 + # Create entry 331 + entry = AtomEntry( 332 + id="https://example.com/new-post", 333 + title="New Post", 334 + link=HttpUrl("https://example.com/new-post"), 335 + updated=datetime.now(), 336 + content="<p>Post content</p>", 337 + content_type="html" 338 + ) 339 + 340 + # Store entry 341 + store.store_entry("johndoe", entry) 342 + store.commit_changes("Add new blog post") 343 + ``` 344 + 345 + ## Zulip Integration 346 + 347 + The Thicket Git store supports Zulip bot integration for automatic feed posting with user mentions. 348 + 349 + ### Zulip Associations 350 + 351 + Users can be associated with their Zulip identities to enable @mentions: 352 + 353 + ```python 354 + # UserMetadata includes zulip_associations field 355 + user.zulip_associations = [ 356 + ZulipAssociation(server="myorg.zulipchat.com", user_id="alice"), 357 + ZulipAssociation(server="other.zulipchat.com", user_id="alice@example.com") 358 + ] 359 + 360 + # Methods for managing associations 361 + user.add_zulip_association("myorg.zulipchat.com", "alice") 362 + user.get_zulip_mention("myorg.zulipchat.com") # Returns "alice" 363 + user.remove_zulip_association("myorg.zulipchat.com", "alice") 364 + ``` 365 + 366 + ### CLI Management 367 + 368 + ```bash 369 + # Add association 370 + thicket zulip-add alice myorg.zulipchat.com alice@example.com 371 + 372 + # Remove association 373 + thicket zulip-remove alice myorg.zulipchat.com alice@example.com 374 + 375 + # List associations 376 + thicket zulip-list # All users 377 + thicket zulip-list alice # Specific user 378 + 379 + # Bulk import from CSV 380 + thicket zulip-import associations.csv 381 + ``` 382 + 383 + ### Bot Behavior 384 + 385 + When the Thicket Zulip bot posts articles: 386 + 387 + 1. It checks for Zulip associations matching the current server 388 + 2. If found, adds @mention to the post: `@**alice** posted:` 389 + 3. The mentioned user receives a notification in Zulip 390 + 391 + This enables automatic notifications when someone's blog post is shared. 392 + 393 + ## Versioning and Compatibility 394 + 395 + This specification describes version 1.1 of the Thicket Git store format. Changes from 1.0: 396 + - Added `zulip_associations` field to UserMetadata (backwards compatible - defaults to empty list) 397 + 398 + Future versions will maintain backward compatibility where possible, with migration tools provided for breaking changes. 399 + 400 + To check the store format version, examine the repository structure and JSON schemas. Stores created by Thicket 0.1.0+ follow this specification.
+30
bot-config/run-bot.sh
··· 1 + #!/bin/bash 2 + 3 + # Script to run the Thicket Zulip bot 4 + # Usage: ./run-bot.sh [config-file] 5 + 6 + set -e 7 + 8 + # Default configuration file 9 + CONFIG_FILE="${1:-./zuliprc}" 10 + 11 + # Check if config file exists 12 + if [ ! -f "$CONFIG_FILE" ]; then 13 + echo "Error: Configuration file '$CONFIG_FILE' not found." 14 + echo "Please copy zuliprc.template to zuliprc and fill in your bot credentials." 15 + exit 1 16 + fi 17 + 18 + # Check if we're in the right directory 19 + if [ ! -f "pyproject.toml" ]; then 20 + echo "Error: Please run this script from the thicket project root directory." 21 + exit 1 22 + fi 23 + 24 + echo "Starting Thicket Zulip bot with config: $CONFIG_FILE" 25 + echo "Bot will be available as @thicket in your Zulip chat." 26 + echo "Type Ctrl+C to stop the bot." 27 + echo "" 28 + 29 + # Run the bot using zulip-run-bot 30 + uv run zulip-run-bot src/thicket/bots/thicket_bot.py --config-file "$CONFIG_FILE"
+5
src/thicket/bots/__init__.py
··· 1 + """Zulip bot integration for thicket.""" 2 + 3 + from .thicket_bot import ThicketBotHandler 4 + 5 + __all__ = ["ThicketBotHandler"]
+7
src/thicket/bots/requirements.txt
··· 1 + # Requirements for Thicket Zulip bot 2 + # These are already included in the main thicket package 3 + pydantic>=2.11.0 4 + GitPython>=3.1.40 5 + feedparser>=6.0.11 6 + httpx>=0.28.0 7 + pyyaml>=6.0.0
+202
src/thicket/bots/test_bot.py
··· 1 + """Test utilities for the Thicket Zulip bot.""" 2 + 3 + import json 4 + from pathlib import Path 5 + from typing import Any, Dict, Optional 6 + from unittest.mock import Mock 7 + 8 + from ..models import AtomEntry, ThicketConfig 9 + from .thicket_bot import ThicketBotHandler 10 + 11 + 12 + class MockBotHandler: 13 + """Mock BotHandler for testing the Thicket bot.""" 14 + 15 + def __init__(self) -> None: 16 + """Initialize mock bot handler.""" 17 + self.storage_data: Dict[str, str] = {} 18 + self.sent_messages: list[Dict[str, Any]] = [] 19 + self.config_info = { 20 + "full_name": "Thicket Bot", 21 + "email": "thicket-bot@example.com" 22 + } 23 + 24 + def get_config_info(self) -> Dict[str, str]: 25 + """Return bot configuration info.""" 26 + return self.config_info 27 + 28 + def send_reply(self, message: Dict[str, Any], content: str) -> None: 29 + """Mock sending a reply.""" 30 + reply = { 31 + "type": "reply", 32 + "to": message.get("sender_id"), 33 + "content": content, 34 + "original_message": message 35 + } 36 + self.sent_messages.append(reply) 37 + 38 + def send_message(self, message: Dict[str, Any]) -> None: 39 + """Mock sending a message.""" 40 + self.sent_messages.append(message) 41 + 42 + @property 43 + def storage(self) -> 'MockStorage': 44 + """Return mock storage.""" 45 + return MockStorage(self.storage_data) 46 + 47 + 48 + class MockStorage: 49 + """Mock storage for bot state.""" 50 + 51 + def __init__(self, storage_data: Dict[str, str]) -> None: 52 + """Initialize with storage data.""" 53 + self.storage_data = storage_data 54 + 55 + def __enter__(self) -> 'MockStorage': 56 + """Context manager entry.""" 57 + return self 58 + 59 + def __exit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None: 60 + """Context manager exit.""" 61 + pass 62 + 63 + def get(self, key: str) -> Optional[str]: 64 + """Get value from storage.""" 65 + return self.storage_data.get(key) 66 + 67 + def put(self, key: str, value: str) -> None: 68 + """Put value in storage.""" 69 + self.storage_data[key] = value 70 + 71 + def contains(self, key: str) -> bool: 72 + """Check if key exists in storage.""" 73 + return key in self.storage_data 74 + 75 + 76 + def create_test_message( 77 + content: str, 78 + sender: str = "Test User", 79 + sender_id: int = 12345, 80 + message_type: str = "stream" 81 + ) -> Dict[str, Any]: 82 + """Create a test message for bot testing.""" 83 + return { 84 + "content": content, 85 + "sender_full_name": sender, 86 + "sender_id": sender_id, 87 + "type": message_type, 88 + "timestamp": 1642694400, # 2022-01-20 12:00:00 UTC 89 + "stream_id": 1, 90 + "subject": "test topic" 91 + } 92 + 93 + 94 + def create_test_entry( 95 + entry_id: str = "test-entry-1", 96 + title: str = "Test Article", 97 + link: str = "https://example.com/test-article" 98 + ) -> AtomEntry: 99 + """Create a test AtomEntry for testing.""" 100 + from datetime import datetime 101 + from pydantic import HttpUrl 102 + 103 + return AtomEntry( 104 + id=entry_id, 105 + title=title, 106 + link=HttpUrl(link), 107 + updated=datetime(2024, 1, 20, 12, 0, 0), 108 + published=datetime(2024, 1, 20, 10, 0, 0), 109 + summary="This is a test article summary", 110 + content="<p>This is test article content</p>", 111 + author={"name": "Test Author", "email": "author@example.com"} 112 + ) 113 + 114 + 115 + class BotTester: 116 + """Helper class for testing bot functionality.""" 117 + 118 + def __init__(self, config_path: Optional[Path] = None) -> None: 119 + """Initialize bot tester.""" 120 + self.bot = ThicketBotHandler() 121 + self.handler = MockBotHandler() 122 + 123 + if config_path: 124 + # Configure bot with test config 125 + self.configure_bot(config_path, "test-stream", "test-topic") 126 + 127 + def configure_bot( 128 + self, 129 + config_path: Path, 130 + stream: str = "test-stream", 131 + topic: str = "test-topic" 132 + ) -> None: 133 + """Configure the bot for testing.""" 134 + # Set bot configuration 135 + config_data = { 136 + "stream_name": stream, 137 + "topic_name": topic, 138 + "sync_interval": 300, 139 + "max_entries_per_sync": 10, 140 + "config_path": str(config_path) 141 + } 142 + 143 + self.handler.storage_data["bot_config"] = json.dumps(config_data) 144 + 145 + # Initialize bot 146 + self.bot._load_bot_config(self.handler) 147 + 148 + def send_command(self, command: str, sender: str = "Test User") -> list[Dict[str, Any]]: 149 + """Send a command to the bot and return responses.""" 150 + message = create_test_message(f"@thicket {command}", sender) 151 + 152 + # Clear previous messages 153 + self.handler.sent_messages.clear() 154 + 155 + # Send command 156 + self.bot.handle_message(message, self.handler) 157 + 158 + return self.handler.sent_messages.copy() 159 + 160 + def get_last_response_content(self) -> Optional[str]: 161 + """Get the content of the last bot response.""" 162 + if self.handler.sent_messages: 163 + return self.handler.sent_messages[-1].get("content") 164 + return None 165 + 166 + def get_last_message(self) -> Optional[Dict[str, Any]]: 167 + """Get the last sent message.""" 168 + if self.handler.sent_messages: 169 + return self.handler.sent_messages[-1] 170 + return None 171 + 172 + def assert_response_contains(self, text: str) -> None: 173 + """Assert that the last response contains specific text.""" 174 + content = self.get_last_response_content() 175 + assert content is not None, "No response received" 176 + assert text in content, f"Response does not contain '{text}': {content}" 177 + 178 + 179 + # Example usage for testing 180 + if __name__ == "__main__": 181 + # Create a test config file 182 + test_config = Path("/tmp/test_thicket.yaml") 183 + 184 + # Create bot tester 185 + tester = BotTester() 186 + 187 + # Test help command 188 + responses = tester.send_command("help") 189 + print(f"Help response: {tester.get_last_response_content()}") 190 + 191 + # Test status command 192 + responses = tester.send_command("status") 193 + print(f"Status response: {tester.get_last_response_content()}") 194 + 195 + # Test configuration 196 + responses = tester.send_command("config stream general") 197 + tester.assert_response_contains("Stream set to") 198 + 199 + responses = tester.send_command("config topic 'Feed Updates'") 200 + tester.assert_response_contains("Topic set to") 201 + 202 + print("All tests passed!")
+754
src/thicket/bots/thicket_bot.py
··· 1 + """Zulip bot for automatically posting thicket feed updates.""" 2 + 3 + import asyncio 4 + import json 5 + import logging 6 + import os 7 + import time 8 + from datetime import datetime 9 + from pathlib import Path 10 + from typing import Any, Dict, List, Optional, Set, Tuple 11 + 12 + from zulip_bots.lib import BotHandler 13 + 14 + # Handle imports for both direct execution and package import 15 + try: 16 + from ..core.git_store import GitStore 17 + from ..models import AtomEntry, ThicketConfig 18 + from ..cli.commands.sync import sync_feed 19 + except ImportError: 20 + # When run directly by zulip-bots, add the package to path 21 + import sys 22 + src_dir = Path(__file__).parent.parent.parent 23 + if str(src_dir) not in sys.path: 24 + sys.path.insert(0, str(src_dir)) 25 + 26 + from thicket.core.git_store import GitStore 27 + from thicket.models import AtomEntry, ThicketConfig 28 + from thicket.cli.commands.sync import sync_feed 29 + 30 + 31 + class ThicketBotHandler: 32 + """Zulip bot that monitors thicket feeds and posts new articles.""" 33 + 34 + def __init__(self) -> None: 35 + """Initialize the thicket bot.""" 36 + self.logger = logging.getLogger(__name__) 37 + self.git_store: Optional[GitStore] = None 38 + self.config: Optional[ThicketConfig] = None 39 + self.posted_entries: Set[str] = set() 40 + 41 + # Bot configuration from storage 42 + self.stream_name: Optional[str] = None 43 + self.topic_name: Optional[str] = None 44 + self.sync_interval: int = 300 # 5 minutes default 45 + self.max_entries_per_sync: int = 10 46 + self.config_path: Optional[Path] = None 47 + 48 + # Debug mode configuration 49 + self.debug_user: Optional[str] = None 50 + self.debug_zulip_user_id: Optional[str] = None 51 + 52 + def usage(self) -> str: 53 + """Return bot usage instructions.""" 54 + return """ 55 + **Thicket Feed Bot** 56 + 57 + This bot automatically monitors thicket feeds and posts new articles. 58 + 59 + Commands: 60 + - `@mention status` - Show current bot status and configuration 61 + - `@mention sync now` - Force an immediate sync 62 + - `@mention reset` - Clear posting history (will repost recent entries) 63 + - `@mention config stream <stream_name>` - Set target stream 64 + - `@mention config topic <topic_name>` - Set target topic 65 + - `@mention config interval <seconds>` - Set sync interval 66 + - `@mention help` - Show this help message 67 + """ 68 + 69 + def initialize(self, bot_handler: BotHandler) -> None: 70 + """Initialize the bot with persistent storage.""" 71 + self.logger.info("Initializing ThicketBot") 72 + 73 + # Get configuration from environment (set by CLI) 74 + self.debug_user = os.getenv("THICKET_DEBUG_USER") 75 + config_path_env = os.getenv("THICKET_CONFIG_PATH") 76 + if config_path_env: 77 + self.config_path = Path(config_path_env) 78 + self.logger.info(f"Using thicket config: {self.config_path}") 79 + 80 + # Load bot configuration from persistent storage 81 + self._load_bot_config(bot_handler) 82 + 83 + # Initialize thicket components 84 + if self.config_path: 85 + try: 86 + self._initialize_thicket() 87 + self._load_posted_entries(bot_handler) 88 + 89 + # Validate debug mode if enabled 90 + if self.debug_user: 91 + self._validate_debug_mode(bot_handler) 92 + 93 + except Exception as e: 94 + self.logger.error(f"Failed to initialize thicket: {e}") 95 + 96 + # Start background sync loop 97 + self._schedule_sync(bot_handler) 98 + 99 + def handle_message(self, message: Dict[str, Any], bot_handler: BotHandler) -> None: 100 + """Handle incoming Zulip messages.""" 101 + content = message["content"].strip() 102 + sender = message["sender_full_name"] 103 + 104 + # Only respond to mentions 105 + if not self._is_mentioned(content, bot_handler): 106 + return 107 + 108 + # Parse command 109 + cleaned_content = self._clean_mention(content, bot_handler) 110 + command_parts = cleaned_content.split() 111 + 112 + if not command_parts: 113 + self._send_help(message, bot_handler) 114 + return 115 + 116 + command = command_parts[0].lower() 117 + 118 + try: 119 + if command == "help": 120 + self._send_help(message, bot_handler) 121 + elif command == "status": 122 + self._send_status(message, bot_handler, sender) 123 + elif command == "sync" and len(command_parts) > 1 and command_parts[1] == "now": 124 + self._handle_force_sync(message, bot_handler, sender) 125 + elif command == "reset": 126 + self._handle_reset_command(message, bot_handler, sender) 127 + elif command == "config": 128 + self._handle_config_command(message, bot_handler, command_parts[1:], sender) 129 + else: 130 + bot_handler.send_reply(message, f"Unknown command: {command}. Type `@mention help` for usage.") 131 + except Exception as e: 132 + self.logger.error(f"Error handling command '{command}': {e}") 133 + bot_handler.send_reply(message, f"Error processing command: {str(e)}") 134 + 135 + def _is_mentioned(self, content: str, bot_handler: BotHandler) -> bool: 136 + """Check if the bot is mentioned in the message.""" 137 + try: 138 + # Get bot's actual name from Zulip 139 + bot_info = bot_handler._client.get_profile() 140 + if bot_info.get('result') == 'success': 141 + bot_name = bot_info.get('full_name', '').lower() 142 + if bot_name: 143 + return f"@{bot_name}" in content.lower() or f"@**{bot_name}**" in content.lower() 144 + except Exception as e: 145 + self.logger.debug(f"Could not get bot profile: {e}") 146 + 147 + # Fallback to generic check 148 + return "@thicket" in content.lower() 149 + 150 + def _clean_mention(self, content: str, bot_handler: BotHandler) -> str: 151 + """Remove bot mention from message content.""" 152 + import re 153 + 154 + try: 155 + # Get bot's actual name from Zulip 156 + bot_info = bot_handler._client.get_profile() 157 + if bot_info.get('result') == 'success': 158 + bot_name = bot_info.get('full_name', '') 159 + if bot_name: 160 + # Remove @bot_name or @**bot_name** 161 + escaped_name = re.escape(bot_name) 162 + content = re.sub(rf'@(?:\*\*)?{escaped_name}(?:\*\*)?', '', content, flags=re.IGNORECASE).strip() 163 + return content 164 + except Exception as e: 165 + self.logger.debug(f"Could not get bot profile for mention cleaning: {e}") 166 + 167 + # Fallback to removing @thicket 168 + content = re.sub(r'@(?:\*\*)?thicket(?:\*\*)?', '', content, flags=re.IGNORECASE).strip() 169 + return content 170 + 171 + def _send_help(self, message: Dict[str, Any], bot_handler: BotHandler) -> None: 172 + """Send help message.""" 173 + bot_handler.send_reply(message, self.usage()) 174 + 175 + def _send_status(self, message: Dict[str, Any], bot_handler: BotHandler, sender: str) -> None: 176 + """Send bot status information.""" 177 + status_lines = [ 178 + f"**Thicket Bot Status** (requested by {sender})", 179 + "", 180 + ] 181 + 182 + # Debug mode status 183 + if self.debug_user: 184 + status_lines.extend([ 185 + f"🐛 **Debug Mode:** ENABLED", 186 + f"🎯 **Debug User:** {self.debug_user}", 187 + "", 188 + ]) 189 + else: 190 + status_lines.extend([ 191 + f"📍 **Stream:** {self.stream_name or 'Not configured'}", 192 + f"📝 **Topic:** {self.topic_name or 'Not configured'}", 193 + "", 194 + ]) 195 + 196 + status_lines.extend([ 197 + f"⏱️ **Sync Interval:** {self.sync_interval}s ({self.sync_interval // 60}m {self.sync_interval % 60}s)", 198 + f"📊 **Max Entries/Sync:** {self.max_entries_per_sync}", 199 + f"📁 **Config Path:** {self.config_path or 'Not configured'}", 200 + "", 201 + f"📄 **Tracked Entries:** {len(self.posted_entries)}", 202 + f"🔄 **Catchup Mode:** {'Active (first run)' if len(self.posted_entries) == 0 else 'Inactive'}", 203 + f"✅ **Thicket Initialized:** {'Yes' if self.git_store else 'No'}", 204 + ]) 205 + 206 + bot_handler.send_reply(message, "\n".join(status_lines)) 207 + 208 + def _handle_force_sync(self, message: Dict[str, Any], bot_handler: BotHandler, sender: str) -> None: 209 + """Handle immediate sync request.""" 210 + if not self._check_initialization(message, bot_handler): 211 + return 212 + 213 + bot_handler.send_reply(message, f"🔄 Starting immediate sync... (requested by {sender})") 214 + 215 + try: 216 + new_entries = self._perform_sync(bot_handler) 217 + bot_handler.send_reply( 218 + message, 219 + f"✅ Sync completed! Found {len(new_entries)} new entries." 220 + ) 221 + except Exception as e: 222 + self.logger.error(f"Force sync failed: {e}") 223 + bot_handler.send_reply(message, f"❌ Sync failed: {str(e)}") 224 + 225 + def _handle_reset_command(self, message: Dict[str, Any], bot_handler: BotHandler, sender: str) -> None: 226 + """Handle reset command to clear posted entries tracking.""" 227 + try: 228 + self.posted_entries.clear() 229 + self._save_posted_entries(bot_handler) 230 + bot_handler.send_reply( 231 + message, 232 + f"✅ Posting history reset! Recent entries will be posted on next sync. (requested by {sender})" 233 + ) 234 + self.logger.info(f"Posted entries tracking reset by {sender}") 235 + except Exception as e: 236 + self.logger.error(f"Reset failed: {e}") 237 + bot_handler.send_reply(message, f"❌ Reset failed: {str(e)}") 238 + 239 + def _handle_config_command( 240 + self, 241 + message: Dict[str, Any], 242 + bot_handler: BotHandler, 243 + args: List[str], 244 + sender: str 245 + ) -> None: 246 + """Handle configuration commands.""" 247 + if len(args) < 2: 248 + bot_handler.send_reply(message, "Usage: `@mention config <setting> <value>`") 249 + return 250 + 251 + setting = args[0].lower() 252 + value = " ".join(args[1:]) 253 + 254 + if setting == "stream": 255 + self.stream_name = value 256 + self._save_bot_config(bot_handler) 257 + bot_handler.send_reply(message, f"✅ Stream set to: **{value}** (by {sender})") 258 + 259 + elif setting == "topic": 260 + self.topic_name = value 261 + self._save_bot_config(bot_handler) 262 + bot_handler.send_reply(message, f"✅ Topic set to: **{value}** (by {sender})") 263 + 264 + elif setting == "interval": 265 + try: 266 + interval = int(value) 267 + if interval < 60: 268 + bot_handler.send_reply(message, "❌ Interval must be at least 60 seconds") 269 + return 270 + self.sync_interval = interval 271 + self._save_bot_config(bot_handler) 272 + bot_handler.send_reply(message, f"✅ Sync interval set to: **{interval}s** (by {sender})") 273 + except ValueError: 274 + bot_handler.send_reply(message, "❌ Invalid interval value. Must be a number of seconds.") 275 + 276 + else: 277 + bot_handler.send_reply( 278 + message, 279 + f"❌ Unknown setting: {setting}. Available: stream, topic, interval" 280 + ) 281 + 282 + def _load_bot_config(self, bot_handler: BotHandler) -> None: 283 + """Load bot configuration from persistent storage.""" 284 + try: 285 + config_data = bot_handler.storage.get("bot_config") 286 + if config_data: 287 + config = json.loads(config_data) 288 + self.stream_name = config.get("stream_name") 289 + self.topic_name = config.get("topic_name") 290 + self.sync_interval = config.get("sync_interval", 300) 291 + self.max_entries_per_sync = config.get("max_entries_per_sync", 10) 292 + except Exception as e: 293 + # Bot config not found on first run is expected 294 + pass 295 + 296 + def _save_bot_config(self, bot_handler: BotHandler) -> None: 297 + """Save bot configuration to persistent storage.""" 298 + try: 299 + config_data = { 300 + "stream_name": self.stream_name, 301 + "topic_name": self.topic_name, 302 + "sync_interval": self.sync_interval, 303 + "max_entries_per_sync": self.max_entries_per_sync, 304 + } 305 + bot_handler.storage.put("bot_config", json.dumps(config_data)) 306 + except Exception as e: 307 + self.logger.error(f"Error saving bot config: {e}") 308 + 309 + def _initialize_thicket(self) -> None: 310 + """Initialize thicket components.""" 311 + if not self.config_path or not self.config_path.exists(): 312 + raise ValueError("Thicket config file not found") 313 + 314 + # Load thicket configuration 315 + import yaml 316 + with open(self.config_path) as f: 317 + config_data = yaml.safe_load(f) 318 + self.config = ThicketConfig(**config_data) 319 + 320 + # Initialize git store 321 + self.git_store = GitStore(self.config.git_store) 322 + 323 + self.logger.info("Thicket components initialized successfully") 324 + 325 + def _validate_debug_mode(self, bot_handler: BotHandler) -> None: 326 + """Validate debug mode configuration.""" 327 + if not self.debug_user or not self.git_store: 328 + return 329 + 330 + # Get current Zulip server from environment 331 + zulip_site_url = os.getenv("THICKET_ZULIP_SITE_URL", "") 332 + server_url = zulip_site_url.replace("https://", "").replace("http://", "") 333 + 334 + # Check if debug user exists in thicket 335 + user = self.git_store.get_user(self.debug_user) 336 + if not user: 337 + raise ValueError(f"Debug user '{self.debug_user}' not found in thicket") 338 + 339 + # Check if user has Zulip association for this server 340 + if not server_url: 341 + raise ValueError("Could not determine Zulip server URL") 342 + 343 + zulip_user_id = user.get_zulip_mention(server_url) 344 + if not zulip_user_id: 345 + raise ValueError(f"User '{self.debug_user}' has no Zulip association for server '{server_url}'") 346 + 347 + # Try to look up the actual Zulip user ID from the email address 348 + # But don't fail if we can't - we'll try again when sending messages 349 + actual_user_id = self._lookup_zulip_user_id(bot_handler, zulip_user_id) 350 + if actual_user_id and actual_user_id != zulip_user_id: 351 + # Successfully resolved to numeric ID 352 + self.debug_zulip_user_id = actual_user_id 353 + self.logger.info(f"Debug mode enabled: Will send DMs to {self.debug_user} (email: {zulip_user_id}, user_id: {actual_user_id}) on {server_url}") 354 + else: 355 + # Keep the email address, will resolve later when sending 356 + self.debug_zulip_user_id = zulip_user_id 357 + self.logger.info(f"Debug mode enabled: Will send DMs to {self.debug_user} ({zulip_user_id}) on {server_url} (will resolve user ID when sending)") 358 + 359 + def _lookup_zulip_user_id(self, bot_handler: BotHandler, email_or_id: str) -> Optional[str]: 360 + """Look up Zulip user ID from email address or return the ID if it's already numeric.""" 361 + # If it's already a numeric user ID, return it 362 + if email_or_id.isdigit(): 363 + return email_or_id 364 + 365 + try: 366 + client = bot_handler._client 367 + if not client: 368 + self.logger.error("No Zulip client available for user lookup") 369 + return None 370 + 371 + # First try the get_user_by_email API if available 372 + try: 373 + user_result = client.get_user_by_email(email_or_id) 374 + if user_result.get('result') == 'success': 375 + user_data = user_result.get('user', {}) 376 + user_id = user_data.get('user_id') 377 + if user_id: 378 + self.logger.info(f"Found user ID {user_id} for '{email_or_id}' via get_user_by_email API") 379 + return str(user_id) 380 + except (AttributeError, Exception): 381 + pass 382 + 383 + # Fallback: Get all users and search through them 384 + users_result = client.get_users() 385 + if users_result.get('result') == 'success': 386 + for user in users_result['members']: 387 + user_email = user.get('email', '') 388 + delivery_email = user.get('delivery_email', '') 389 + 390 + if (user_email == email_or_id or 391 + delivery_email == email_or_id or 392 + str(user.get('user_id')) == email_or_id): 393 + user_id = user.get('user_id') 394 + return str(user_id) 395 + 396 + self.logger.error(f"No user found with identifier '{email_or_id}'. Searched {len(users_result['members'])} users.") 397 + return None 398 + else: 399 + self.logger.error(f"Failed to get users: {users_result.get('msg', 'Unknown error')}") 400 + return None 401 + 402 + except Exception as e: 403 + self.logger.error(f"Error looking up user ID for '{email_or_id}': {e}") 404 + return None 405 + 406 + def _lookup_zulip_user_info(self, bot_handler: BotHandler, email_or_id: str) -> Tuple[Optional[str], Optional[str]]: 407 + """Look up both Zulip user ID and full name from email address.""" 408 + if email_or_id.isdigit(): 409 + return email_or_id, None 410 + 411 + try: 412 + client = bot_handler._client 413 + if not client: 414 + return None, None 415 + 416 + # Try get_user_by_email API first 417 + try: 418 + user_result = client.get_user_by_email(email_or_id) 419 + if user_result.get('result') == 'success': 420 + user_data = user_result.get('user', {}) 421 + user_id = user_data.get('user_id') 422 + full_name = user_data.get('full_name', '') 423 + if user_id: 424 + return str(user_id), full_name 425 + except AttributeError: 426 + pass 427 + 428 + # Fallback: search all users 429 + users_result = client.get_users() 430 + if users_result.get('result') == 'success': 431 + for user in users_result['members']: 432 + if (user.get('email') == email_or_id or 433 + user.get('delivery_email') == email_or_id): 434 + return str(user.get('user_id')), user.get('full_name', '') 435 + 436 + return None, None 437 + 438 + except Exception as e: 439 + self.logger.error(f"Error looking up user info for '{email_or_id}': {e}") 440 + return None, None 441 + 442 + def _load_posted_entries(self, bot_handler: BotHandler) -> None: 443 + """Load the set of already posted entries.""" 444 + try: 445 + posted_data = bot_handler.storage.get("posted_entries") 446 + if posted_data: 447 + self.posted_entries = set(json.loads(posted_data)) 448 + except Exception: 449 + # Empty set on first run is expected 450 + self.posted_entries = set() 451 + 452 + def _save_posted_entries(self, bot_handler: BotHandler) -> None: 453 + """Save the set of posted entries.""" 454 + try: 455 + bot_handler.storage.put("posted_entries", json.dumps(list(self.posted_entries))) 456 + except Exception as e: 457 + self.logger.error(f"Error saving posted entries: {e}") 458 + 459 + def _check_initialization(self, message: Dict[str, Any], bot_handler: BotHandler) -> bool: 460 + """Check if thicket is properly initialized.""" 461 + if not self.git_store or not self.config: 462 + bot_handler.send_reply( 463 + message, 464 + "❌ Thicket not initialized. Please check configuration." 465 + ) 466 + return False 467 + 468 + # In debug mode, we don't need stream/topic configuration 469 + if self.debug_user: 470 + return True 471 + 472 + if not self.stream_name or not self.topic_name: 473 + bot_handler.send_reply( 474 + message, 475 + "❌ Stream and topic must be configured first. Use `@mention config stream <name>` and `@mention config topic <name>`" 476 + ) 477 + return False 478 + 479 + return True 480 + 481 + def _schedule_sync(self, bot_handler: BotHandler) -> None: 482 + """Schedule periodic sync operations.""" 483 + def sync_loop(): 484 + while True: 485 + try: 486 + # Check if we can sync 487 + can_sync = (self.git_store and 488 + ((self.stream_name and self.topic_name) or 489 + self.debug_user)) 490 + 491 + if can_sync: 492 + self._perform_sync(bot_handler) 493 + 494 + time.sleep(self.sync_interval) 495 + except Exception as e: 496 + self.logger.error(f"Error in sync loop: {e}") 497 + time.sleep(60) # Wait before retrying 498 + 499 + # Start background thread 500 + import threading 501 + sync_thread = threading.Thread(target=sync_loop, daemon=True) 502 + sync_thread.start() 503 + 504 + def _perform_sync(self, bot_handler: BotHandler) -> List[AtomEntry]: 505 + """Perform thicket sync and return new entries.""" 506 + if not self.config or not self.git_store: 507 + return [] 508 + 509 + new_entries: List[Tuple[AtomEntry, str]] = [] # (entry, username) pairs 510 + is_first_run = len(self.posted_entries) == 0 511 + 512 + # Get all users and their feeds from git store 513 + users_with_feeds = self.git_store.list_all_users_with_feeds() 514 + 515 + # Sync each user's feeds 516 + for username, feed_urls in users_with_feeds: 517 + for feed_url in feed_urls: 518 + try: 519 + # Run async sync function 520 + loop = asyncio.new_event_loop() 521 + asyncio.set_event_loop(loop) 522 + try: 523 + new_count, _ = loop.run_until_complete( 524 + sync_feed(self.git_store, username, str(feed_url), dry_run=False) 525 + ) 526 + 527 + entries_to_check = [] 528 + 529 + if new_count > 0: 530 + # Get the newly added entries 531 + entries_to_check = self.git_store.list_entries(username, limit=new_count) 532 + 533 + # Always check for catchup mode on first run 534 + if is_first_run: 535 + # Catchup mode: get last 5 entries on first run 536 + catchup_entries = self.git_store.list_entries(username, limit=5) 537 + entries_to_check = catchup_entries if not entries_to_check else entries_to_check 538 + 539 + for entry in entries_to_check: 540 + entry_key = f"{username}:{entry.id}" 541 + if entry_key not in self.posted_entries: 542 + new_entries.append((entry, username)) 543 + if len(new_entries) >= self.max_entries_per_sync: 544 + break 545 + 546 + finally: 547 + loop.close() 548 + 549 + except Exception as e: 550 + self.logger.error(f"Error syncing feed {feed_url} for user {username}: {e}") 551 + 552 + if len(new_entries) >= self.max_entries_per_sync: 553 + break 554 + 555 + # Post new entries to Zulip with rate limiting 556 + if new_entries: 557 + posted_count = 0 558 + 559 + for i, (entry, username) in enumerate(new_entries): 560 + self._post_entry_to_zulip(entry, bot_handler, username) 561 + self.posted_entries.add(f"{username}:{entry.id}") 562 + posted_count += 1 563 + 564 + # Rate limiting: pause after every 5 messages 565 + if posted_count % 5 == 0 and i < len(new_entries) - 1: 566 + time.sleep(5) 567 + 568 + self._save_posted_entries(bot_handler) 569 + 570 + return [entry for entry, _ in new_entries] 571 + 572 + def _post_entry_to_zulip(self, entry: AtomEntry, bot_handler: BotHandler, username: str) -> None: 573 + """Post a single entry to the configured Zulip stream/topic or debug user DM.""" 574 + try: 575 + # Get current Zulip server from environment 576 + zulip_site_url = os.getenv("THICKET_ZULIP_SITE_URL", "") 577 + server_url = zulip_site_url.replace("https://", "").replace("http://", "") 578 + 579 + # Build author/date info consistently 580 + mention_info = "" 581 + if server_url and self.git_store: 582 + user = self.git_store.get_user(username) 583 + if user: 584 + zulip_user_id = user.get_zulip_mention(server_url) 585 + if zulip_user_id: 586 + # Look up the actual Zulip full name for proper @mention 587 + _, zulip_full_name = self._lookup_zulip_user_info(bot_handler, zulip_user_id) 588 + display_name = zulip_full_name or user.display_name or username 589 + 590 + # Check if author is different from the user - avoid redundancy 591 + author_name = entry.author and entry.author.get("name") 592 + if author_name and author_name.lower() != display_name.lower(): 593 + author_info = f" (by {author_name})" 594 + else: 595 + author_info = "" 596 + 597 + published_info = "" 598 + if entry.published: 599 + published_info = f" • {entry.published.strftime('%Y-%m-%d')}" 600 + 601 + mention_info = f"@**{display_name}** posted{author_info}{published_info}:\n\n" 602 + 603 + # If no Zulip user found, use consistent format without @mention 604 + if not mention_info: 605 + user = self.git_store.get_user(username) if self.git_store else None 606 + display_name = user.display_name if user else username 607 + 608 + author_name = entry.author and entry.author.get("name") 609 + if author_name and author_name.lower() != display_name.lower(): 610 + author_info = f" (by {author_name})" 611 + else: 612 + author_info = "" 613 + 614 + published_info = "" 615 + if entry.published: 616 + published_info = f" • {entry.published.strftime('%Y-%m-%d')}" 617 + 618 + mention_info = f"**{display_name}** posted{author_info}{published_info}:\n\n" 619 + 620 + # Format the message with HTML processing 621 + message_lines = [ 622 + f"**{entry.title}**", 623 + f"🔗 {entry.link}", 624 + ] 625 + 626 + if entry.summary: 627 + # Process HTML in summary and truncate if needed 628 + processed_summary = self._process_html_content(entry.summary) 629 + if len(processed_summary) > 400: 630 + processed_summary = processed_summary[:397] + "..." 631 + message_lines.append(f"\n{processed_summary}") 632 + 633 + message_content = mention_info + "\n".join(message_lines) 634 + 635 + # Choose destination based on mode 636 + if self.debug_user and self.debug_zulip_user_id: 637 + # Debug mode: send DM 638 + debug_message = f"🐛 **DEBUG:** New article from thicket user `{username}`:\n\n{message_content}" 639 + 640 + # Ensure we have the numeric user ID 641 + user_id_to_use = self.debug_zulip_user_id 642 + if not user_id_to_use.isdigit(): 643 + # Need to look up the numeric ID 644 + resolved_id = self._lookup_zulip_user_id(bot_handler, user_id_to_use) 645 + if resolved_id: 646 + user_id_to_use = resolved_id 647 + self.logger.debug(f"Resolved {self.debug_zulip_user_id} to user ID {user_id_to_use}") 648 + else: 649 + self.logger.error(f"Could not resolve user ID for {self.debug_zulip_user_id}") 650 + return 651 + 652 + try: 653 + # For private messages, user_id needs to be an integer, not string 654 + user_id_int = int(user_id_to_use) 655 + bot_handler.send_message({ 656 + "type": "private", 657 + "to": [user_id_int], # Use integer user ID 658 + "content": debug_message 659 + }) 660 + except ValueError: 661 + # If conversion to int fails, user_id_to_use might be an email 662 + try: 663 + bot_handler.send_message({ 664 + "type": "private", 665 + "to": [user_id_to_use], # Try as string (email) 666 + "content": debug_message 667 + }) 668 + except Exception as e2: 669 + self.logger.error(f"Failed to send DM to {self.debug_user} (tried both int and string): {e2}") 670 + return 671 + except Exception as e: 672 + self.logger.error(f"Failed to send DM to {self.debug_user} ({user_id_to_use}): {e}") 673 + return 674 + self.logger.info(f"Posted entry to debug user {self.debug_user}: {entry.title}") 675 + else: 676 + # Normal mode: send to stream/topic 677 + bot_handler.send_message({ 678 + "type": "stream", 679 + "to": self.stream_name, 680 + "subject": self.topic_name, 681 + "content": message_content 682 + }) 683 + self.logger.info(f"Posted entry to stream: {entry.title} (user: {username})") 684 + 685 + except Exception as e: 686 + self.logger.error(f"Error posting entry to Zulip: {e}") 687 + 688 + def _process_html_content(self, html_content: str) -> str: 689 + """Process HTML content from feeds to clean Zulip-compatible markdown.""" 690 + if not html_content: 691 + return "" 692 + 693 + try: 694 + # Try to use markdownify for proper HTML to Markdown conversion 695 + from markdownify import markdownify as md 696 + 697 + # Convert HTML to Markdown with compact settings for summaries 698 + markdown = md( 699 + html_content, 700 + heading_style="ATX", # Use # for headings (but we'll post-process these) 701 + bullets="-", # Use - for bullets 702 + convert=['a', 'b', 'strong', 'i', 'em', 'code', 'pre', 'p', 'br', 'ul', 'ol', 'li', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6'] 703 + ).strip() 704 + 705 + # Post-process to convert headings to bold for compact summaries 706 + import re 707 + # Convert markdown headers to bold with period 708 + markdown = re.sub(r'^#{1,6}\s*(.+)$', r'**\1.**', markdown, flags=re.MULTILINE) 709 + 710 + # Clean up excessive newlines and make more compact 711 + markdown = re.sub(r'\n\s*\n\s*\n+', ' ', markdown) # Multiple newlines become space 712 + markdown = re.sub(r'\n\s*\n', '. ', markdown) # Double newlines become sentence breaks 713 + markdown = re.sub(r'\n', ' ', markdown) # Single newlines become spaces 714 + 715 + # Clean up double periods and excessive whitespace 716 + markdown = re.sub(r'\.\.+', '.', markdown) 717 + markdown = re.sub(r'\s+', ' ', markdown) 718 + return markdown.strip() 719 + 720 + except ImportError: 721 + # Fallback: manual HTML processing 722 + import re 723 + content = html_content 724 + 725 + # Convert headings to bold with periods for compact summaries 726 + content = re.sub(r'<h[1-6](?:\s[^>]*)?>([^<]*)</h[1-6]>', r'**\1.** ', content, flags=re.IGNORECASE) 727 + 728 + # Convert common HTML elements to Markdown 729 + content = re.sub(r'<(?:strong|b)(?:\s[^>]*)?>([^<]*)</(?:strong|b)>', r'**\1**', content, flags=re.IGNORECASE) 730 + content = re.sub(r'<(?:em|i)(?:\s[^>]*)?>([^<]*)</(?:em|i)>', r'*\1*', content, flags=re.IGNORECASE) 731 + content = re.sub(r'<code(?:\s[^>]*)?>([^<]*)</code>', r'`\1`', content, flags=re.IGNORECASE) 732 + content = re.sub(r'<a(?:\s[^>]*?)?\s*href=["\']([^"\']*)["\'](?:\s[^>]*)?>([^<]*)</a>', r'[\2](\1)', content, flags=re.IGNORECASE) 733 + 734 + # Convert block elements to spaces instead of newlines for compactness 735 + content = re.sub(r'<br\s*/?>', ' ', content, flags=re.IGNORECASE) 736 + content = re.sub(r'</p>\s*<p>', '. ', content, flags=re.IGNORECASE) 737 + content = re.sub(r'</?(?:p|div)(?:\s[^>]*)?>', ' ', content, flags=re.IGNORECASE) 738 + 739 + # Remove remaining HTML tags 740 + content = re.sub(r'<[^>]+>', '', content) 741 + 742 + # Clean up whitespace and make compact 743 + content = re.sub(r'\s+', ' ', content) # Multiple whitespace becomes single space 744 + content = re.sub(r'\.\.+', '.', content) # Multiple periods become single period 745 + return content.strip() 746 + 747 + except Exception as e: 748 + self.logger.error(f"Error processing HTML content: {e}") 749 + # Last resort: just strip HTML tags 750 + import re 751 + return re.sub(r'<[^>]+>', '', html_content).strip() 752 + 753 + 754 + handler_class = ThicketBotHandler
+229
src/thicket/cli/commands/bot.py
··· 1 + """Bot management commands for thicket.""" 2 + 3 + import subprocess 4 + import sys 5 + from pathlib import Path 6 + 7 + import typer 8 + from rich.console import Console 9 + 10 + from ..main import app 11 + from ..utils import print_error, print_info, print_success 12 + 13 + console = Console() 14 + 15 + 16 + @app.command() 17 + def bot( 18 + action: str = typer.Argument(..., help="Action: run, test, or status"), 19 + config_file: Path = typer.Option( 20 + Path("bot-config/zuliprc"), 21 + "--config", 22 + "-c", 23 + help="Zulip bot configuration file", 24 + ), 25 + thicket_config: Path = typer.Option( 26 + Path("thicket.yaml"), 27 + "--thicket-config", 28 + help="Path to thicket configuration file", 29 + ), 30 + daemon: bool = typer.Option( 31 + False, 32 + "--daemon", 33 + "-d", 34 + help="Run bot in daemon mode (background)", 35 + ), 36 + debug_user: str = typer.Option( 37 + None, 38 + "--debug-user", 39 + help="Debug mode: send DMs to this thicket username instead of posting to streams", 40 + ), 41 + ) -> None: 42 + """Manage the Thicket Zulip bot. 43 + 44 + Actions: 45 + - run: Start the Zulip bot 46 + - test: Test bot functionality 47 + - status: Show bot status 48 + """ 49 + 50 + if action == "run": 51 + _run_bot(config_file, thicket_config, daemon, debug_user) 52 + elif action == "test": 53 + _test_bot() 54 + elif action == "status": 55 + _bot_status(config_file) 56 + else: 57 + print_error(f"Unknown action: {action}") 58 + print_info("Available actions: run, test, status") 59 + raise typer.Exit(1) 60 + 61 + 62 + def _run_bot(config_file: Path, thicket_config: Path, daemon: bool, debug_user: str = None) -> None: 63 + """Run the Zulip bot.""" 64 + if not config_file.exists(): 65 + print_error(f"Configuration file not found: {config_file}") 66 + print_info(f"Copy bot-config/zuliprc.template to {config_file} and configure it") 67 + raise typer.Exit(1) 68 + 69 + if not thicket_config.exists(): 70 + print_error(f"Thicket configuration file not found: {thicket_config}") 71 + print_info("Run `thicket init` to create a thicket.yaml file") 72 + raise typer.Exit(1) 73 + 74 + # Parse zuliprc to extract server URL 75 + zulip_site_url = _parse_zulip_config(config_file) 76 + 77 + print_info(f"Starting Thicket Zulip bot with config: {config_file}") 78 + print_info(f"Using thicket config: {thicket_config}") 79 + 80 + if debug_user: 81 + print_info(f"🐛 DEBUG MODE: Will send DMs to thicket user '{debug_user}' instead of posting to streams") 82 + 83 + if daemon: 84 + print_info("Running in daemon mode...") 85 + else: 86 + print_info("Bot will be available as @thicket in your Zulip chat") 87 + print_info("Press Ctrl+C to stop the bot") 88 + 89 + try: 90 + # Build the command 91 + cmd = [ 92 + sys.executable, "-m", "zulip_bots.run", 93 + "src/thicket/bots/thicket_bot.py", 94 + "--config-file", str(config_file) 95 + ] 96 + 97 + # Add environment variables for bot configuration 98 + import os 99 + env = os.environ.copy() 100 + 101 + # Always pass thicket config path 102 + env["THICKET_CONFIG_PATH"] = str(thicket_config.absolute()) 103 + 104 + # Add debug user if specified 105 + if debug_user: 106 + env["THICKET_DEBUG_USER"] = debug_user 107 + 108 + # Pass Zulip server URL to bot 109 + if zulip_site_url: 110 + env["THICKET_ZULIP_SITE_URL"] = zulip_site_url 111 + 112 + if daemon: 113 + # Run in background 114 + process = subprocess.Popen( 115 + cmd, 116 + stdout=subprocess.DEVNULL, 117 + stderr=subprocess.DEVNULL, 118 + start_new_session=True, 119 + env=env 120 + ) 121 + print_success(f"Bot started in background with PID {process.pid}") 122 + else: 123 + # Run in foreground 124 + subprocess.run(cmd, check=True, env=env) 125 + 126 + except subprocess.CalledProcessError as e: 127 + print_error(f"Failed to start bot: {e}") 128 + raise typer.Exit(1) 129 + except KeyboardInterrupt: 130 + print_info("Bot stopped by user") 131 + 132 + 133 + def _parse_zulip_config(config_file: Path) -> str: 134 + """Parse zuliprc file to extract the site URL.""" 135 + try: 136 + import configparser 137 + 138 + config = configparser.ConfigParser() 139 + config.read(config_file) 140 + 141 + if "api" in config and "site" in config["api"]: 142 + site_url = config["api"]["site"] 143 + print_info(f"Detected Zulip server: {site_url}") 144 + return site_url 145 + else: 146 + print_error("Could not find 'site' in zuliprc [api] section") 147 + return "" 148 + 149 + except Exception as e: 150 + print_error(f"Error parsing zuliprc: {e}") 151 + return "" 152 + 153 + 154 + def _test_bot() -> None: 155 + """Test bot functionality.""" 156 + print_info("Testing Thicket Zulip bot...") 157 + 158 + try: 159 + from ...bots.test_bot import BotTester 160 + 161 + # Create bot tester 162 + tester = BotTester() 163 + 164 + # Test basic functionality 165 + console.print("✓ Testing help command...", style="green") 166 + responses = tester.send_command("help") 167 + assert len(responses) == 1 168 + assert "Thicket Feed Bot" in tester.get_last_response_content() 169 + 170 + console.print("✓ Testing status command...", style="green") 171 + responses = tester.send_command("status") 172 + assert len(responses) == 1 173 + assert "Status" in tester.get_last_response_content() 174 + 175 + console.print("✓ Testing config commands...", style="green") 176 + responses = tester.send_command("config stream test-stream") 177 + tester.assert_response_contains("Stream set to") 178 + 179 + responses = tester.send_command("config topic test-topic") 180 + tester.assert_response_contains("Topic set to") 181 + 182 + responses = tester.send_command("config interval 300") 183 + tester.assert_response_contains("Sync interval set to") 184 + 185 + print_success("All bot tests passed!") 186 + 187 + except Exception as e: 188 + print_error(f"Bot test failed: {e}") 189 + raise typer.Exit(1) 190 + 191 + 192 + def _bot_status(config_file: Path) -> None: 193 + """Show bot status.""" 194 + console.print("Thicket Zulip Bot Status", style="bold blue") 195 + console.print() 196 + 197 + # Check config file 198 + if config_file.exists(): 199 + console.print(f"✓ Config file: {config_file}", style="green") 200 + else: 201 + console.print(f"✗ Config file not found: {config_file}", style="red") 202 + console.print(" Copy bot-config/zuliprc.template and configure it", style="yellow") 203 + 204 + # Check dependencies 205 + try: 206 + import zulip_bots 207 + version = getattr(zulip_bots, '__version__', 'unknown') 208 + console.print(f"✓ zulip-bots version: {version}", style="green") 209 + except ImportError: 210 + console.print("✗ zulip-bots not installed", style="red") 211 + 212 + try: 213 + from ...bots.thicket_bot import ThicketBotHandler 214 + console.print("✓ ThicketBotHandler available", style="green") 215 + except ImportError as e: 216 + console.print(f"✗ Bot handler not available: {e}", style="red") 217 + 218 + # Check bot file 219 + bot_file = Path("src/thicket/bots/thicket_bot.py") 220 + if bot_file.exists(): 221 + console.print(f"✓ Bot file: {bot_file}", style="green") 222 + else: 223 + console.print(f"✗ Bot file not found: {bot_file}", style="red") 224 + 225 + console.print() 226 + console.print("To run the bot:", style="bold") 227 + console.print(f" thicket bot run --config {config_file}") 228 + console.print() 229 + console.print("For help setting up the bot, see: docs/ZULIP_BOT.md", style="dim")
+258
src/thicket/cli/commands/zulip.py
··· 1 + """Zulip association management commands for thicket.""" 2 + 3 + from pathlib import Path 4 + from typing import Optional 5 + 6 + import typer 7 + from rich.console import Console 8 + from rich.table import Table 9 + 10 + from ...core.git_store import GitStore 11 + from ..main import app 12 + from ..utils import load_config, print_error, print_info, print_success 13 + 14 + console = Console() 15 + 16 + 17 + @app.command() 18 + def zulip_add( 19 + username: str = typer.Argument(..., help="Username to associate with Zulip"), 20 + server: str = typer.Argument(..., help="Zulip server (e.g., yourorg.zulipchat.com)"), 21 + user_id: str = typer.Argument(..., help="Zulip user ID or email for @mentions"), 22 + config_file: Path = typer.Option( 23 + Path("thicket.yaml"), 24 + "--config", 25 + "-c", 26 + help="Path to thicket configuration file", 27 + ), 28 + ) -> None: 29 + """Add a Zulip association for a user. 30 + 31 + This associates a thicket user with their Zulip identity, enabling 32 + @mentions when the bot posts their articles. 33 + 34 + Example: 35 + thicket zulip-add alice myorg.zulipchat.com alice@example.com 36 + """ 37 + try: 38 + config = load_config(config_file) 39 + git_store = GitStore(config.git_store) 40 + 41 + # Check if user exists 42 + user = git_store.get_user(username) 43 + if not user: 44 + print_error(f"User '{username}' not found") 45 + raise typer.Exit(1) 46 + 47 + # Add association 48 + if git_store.add_zulip_association(username, server, user_id): 49 + print_success(f"Added Zulip association for {username}: {user_id}@{server}") 50 + git_store.commit_changes(f"Add Zulip association for {username}") 51 + else: 52 + print_info(f"Association already exists for {username}: {user_id}@{server}") 53 + 54 + except Exception as e: 55 + print_error(f"Failed to add Zulip association: {e}") 56 + raise typer.Exit(1) 57 + 58 + 59 + @app.command() 60 + def zulip_remove( 61 + username: str = typer.Argument(..., help="Username to remove association from"), 62 + server: str = typer.Argument(..., help="Zulip server"), 63 + user_id: str = typer.Argument(..., help="Zulip user ID or email"), 64 + config_file: Path = typer.Option( 65 + Path("thicket.yaml"), 66 + "--config", 67 + "-c", 68 + help="Path to thicket configuration file", 69 + ), 70 + ) -> None: 71 + """Remove a Zulip association from a user. 72 + 73 + Example: 74 + thicket zulip-remove alice myorg.zulipchat.com alice@example.com 75 + """ 76 + try: 77 + config = load_config(config_file) 78 + git_store = GitStore(config.git_store) 79 + 80 + # Check if user exists 81 + user = git_store.get_user(username) 82 + if not user: 83 + print_error(f"User '{username}' not found") 84 + raise typer.Exit(1) 85 + 86 + # Remove association 87 + if git_store.remove_zulip_association(username, server, user_id): 88 + print_success(f"Removed Zulip association for {username}: {user_id}@{server}") 89 + git_store.commit_changes(f"Remove Zulip association for {username}") 90 + else: 91 + print_error(f"Association not found for {username}: {user_id}@{server}") 92 + raise typer.Exit(1) 93 + 94 + except Exception as e: 95 + print_error(f"Failed to remove Zulip association: {e}") 96 + raise typer.Exit(1) 97 + 98 + 99 + @app.command() 100 + def zulip_list( 101 + username: Optional[str] = typer.Argument(None, help="Username to list associations for"), 102 + config_file: Path = typer.Option( 103 + Path("thicket.yaml"), 104 + "--config", 105 + "-c", 106 + help="Path to thicket configuration file", 107 + ), 108 + ) -> None: 109 + """List Zulip associations for users. 110 + 111 + If no username is provided, lists associations for all users. 112 + 113 + Examples: 114 + thicket zulip-list # List all associations 115 + thicket zulip-list alice # List associations for alice 116 + """ 117 + try: 118 + config = load_config(config_file) 119 + git_store = GitStore(config.git_store) 120 + 121 + # Create table 122 + table = Table(title="Zulip Associations") 123 + table.add_column("Username", style="cyan") 124 + table.add_column("Server", style="green") 125 + table.add_column("User ID", style="yellow") 126 + 127 + if username: 128 + # List for specific user 129 + user = git_store.get_user(username) 130 + if not user: 131 + print_error(f"User '{username}' not found") 132 + raise typer.Exit(1) 133 + 134 + if not user.zulip_associations: 135 + print_info(f"No Zulip associations for {username}") 136 + return 137 + 138 + for assoc in user.zulip_associations: 139 + table.add_row(username, assoc.server, assoc.user_id) 140 + else: 141 + # List for all users 142 + index = git_store._load_index() 143 + has_associations = False 144 + 145 + for username, user in index.users.items(): 146 + for assoc in user.zulip_associations: 147 + table.add_row(username, assoc.server, assoc.user_id) 148 + has_associations = True 149 + 150 + if not has_associations: 151 + print_info("No Zulip associations found") 152 + return 153 + 154 + console.print(table) 155 + 156 + except Exception as e: 157 + print_error(f"Failed to list Zulip associations: {e}") 158 + raise typer.Exit(1) 159 + 160 + 161 + @app.command() 162 + def zulip_import( 163 + csv_file: Path = typer.Argument(..., help="CSV file with username,server,user_id"), 164 + config_file: Path = typer.Option( 165 + Path("thicket.yaml"), 166 + "--config", 167 + "-c", 168 + help="Path to thicket configuration file", 169 + ), 170 + dry_run: bool = typer.Option( 171 + False, 172 + "--dry-run", 173 + help="Show what would be imported without making changes", 174 + ), 175 + ) -> None: 176 + """Import Zulip associations from a CSV file. 177 + 178 + CSV format (no header): 179 + username,server,user_id 180 + alice,myorg.zulipchat.com,alice@example.com 181 + bob,myorg.zulipchat.com,bob.smith 182 + 183 + Example: 184 + thicket zulip-import associations.csv 185 + """ 186 + import csv 187 + 188 + try: 189 + config = load_config(config_file) 190 + git_store = GitStore(config.git_store) 191 + 192 + if not csv_file.exists(): 193 + print_error(f"CSV file not found: {csv_file}") 194 + raise typer.Exit(1) 195 + 196 + added = 0 197 + skipped = 0 198 + errors = 0 199 + 200 + with open(csv_file, 'r') as f: 201 + reader = csv.reader(f) 202 + for row_num, row in enumerate(reader, 1): 203 + if len(row) != 3: 204 + print_error(f"Line {row_num}: Invalid format (expected 3 columns)") 205 + errors += 1 206 + continue 207 + 208 + username, server, user_id = [col.strip() for col in row] 209 + 210 + # Skip empty lines 211 + if not username: 212 + continue 213 + 214 + # Check if user exists 215 + user = git_store.get_user(username) 216 + if not user: 217 + print_error(f"Line {row_num}: User '{username}' not found") 218 + errors += 1 219 + continue 220 + 221 + if dry_run: 222 + # Check if association would be added 223 + exists = any( 224 + a.server == server and a.user_id == user_id 225 + for a in user.zulip_associations 226 + ) 227 + if exists: 228 + print_info(f"Would skip existing: {username} -> {user_id}@{server}") 229 + skipped += 1 230 + else: 231 + print_info(f"Would add: {username} -> {user_id}@{server}") 232 + added += 1 233 + else: 234 + # Actually add association 235 + if git_store.add_zulip_association(username, server, user_id): 236 + print_success(f"Added: {username} -> {user_id}@{server}") 237 + added += 1 238 + else: 239 + print_info(f"Skipped existing: {username} -> {user_id}@{server}") 240 + skipped += 1 241 + 242 + # Summary 243 + console.print() 244 + if dry_run: 245 + console.print(f"[bold]Dry run summary:[/bold]") 246 + console.print(f" Would add: {added}") 247 + else: 248 + console.print(f"[bold]Import summary:[/bold]") 249 + console.print(f" Added: {added}") 250 + if not dry_run and added > 0: 251 + git_store.commit_changes(f"Import {added} Zulip associations from CSV") 252 + 253 + console.print(f" Skipped: {skipped}") 254 + console.print(f" Errors: {errors}") 255 + 256 + except Exception as e: 257 + print_error(f"Failed to import Zulip associations: {e}") 258 + raise typer.Exit(1)
+297
tests/test_bot.py
··· 1 + """Tests for the Thicket Zulip bot.""" 2 + 3 + import json 4 + import tempfile 5 + from pathlib import Path 6 + from unittest.mock import patch 7 + 8 + import pytest 9 + 10 + from thicket.bots.test_bot import BotTester, MockBotHandler, create_test_message, create_test_entry 11 + from thicket.bots.thicket_bot import ThicketBotHandler 12 + 13 + 14 + class TestThicketBot: 15 + """Test suite for ThicketBotHandler.""" 16 + 17 + def setup_method(self) -> None: 18 + """Set up test environment.""" 19 + self.bot = ThicketBotHandler() 20 + self.handler = MockBotHandler() 21 + 22 + def test_usage(self) -> None: 23 + """Test bot usage message.""" 24 + usage = self.bot.usage() 25 + assert "Thicket Feed Bot" in usage 26 + assert "@thicket status" in usage 27 + assert "@thicket config" in usage 28 + 29 + def test_help_command(self) -> None: 30 + """Test help command response.""" 31 + message = create_test_message("@thicket help") 32 + self.bot.handle_message(message, self.handler) 33 + 34 + assert len(self.handler.sent_messages) == 1 35 + response = self.handler.sent_messages[0]["content"] 36 + assert "Thicket Feed Bot" in response 37 + 38 + def test_status_command_unconfigured(self) -> None: 39 + """Test status command when bot is not configured.""" 40 + message = create_test_message("@thicket status") 41 + self.bot.handle_message(message, self.handler) 42 + 43 + assert len(self.handler.sent_messages) == 1 44 + response = self.handler.sent_messages[0]["content"] 45 + assert "Not configured" in response 46 + assert "Stream:" in response 47 + assert "Topic:" in response 48 + 49 + def test_config_stream_command(self) -> None: 50 + """Test setting stream configuration.""" 51 + message = create_test_message("@thicket config stream general") 52 + self.bot.handle_message(message, self.handler) 53 + 54 + assert len(self.handler.sent_messages) == 1 55 + response = self.handler.sent_messages[0]["content"] 56 + assert "Stream set to: **general**" in response 57 + assert self.bot.stream_name == "general" 58 + 59 + def test_config_topic_command(self) -> None: 60 + """Test setting topic configuration.""" 61 + message = create_test_message("@thicket config topic 'Feed Updates'") 62 + self.bot.handle_message(message, self.handler) 63 + 64 + assert len(self.handler.sent_messages) == 1 65 + response = self.handler.sent_messages[0]["content"] 66 + assert "Topic set to:" in response and "Feed Updates" in response 67 + assert self.bot.topic_name == "'Feed Updates'" 68 + 69 + def test_config_interval_command(self) -> None: 70 + """Test setting sync interval.""" 71 + message = create_test_message("@thicket config interval 600") 72 + self.bot.handle_message(message, self.handler) 73 + 74 + assert len(self.handler.sent_messages) == 1 75 + response = self.handler.sent_messages[0]["content"] 76 + assert "Sync interval set to: **600s**" in response 77 + assert self.bot.sync_interval == 600 78 + 79 + def test_config_interval_too_small(self) -> None: 80 + """Test setting sync interval that's too small.""" 81 + message = create_test_message("@thicket config interval 30") 82 + self.bot.handle_message(message, self.handler) 83 + 84 + assert len(self.handler.sent_messages) == 1 85 + response = self.handler.sent_messages[0]["content"] 86 + assert "must be at least 60 seconds" in response 87 + assert self.bot.sync_interval != 30 88 + 89 + def test_config_path_nonexistent(self) -> None: 90 + """Test setting config path that doesn't exist.""" 91 + message = create_test_message("@thicket config path /nonexistent/config.yaml") 92 + self.bot.handle_message(message, self.handler) 93 + 94 + assert len(self.handler.sent_messages) == 1 95 + response = self.handler.sent_messages[0]["content"] 96 + assert "Config file not found" in response 97 + 98 + def test_unknown_command(self) -> None: 99 + """Test unknown command handling.""" 100 + message = create_test_message("@thicket unknown") 101 + self.bot.handle_message(message, self.handler) 102 + 103 + assert len(self.handler.sent_messages) == 1 104 + response = self.handler.sent_messages[0]["content"] 105 + assert "Unknown command: unknown" in response 106 + 107 + def test_config_persistence(self) -> None: 108 + """Test that configuration is persisted.""" 109 + # Set some config 110 + self.bot.stream_name = "test-stream" 111 + self.bot.topic_name = "test-topic" 112 + self.bot.sync_interval = 600 113 + 114 + # Save config 115 + self.bot._save_bot_config(self.handler) 116 + 117 + # Create new bot instance 118 + new_bot = ThicketBotHandler() 119 + new_bot._load_bot_config(self.handler) 120 + 121 + # Check config was loaded 122 + assert new_bot.stream_name == "test-stream" 123 + assert new_bot.topic_name == "test-topic" 124 + assert new_bot.sync_interval == 600 125 + 126 + def test_posted_entries_persistence(self) -> None: 127 + """Test that posted entries are persisted.""" 128 + # Add some entries 129 + self.bot.posted_entries = {"user1:entry1", "user2:entry2"} 130 + 131 + # Save entries 132 + self.bot._save_posted_entries(self.handler) 133 + 134 + # Create new bot instance 135 + new_bot = ThicketBotHandler() 136 + new_bot._load_posted_entries(self.handler) 137 + 138 + # Check entries were loaded 139 + assert new_bot.posted_entries == {"user1:entry1", "user2:entry2"} 140 + 141 + def test_mention_detection(self) -> None: 142 + """Test bot mention detection.""" 143 + assert self.bot._is_mentioned("@Thicket Bot help", self.handler) 144 + assert self.bot._is_mentioned("@thicket status", self.handler) 145 + assert not self.bot._is_mentioned("regular message", self.handler) 146 + 147 + def test_mention_cleaning(self) -> None: 148 + """Test cleaning mentions from messages.""" 149 + cleaned = self.bot._clean_mention("@Thicket Bot status", self.handler) 150 + assert cleaned == "status" 151 + 152 + cleaned = self.bot._clean_mention("@thicket help", self.handler) 153 + assert cleaned == "help" 154 + 155 + def test_sync_now_uninitialized(self) -> None: 156 + """Test sync now command when not initialized.""" 157 + message = create_test_message("@thicket sync now") 158 + self.bot.handle_message(message, self.handler) 159 + 160 + assert len(self.handler.sent_messages) == 1 161 + response = self.handler.sent_messages[0]["content"] 162 + assert "not initialized" in response.lower() 163 + 164 + def test_debug_mode_initialization(self) -> None: 165 + """Test debug mode initialization.""" 166 + import os 167 + 168 + # Mock environment variable 169 + os.environ["THICKET_DEBUG_USER"] = "testuser" 170 + 171 + try: 172 + bot = ThicketBotHandler() 173 + # Simulate initialize call 174 + bot.debug_user = os.getenv("THICKET_DEBUG_USER") 175 + 176 + assert bot.debug_user == "testuser" 177 + assert bot.debug_zulip_user_id is None # Not validated yet 178 + finally: 179 + # Clean up 180 + if "THICKET_DEBUG_USER" in os.environ: 181 + del os.environ["THICKET_DEBUG_USER"] 182 + 183 + def test_debug_mode_status(self) -> None: 184 + """Test status command in debug mode.""" 185 + self.bot.debug_user = "testuser" 186 + self.bot.debug_zulip_user_id = "test.user" 187 + 188 + message = create_test_message("@thicket status") 189 + self.bot.handle_message(message, self.handler) 190 + 191 + assert len(self.handler.sent_messages) == 1 192 + response = self.handler.sent_messages[0]["content"] 193 + assert "**Debug Mode:** ENABLED" in response 194 + assert "**Debug User:** testuser" in response 195 + assert "**Debug Zulip ID:** test.user" in response 196 + 197 + def test_debug_mode_check_initialization(self) -> None: 198 + """Test initialization check in debug mode.""" 199 + from unittest.mock import Mock 200 + 201 + # Setup mock git store and config 202 + self.bot.git_store = Mock() 203 + self.bot.config = Mock() 204 + self.bot.debug_user = "testuser" 205 + self.bot.debug_zulip_user_id = "test.user" 206 + 207 + message = create_test_message("@thicket sync now") 208 + 209 + # Should pass with debug mode properly set up 210 + result = self.bot._check_initialization(message, self.handler) 211 + assert result is True 212 + 213 + # Should fail if debug_zulip_user_id is missing 214 + self.bot.debug_zulip_user_id = None 215 + result = self.bot._check_initialization(message, self.handler) 216 + assert result is False 217 + assert len(self.handler.sent_messages) == 1 218 + assert "Debug mode validation failed" in self.handler.sent_messages[0]["content"] 219 + 220 + def test_debug_mode_dm_posting(self) -> None: 221 + """Test that debug mode posts DMs instead of stream messages.""" 222 + from unittest.mock import Mock 223 + from datetime import datetime 224 + from pydantic import HttpUrl 225 + 226 + # Setup bot in debug mode 227 + self.bot.debug_user = "testuser" 228 + self.bot.debug_zulip_user_id = "test.user@example.com" 229 + self.bot.git_store = Mock() 230 + 231 + # Create a test entry 232 + entry = create_test_entry() 233 + 234 + # Mock the handler config 235 + self.handler.config_info = { 236 + "full_name": "Thicket Bot", 237 + "email": "thicket-bot@example.com", 238 + "site": "https://example.zulipchat.com" 239 + } 240 + 241 + # Mock git store user 242 + mock_user = Mock() 243 + mock_user.get_zulip_mention.return_value = "author.user" 244 + self.bot.git_store.get_user.return_value = mock_user 245 + 246 + # Post entry 247 + self.bot._post_entry_to_zulip(entry, self.handler, "testauthor") 248 + 249 + # Check that a DM was sent 250 + assert len(self.handler.sent_messages) == 1 251 + message = self.handler.sent_messages[0] 252 + 253 + # Verify it's a DM 254 + assert message["type"] == "private" 255 + assert message["to"] == ["test.user@example.com"] 256 + assert "DEBUG:" in message["content"] 257 + assert entry.title in message["content"] 258 + assert "@**author.user** posted:" in message["content"] 259 + 260 + 261 + class TestBotTester: 262 + """Test the bot testing utilities.""" 263 + 264 + def test_bot_tester_basic(self) -> None: 265 + """Test basic bot tester functionality.""" 266 + tester = BotTester() 267 + 268 + # Test help command 269 + responses = tester.send_command("help") 270 + assert len(responses) == 1 271 + assert "Thicket Feed Bot" in tester.get_last_response_content() 272 + 273 + def test_bot_tester_config(self) -> None: 274 + """Test bot tester configuration.""" 275 + tester = BotTester() 276 + 277 + # Configure stream 278 + responses = tester.send_command("config stream general") 279 + tester.assert_response_contains("Stream set to") 280 + 281 + # Configure topic 282 + responses = tester.send_command("config topic test") 283 + tester.assert_response_contains("Topic set to") 284 + 285 + def test_assert_response_contains(self) -> None: 286 + """Test response assertion helper.""" 287 + tester = BotTester() 288 + 289 + # Send command 290 + tester.send_command("help") 291 + 292 + # This should pass 293 + tester.assert_response_contains("Thicket Feed Bot") 294 + 295 + # This should fail 296 + with pytest.raises(AssertionError): 297 + tester.assert_response_contains("nonexistent text")