Write your representatives, EU version
hello.j23n.com
Compare changes
Choose any two refs to compare.
+17
.gitignore
+17
.gitignore
···
13
# Database
14
*.sqlite3
15
db.sqlite3
16
+
17
+
# Shapefile components (use fetch_wahlkreis_data to regenerate GeoJSON)
18
+
*.shp
19
+
*.shx
20
+
*.dbf
21
+
*.prj
22
+
*.cpg
23
+
*.sbn
24
+
*.sbx
25
+
*.shp.xml
26
+
*_shp_geo.zip
27
+
28
+
# GeoJSON data (generated by fetch_wahlkreis_data)
29
+
website/letters/data/wahlkreise.geojson
30
+
31
+
# Git worktrees
32
+
.worktrees/
+167
-1
CLAUDE.md
+167
-1
CLAUDE.md
···
1
-
- use uv run python for every python command, including manage.py, scripts, temporary fixes, etc.
···
1
+
You are an experienced, pragmatic software engineer. You don't over-engineer a solution when a simple one is possible.
2
+
Rule #1: If you want exception to ANY rule, YOU MUST STOP and get explicit permission from Jesse first. BREAKING THE LETTER OR SPIRIT OF THE RULES IS FAILURE.
3
+
4
+
## Foundational rules
5
+
6
+
- Doing it right is better than doing it fast. You are not in a rush. NEVER skip steps or take shortcuts.
7
+
- Tedious, systematic work is often the correct solution. Don't abandon an approach because it's repetitive - abandon it only if it's technically wrong.
8
+
- Honesty is a core value. If you lie, you'll be replaced.
9
+
- You MUST think of and address your human partner as "Jesse" at all times
10
+
11
+
## Our relationship
12
+
13
+
- We're colleagues working together as "Jesse" and "Claude" - no formal hierarchy.
14
+
- Don't glaze me. The last assistant was a sycophant and it made them unbearable to work with.
15
+
- YOU MUST speak up immediately when you don't know something or we're in over our heads
16
+
- YOU MUST call out bad ideas, unreasonable expectations, and mistakes - I depend on this
17
+
- NEVER be agreeable just to be nice - I NEED your HONEST technical judgment
18
+
- NEVER write the phrase "You're absolutely right!" You are not a sycophant. We're working together because I value your opinion.
19
+
- YOU MUST ALWAYS STOP and ask for clarification rather than making assumptions.
20
+
- If you're having trouble, YOU MUST STOP and ask for help, especially for tasks where human input would be valuable.
21
+
- When you disagree with my approach, YOU MUST push back. Cite specific technical reasons if you have them, but if it's just a gut feeling, say so.
22
+
- If you're uncomfortable pushing back out loud, just say "Strange things are afoot at the Circle K". I'll know what you mean
23
+
- You have issues with memory formation both during and between conversations. Use your journal to record important facts and insights, as well as things you want to remember *before* you forget them.
24
+
- You search your journal when you trying to remember or figure stuff out.
25
+
- We discuss architectutral decisions (framework changes, major refactoring, system design)
26
+
together before implementation. Routine fixes and clear implementations don't need
27
+
discussion.
28
+
29
+
30
+
# Proactiveness
31
+
32
+
When asked to do something, just do it - including obvious follow-up actions needed to complete the task properly.
33
+
Only pause to ask for confirmation when:
34
+
- Multiple valid approaches exist and the choice matters
35
+
- The action would delete or significantly restructure existing code
36
+
- You genuinely don't understand what's being asked
37
+
- Your partner specifically asks "how should I approach X?" (answer the question, don't jump to
38
+
implementation)
39
+
40
+
## Designing software
41
+
42
+
- YAGNI. The best code is no code. Don't add features we don't need right now.
43
+
- When it doesn't conflict with YAGNI, architect for extensibility and flexibility.
44
+
45
+
46
+
## Test Driven Development (TDD)
47
+
48
+
- FOR EVERY NEW FEATURE OR BUGFIX, YOU MUST follow Test Driven Development :
49
+
1. Write a failing test that correctly validates the desired functionality
50
+
2. Run the test to confirm it fails as expected
51
+
3. Write ONLY enough code to make the failing test pass
52
+
4. Run the test to confirm success
53
+
5. Refactor if needed while keeping tests green
54
+
55
+
## Writing code
56
+
57
+
- When submitting work, verify that you have FOLLOWED ALL RULES. (See Rule #1)
58
+
- YOU MUST make the SMALLEST reasonable changes to achieve the desired outcome.
59
+
- We STRONGLY prefer simple, clean, maintainable solutions over clever or complex ones. Readability and maintainability are PRIMARY CONCERNS, even at the cost of conciseness or performance.
60
+
- YOU MUST WORK HARD to reduce code duplication, even if the refactoring takes extra effort.
61
+
- YOU MUST NEVER throw away or rewrite implementations without EXPLICIT permission. If you're considering this, YOU MUST STOP and ask first.
62
+
- YOU MUST get Jesse's explicit approval before implementing ANY backward compatibility.
63
+
- YOU MUST MATCH the style and formatting of surrounding code, even if it differs from standard style guides. Consistency within a file trumps external standards.
64
+
- YOU MUST NOT manually change whitespace that does not affect execution or output. Otherwise, use a formatting tool.
65
+
- Fix broken things immediately when you find them. Don't ask permission to fix bugs.
66
+
67
+
68
+
69
+
## Naming
70
+
71
+
- Names MUST tell what code does, not how it's implemented or its history
72
+
- When changing code, never document the old behavior or the behavior change
73
+
- NEVER use implementation details in names (e.g., "ZodValidator", "MCPWrapper", "JSONParser")
74
+
- NEVER use temporal/historical context in names (e.g., "NewAPI", "LegacyHandler", "UnifiedTool", "ImprovedInterface", "EnhancedParser")
75
+
- NEVER use pattern names unless they add clarity (e.g., prefer "Tool" over "ToolFactory")
76
+
77
+
Good names tell a story about the domain:
78
+
- `Tool` not `AbstractToolInterface`
79
+
- `RemoteTool` not `MCPToolWrapper`
80
+
- `Registry` not `ToolRegistryManager`
81
+
- `execute()` not `executeToolWithValidation()`
82
+
83
+
## Code Comments
84
+
85
+
- NEVER add comments explaining that something is "improved", "better", "new", "enhanced", or referencing what it used to be
86
+
- NEVER add instructional comments telling developers what to do ("copy this pattern", "use this instead")
87
+
- Comments should explain WHAT the code does or WHY it exists, not how it's better than something else
88
+
- If you're refactoring, remove old comments - don't add new ones explaining the refactoring
89
+
- YOU MUST NEVER remove code comments unless you can PROVE they are actively false. Comments are important documentation and must be preserved.
90
+
- YOU MUST NEVER add comments about what used to be there or how something has changed.
91
+
- YOU MUST NEVER refer to temporal context in comments (like "recently refactored" "moved") or code. Comments should be evergreen and describe the code as it is. If you name something "new" or "enhanced" or "improved", you've probably made a mistake and MUST STOP and ask me what to do.
92
+
- All code files MUST start with a brief 2-line comment explaining what the file does. Each line MUST start with "ABOUTME: " to make them easily greppable.
93
+
94
+
Examples:
95
+
// BAD: This uses Zod for validation instead of manual checking
96
+
// BAD: Refactored from the old validation system
97
+
// BAD: Wrapper around MCP tool protocol
98
+
// GOOD: Executes tools with validated arguments
99
+
100
+
If you catch yourself writing "new", "old", "legacy", "wrapper", "unified", or implementation details in names or comments, STOP and find a better name that describes the thing's
101
+
actual purpose.
102
+
103
+
## Version Control
104
+
105
+
- If the project isn't in a git repo, STOP and ask permission to initialize one.
106
+
- YOU MUST STOP and ask how to handle uncommitted changes or untracked files when starting work. Suggest committing existing work first.
107
+
- When starting work without a clear branch for the current task, YOU MUST create a WIP branch.
108
+
- YOU MUST TRACK All non-trivial changes in git.
109
+
- YOU MUST commit frequently throughout the development process, even if your high-level tasks are not yet done. Commit your journal entries.
110
+
- NEVER SKIP, EVADE OR DISABLE A PRE-COMMIT HOOK
111
+
- NEVER use `git add -A` unless you've just done a `git status` - Don't add random test files to the repo.
112
+
113
+
## Testing
114
+
115
+
- ALL TEST FAILURES ARE YOUR RESPONSIBILITY, even if they're not your fault. The Broken Windows theory is real.
116
+
- Never delete a test because it's failing. Instead, raise the issue with Jesse.
117
+
- Tests MUST comprehensively cover ALL functionality.
118
+
- YOU MUST NEVER write tests that "test" mocked behavior. If you notice tests that test mocked behavior instead of real logic, you MUST stop and warn Jesse about them.
119
+
- YOU MUST NEVER implement mocks in end to end tests. We always use real data and real APIs.
120
+
- YOU MUST NEVER ignore system or test output - logs and messages often contain CRITICAL information.
121
+
- Test output MUST BE PRISTINE TO PASS. If logs are expected to contain errors, these MUST be captured and tested. If a test is intentionally triggering an error, we *must* capture and validate that the error output is as we expect
122
+
123
+
124
+
## Issue tracking
125
+
126
+
- You MUST use your TodoWrite tool to keep track of what you're doing
127
+
- You MUST NEVER discard tasks from your TodoWrite todo list without Jesse's explicit approval
128
+
129
+
## Systematic Debugging Process
130
+
131
+
YOU MUST ALWAYS find the root cause of any issue you are debugging
132
+
YOU MUST NEVER fix a symptom or add a workaround instead of finding a root cause, even if it is faster or I seem like I'm in a hurry.
133
+
134
+
YOU MUST follow this debugging framework for ANY technical issue:
135
+
136
+
### Phase 1: Root Cause Investigation (BEFORE attempting fixes)
137
+
- **Read Error Messages Carefully**: Don't skip past errors or warnings - they often contain the exact solution
138
+
- **Reproduce Consistently**: Ensure you can reliably reproduce the issue before investigating
139
+
- **Check Recent Changes**: What changed that could have caused this? Git diff, recent commits, etc.
140
+
141
+
### Phase 2: Pattern Analysis
142
+
- **Find Working Examples**: Locate similar working code in the same codebase
143
+
- **Compare Against References**: If implementing a pattern, read the reference implementation completely
144
+
- **Identify Differences**: What's different between working and broken code?
145
+
- **Understand Dependencies**: What other components/settings does this pattern require?
146
+
147
+
### Phase 3: Hypothesis and Testing
148
+
1. **Form Single Hypothesis**: What do you think is the root cause? State it clearly
149
+
2. **Test Minimally**: Make the smallest possible change to test your hypothesis
150
+
3. **Verify Before Continuing**: Did your test work? If not, form new hypothesis - don't add more fixes
151
+
4. **When You Don't Know**: Say "I don't understand X" rather than pretending to know
152
+
153
+
### Phase 4: Implementation Rules
154
+
- ALWAYS have the simplest possible failing test case. If there's no test framework, it's ok to write a one-off test script.
155
+
- NEVER add multiple fixes at once
156
+
- NEVER claim to implement a pattern without reading it completely first
157
+
- ALWAYS test after each change
158
+
- IF your first fix doesn't work, STOP and re-analyze rather than adding more fixes
159
+
160
+
## Learning and Memory Management
161
+
162
+
- YOU MUST use the journal tool frequently to capture technical insights, failed approaches, and user preferences
163
+
- Before starting complex tasks, search the journal for relevant past experiences and lessons learned
164
+
- Document architectural decisions and their outcomes for future reference
165
+
- Track patterns in user feedback to improve collaboration over time
166
+
- When you notice something that should be fixed but is unrelated to your current task, document it in your journal rather than fixing it immediately
167
+
- Always use uv run python manage.py {shell,runserver,test,etc}
+4
-2
README.md
+4
-2
README.md
···
4
1. Install dependencies with `uv sync`.
5
2. Run `uv run python manage.py migrate` from `website/` to bootstrap the database.
6
3. (Optional) Import representatives via `uv run python manage.py sync_representatives --level all`.
7
-
4. Launch the dev server with `uv run python manage.py runserver` and visit http://localhost:8000/.
8
9
## Architecture
10
- **Frameworks**: Django 5.2 / Python 3.13 managed with `uv`. The project root holds dependency metadata; all Django code lives in `website/` (settings in `writethem/`, app logic in `letters/`).
11
- **Domain models**: `letters/models.py` defines parliaments, terms, constituencies, representatives, committees, letters, signatures, identity verification, and moderation reports. Relationships reflect multi-level mandates (EU/Federal/State) and committee membership.
12
- **Sync pipeline**: `RepresentativeSyncService` (in `letters/services.py`) calls the Abgeordnetenwatch v2 API to create/update parliaments, terms, electoral districts, constituencies, representatives, and committee memberships. Management command `sync_representatives` orchestrates the import.
13
-
- **Suggestion engine**: `ConstituencySuggestionService` analyses letter titles + postal codes to recommend representatives, tags, and similar letters. The HTMX partial `letters/partials/suggestions.html` renders the live preview used on the letter form.
14
- **Identity & signatures**: `IdentityVerificationService` (stub) attaches address information to users; signature counts and verification badges are derived from the associated verification records.
15
- **Presentation**: Class-based views in `letters/views.py` back the main pages (letter list/detail, creation, representative detail, user profile). Templates under `letters/templates/` share layout via partials (e.g., `letter_card.html`).
16
- **Utilities**: Management commands in `letters/management/commands/` cover representative sync, taxonomy tests, and helper scripts. Tests in `letters/tests.py` exercise model behaviour, letter flows, and the suggestion service.
···
4
1. Install dependencies with `uv sync`.
5
2. Run `uv run python manage.py migrate` from `website/` to bootstrap the database.
6
3. (Optional) Import representatives via `uv run python manage.py sync_representatives --level all`.
7
+
4. Download constituency boundaries: `uv run python manage.py fetch_wahlkreis_data` (required for accurate address-based matching).
8
+
5. Launch the dev server with `uv run python manage.py runserver` and visit http://localhost:8000/.
9
10
## Architecture
11
- **Frameworks**: Django 5.2 / Python 3.13 managed with `uv`. The project root holds dependency metadata; all Django code lives in `website/` (settings in `writethem/`, app logic in `letters/`).
12
- **Domain models**: `letters/models.py` defines parliaments, terms, constituencies, representatives, committees, letters, signatures, identity verification, and moderation reports. Relationships reflect multi-level mandates (EU/Federal/State) and committee membership.
13
- **Sync pipeline**: `RepresentativeSyncService` (in `letters/services.py`) calls the Abgeordnetenwatch v2 API to create/update parliaments, terms, electoral districts, constituencies, representatives, and committee memberships. Management command `sync_representatives` orchestrates the import.
14
+
- **Constituency matching**: `AddressGeocoder` converts full addresses to coordinates via OSM Nominatim (cached in `GeocodeCache`). `WahlkreisLocator` performs point-in-polygon lookups against official Bundestag GeoJSON boundaries. `ConstituencyLocator` integrates both with PLZ fallback. See `docs/matching-algorithm.md` for details.
15
+
- **Suggestion engine**: `ConstituencySuggestionService` analyses letter titles + addresses to recommend representatives, tags, and similar letters. The HTMX partial `letters/partials/suggestions.html` renders the live preview used on the letter form.
16
- **Identity & signatures**: `IdentityVerificationService` (stub) attaches address information to users; signature counts and verification badges are derived from the associated verification records.
17
- **Presentation**: Class-based views in `letters/views.py` back the main pages (letter list/detail, creation, representative detail, user profile). Templates under `letters/templates/` share layout via partials (e.g., `letter_card.html`).
18
- **Utilities**: Management commands in `letters/management/commands/` cover representative sync, taxonomy tests, and helper scripts. Tests in `letters/tests.py` exercise model behaviour, letter flows, and the suggestion service.
+83
docs/matching-algorithm.md
+83
docs/matching-algorithm.md
···
···
1
+
# Constituency Matching Algorithm
2
+
3
+
## Overview
4
+
5
+
WriteThem.eu uses a two-stage process to match users to their correct Bundestag constituency:
6
+
7
+
1. **Address Geocoding**: Convert user's address to latitude/longitude coordinates
8
+
2. **Point-in-Polygon Lookup**: Find which constituency polygon contains those coordinates
9
+
10
+
## Stage 1: Address Geocoding
11
+
12
+
We use OpenStreetMap's Nominatim API to convert addresses to coordinates.
13
+
14
+
### Process:
15
+
1. User provides: Street, Postal Code, City
16
+
2. System checks cache (GeocodeCache table) for previous results
17
+
3. If not cached, query Nominatim API with rate limiting (1 req/sec)
18
+
4. Cache result (success or failure) to minimize API calls
19
+
5. Return (latitude, longitude) or None
20
+
21
+
### Fallback:
22
+
If geocoding fails or user only provides postal code, fall back to PLZ prefix heuristic (maps first 2 digits to state).
23
+
24
+
## Stage 2: Point-in-Polygon Lookup
25
+
26
+
We use official Bundestag constituency boundaries (GeoJSON format) with shapely for geometric queries.
27
+
28
+
### Process:
29
+
1. Load GeoJSON with 299 Bundestag constituencies on startup
30
+
2. Create shapely Point from coordinates
31
+
3. Check which constituency Polygon contains the point
32
+
4. Look up Constituency object in database by external_id
33
+
5. Return Constituency or None
34
+
35
+
### Performance:
36
+
- GeoJSON loaded once at startup (~2MB in memory)
37
+
- Class-level caching prevents repeated loads
38
+
- Lookup typically takes 10-50ms
39
+
- No external API calls required
40
+
41
+
## Data Sources
42
+
43
+
- **Constituency Boundaries**: [dknx01/wahlkreissuche](https://github.com/dknx01/wahlkreissuche) (Open Data)
44
+
- **Geocoding**: [OpenStreetMap Nominatim](https://nominatim.openstreetmap.org/) (Open Data)
45
+
- **Representative Data**: [Abgeordnetenwatch API](https://www.abgeordnetenwatch.de/api)
46
+
47
+
## Accuracy
48
+
49
+
This approach provides constituency-accurate matching (exact Wahlkreis), significantly more precise than PLZ-based heuristics which only provide state-level accuracy.
50
+
51
+
### Known Limitations:
52
+
- Requires valid German address
53
+
- Dependent on OSM geocoding quality
54
+
- Rate limited to 1 request/second (public API)
55
+
56
+
## Implementation Details
57
+
58
+
### Services
59
+
60
+
- **AddressGeocoder** (`letters/services.py`): Handles geocoding with caching
61
+
- **WahlkreisLocator** (`letters/services.py`): Performs point-in-polygon matching
62
+
- **ConstituencyLocator** (`letters/services.py`): Integrates both services with PLZ fallback
63
+
64
+
### Database Models
65
+
66
+
- **GeocodeCache** (`letters/models.py`): Caches geocoding results to minimize API calls
67
+
- **Constituency** (`letters/models.py`): Stores constituency information with external_id mapping to GeoJSON
68
+
69
+
### Management Commands
70
+
71
+
- **fetch_wahlkreis_data**: Downloads official Bundestag constituency boundaries
72
+
- **query_wahlkreis**: Query constituency by address or postal code
73
+
- **query_topics**: Find matching topics for letter text
74
+
- **query_representatives**: Find representatives by address and/or topics
75
+
76
+
### Testing
77
+
78
+
Run the test suite:
79
+
```bash
80
+
python manage.py test letters.tests.test_address_matching
81
+
python manage.py test letters.tests.test_topic_mapping
82
+
python manage.py test letters.tests.test_constituency_suggestions
83
+
```
+1414
docs/plans/2025-10-11-accurate-constituency-matching.md
+1414
docs/plans/2025-10-11-accurate-constituency-matching.md
···
···
1
+
# Accurate Constituency Matching Implementation Plan
2
+
3
+
> **For Claude:** Use `${CLAUDE_PLUGIN_ROOT}/skills/collaboration/executing-plans/SKILL.md` to implement this plan task-by-task.
4
+
5
+
**Goal:** Replace PLZ prefix heuristic with accurate address-based constituency matching using OSM Nominatim geocoding and GeoJSON point-in-polygon lookup.
6
+
7
+
**Architecture:** Two-layer approach: (1) AddressGeocoder service converts full German addresses to lat/lng coordinates via OSM Nominatim API with database caching, (2) WahlkreisLocator service uses shapely to perform point-in-polygon queries against Bundestag GeoJSON boundaries. PLZ prefix fallback remains for partial data.
8
+
9
+
**Tech Stack:** Django 5.x, shapely 2.x, requests, OSM Nominatim API, GeoJSON
10
+
11
+
---
12
+
13
+
## Task 1: Database Model for Geocoding Cache
14
+
15
+
**Files:**
16
+
- Create: `website/letters/models.py` (add new model)
17
+
- Create: `website/letters/migrations/0012_geocodecache.py` (auto-generated)
18
+
- Test: `website/letters/tests.py`
19
+
20
+
**Step 1: Write the failing test**
21
+
22
+
Add to `website/letters/tests.py`:
23
+
24
+
```python
25
+
class GeocodeCache Tests(TestCase):
26
+
"""Test geocoding cache model."""
27
+
28
+
def test_cache_stores_and_retrieves_coordinates(self):
29
+
from .models import GeocodeCache
30
+
31
+
cache_entry = GeocodeCache.objects.create(
32
+
address_hash='test_hash_123',
33
+
street='Unter den Linden 77',
34
+
postal_code='10117',
35
+
city='Berlin',
36
+
latitude=52.5170365,
37
+
longitude=13.3888599,
38
+
)
39
+
40
+
retrieved = GeocodeCache.objects.get(address_hash='test_hash_123')
41
+
self.assertEqual(retrieved.latitude, 52.5170365)
42
+
self.assertEqual(retrieved.longitude, 13.3888599)
43
+
self.assertEqual(retrieved.street, 'Unter den Linden 77')
44
+
```
45
+
46
+
**Step 2: Run test to verify it fails**
47
+
48
+
Run: `uv run python manage.py test letters.tests.GeocodeCacheTests -v`
49
+
Expected: FAIL with "No module named 'GeocodeCache'"
50
+
51
+
**Step 3: Add GeocodeCache model**
52
+
53
+
Add to `website/letters/models.py` after the existing models:
54
+
55
+
```python
56
+
class GeocodeCache(models.Model):
57
+
"""Cache geocoding results to minimize API calls."""
58
+
59
+
address_hash = models.CharField(
60
+
max_length=64,
61
+
unique=True,
62
+
db_index=True,
63
+
help_text="SHA256 hash of normalized address for fast lookup"
64
+
)
65
+
street = models.CharField(max_length=255, blank=True)
66
+
postal_code = models.CharField(max_length=10, blank=True)
67
+
city = models.CharField(max_length=100, blank=True)
68
+
country = models.CharField(max_length=2, default='DE')
69
+
70
+
latitude = models.FloatField(null=True, blank=True)
71
+
longitude = models.FloatField(null=True, blank=True)
72
+
73
+
success = models.BooleanField(
74
+
default=True,
75
+
help_text="False if geocoding failed, to avoid repeated failed lookups"
76
+
)
77
+
error_message = models.TextField(blank=True)
78
+
79
+
created_at = models.DateTimeField(auto_now_add=True)
80
+
updated_at = models.DateTimeField(auto_now=True)
81
+
82
+
class Meta:
83
+
verbose_name = "Geocode Cache Entry"
84
+
verbose_name_plural = "Geocode Cache Entries"
85
+
ordering = ['-created_at']
86
+
87
+
def __str__(self):
88
+
if self.latitude and self.longitude:
89
+
return f"{self.city} ({self.latitude}, {self.longitude})"
90
+
return f"{self.city} (failed)"
91
+
```
92
+
93
+
**Step 4: Create migration**
94
+
95
+
Run: `uv run python manage.py makemigrations letters`
96
+
Expected: Migration 0012_geocodecache.py created
97
+
98
+
**Step 5: Run migration**
99
+
100
+
Run: `uv run python manage.py migrate`
101
+
Expected: "Applying letters.0012_geocodecache... OK"
102
+
103
+
**Step 6: Run test to verify it passes**
104
+
105
+
Run: `uv run python manage.py test letters.tests.GeocodeCacheTests -v`
106
+
Expected: PASS
107
+
108
+
**Step 7: Commit**
109
+
110
+
```bash
111
+
git add website/letters/models.py website/letters/migrations/0012_geocodecache.py website/letters/tests.py
112
+
git commit -m "feat: add GeocodeCache model for address geocoding results"
113
+
```
114
+
115
+
---
116
+
117
+
## Task 2: OSM Nominatim API Client
118
+
119
+
**Files:**
120
+
- Modify: `website/letters/services.py` (add AddressGeocoder class)
121
+
- Test: `website/letters/tests.py`
122
+
123
+
**Step 1: Write the failing test**
124
+
125
+
Add to `website/letters/tests.py`:
126
+
127
+
```python
128
+
from unittest.mock import patch, MagicMock
129
+
130
+
131
+
class AddressGeocoderTests(TestCase):
132
+
"""Test OSM Nominatim address geocoding."""
133
+
134
+
def test_geocode_returns_coordinates_for_valid_address(self):
135
+
from .services import AddressGeocoder
136
+
137
+
# Mock the Nominatim API response
138
+
mock_response = MagicMock()
139
+
mock_response.json.return_value = [{
140
+
'lat': '52.5170365',
141
+
'lon': '13.3888599',
142
+
'display_name': 'Unter den Linden 77, Mitte, Berlin, 10117, Deutschland'
143
+
}]
144
+
mock_response.status_code = 200
145
+
146
+
with patch('requests.get', return_value=mock_response):
147
+
result = AddressGeocoder.geocode(
148
+
street='Unter den Linden 77',
149
+
postal_code='10117',
150
+
city='Berlin'
151
+
)
152
+
153
+
self.assertIsNotNone(result)
154
+
lat, lng = result
155
+
self.assertAlmostEqual(lat, 52.5170365, places=5)
156
+
self.assertAlmostEqual(lng, 13.3888599, places=5)
157
+
158
+
def test_geocode_caches_results(self):
159
+
from .services import AddressGeocoder
160
+
from .models import GeocodeCache
161
+
162
+
mock_response = MagicMock()
163
+
mock_response.json.return_value = [{
164
+
'lat': '52.5170365',
165
+
'lon': '13.3888599',
166
+
}]
167
+
mock_response.status_code = 200
168
+
169
+
with patch('requests.get', return_value=mock_response) as mock_get:
170
+
# First call should hit API
171
+
result1 = AddressGeocoder.geocode(
172
+
street='Unter den Linden 77',
173
+
postal_code='10117',
174
+
city='Berlin'
175
+
)
176
+
177
+
# Second call should use cache
178
+
result2 = AddressGeocoder.geocode(
179
+
street='Unter den Linden 77',
180
+
postal_code='10117',
181
+
city='Berlin'
182
+
)
183
+
184
+
# API should only be called once
185
+
self.assertEqual(mock_get.call_count, 1)
186
+
self.assertEqual(result1, result2)
187
+
188
+
# Verify cache entry exists
189
+
self.assertTrue(
190
+
GeocodeCache.objects.filter(
191
+
city='Berlin',
192
+
postal_code='10117'
193
+
).exists()
194
+
)
195
+
```
196
+
197
+
**Step 2: Run test to verify it fails**
198
+
199
+
Run: `uv run python manage.py test letters.tests.AddressGeocoderTests -v`
200
+
Expected: FAIL with "No module named 'AddressGeocoder'"
201
+
202
+
**Step 3: Implement AddressGeocoder service**
203
+
204
+
Add to `website/letters/services.py` after the existing classes:
205
+
206
+
```python
207
+
import hashlib
208
+
import time
209
+
from typing import Optional, Tuple
210
+
211
+
212
+
class AddressGeocoder:
213
+
"""Geocode German addresses using OSM Nominatim API."""
214
+
215
+
NOMINATIM_URL = "https://nominatim.openstreetmap.org/search"
216
+
USER_AGENT = "WriteThem.eu/1.0 (https://writethem.eu; contact@writethem.eu)"
217
+
REQUEST_TIMEOUT = 10
218
+
RATE_LIMIT_DELAY = 1.0 # seconds between requests
219
+
220
+
_last_request_time = 0
221
+
222
+
@classmethod
223
+
def geocode(
224
+
cls,
225
+
street: str,
226
+
postal_code: str,
227
+
city: str,
228
+
country: str = 'DE'
229
+
) -> Optional[Tuple[float, float]]:
230
+
"""
231
+
Geocode a German address to lat/lng coordinates.
232
+
233
+
Args:
234
+
street: Street address (e.g., "Unter den Linden 77")
235
+
postal_code: German postal code (e.g., "10117")
236
+
city: City name (e.g., "Berlin")
237
+
country: Country code (default: 'DE')
238
+
239
+
Returns:
240
+
Tuple of (latitude, longitude) or None if geocoding fails
241
+
"""
242
+
from .models import GeocodeCache
243
+
244
+
# Normalize inputs
245
+
street = (street or '').strip()
246
+
postal_code = (postal_code or '').strip()
247
+
city = (city or '').strip()
248
+
country = (country or 'DE').upper()
249
+
250
+
if not city:
251
+
logger.warning("City is required for geocoding")
252
+
return None
253
+
254
+
# Generate cache key
255
+
address_string = f"{street}|{postal_code}|{city}|{country}".lower()
256
+
address_hash = hashlib.sha256(address_string.encode()).hexdigest()
257
+
258
+
# Check cache first
259
+
cached = GeocodeCache.objects.filter(address_hash=address_hash).first()
260
+
if cached:
261
+
if cached.success and cached.latitude and cached.longitude:
262
+
logger.debug(f"Cache hit for {city}: ({cached.latitude}, {cached.longitude})")
263
+
return (cached.latitude, cached.longitude)
264
+
elif not cached.success:
265
+
logger.debug(f"Cache hit for {city}: previous failure")
266
+
return None
267
+
268
+
# Rate limiting
269
+
cls._rate_limit()
270
+
271
+
# Build query
272
+
query_parts = []
273
+
if street:
274
+
query_parts.append(street)
275
+
if postal_code:
276
+
query_parts.append(postal_code)
277
+
query_parts.append(city)
278
+
query_parts.append(country)
279
+
280
+
query = ', '.join(query_parts)
281
+
282
+
params = {
283
+
'q': query,
284
+
'format': 'json',
285
+
'limit': 1,
286
+
'addressdetails': 1,
287
+
'countrycodes': country.lower(),
288
+
}
289
+
290
+
headers = {
291
+
'User-Agent': cls.USER_AGENT
292
+
}
293
+
294
+
try:
295
+
logger.info(f"Geocoding address: {query}")
296
+
response = requests.get(
297
+
cls.NOMINATIM_URL,
298
+
params=params,
299
+
headers=headers,
300
+
timeout=cls.REQUEST_TIMEOUT
301
+
)
302
+
response.raise_for_status()
303
+
304
+
results = response.json()
305
+
306
+
if not results:
307
+
logger.warning(f"No geocoding results for: {query}")
308
+
cls._cache_failure(address_hash, street, postal_code, city, country, "No results")
309
+
return None
310
+
311
+
# Extract coordinates
312
+
result = results[0]
313
+
latitude = float(result['lat'])
314
+
longitude = float(result['lon'])
315
+
316
+
# Cache success
317
+
GeocodeCache.objects.update_or_create(
318
+
address_hash=address_hash,
319
+
defaults={
320
+
'street': street,
321
+
'postal_code': postal_code,
322
+
'city': city,
323
+
'country': country,
324
+
'latitude': latitude,
325
+
'longitude': longitude,
326
+
'success': True,
327
+
'error_message': '',
328
+
}
329
+
)
330
+
331
+
logger.info(f"Geocoded {city} to ({latitude}, {longitude})")
332
+
return (latitude, longitude)
333
+
334
+
except requests.RequestException as e:
335
+
error_msg = f"Nominatim API error: {e}"
336
+
logger.error(error_msg)
337
+
cls._cache_failure(address_hash, street, postal_code, city, country, error_msg)
338
+
return None
339
+
except (KeyError, ValueError, TypeError) as e:
340
+
error_msg = f"Invalid geocoding response: {e}"
341
+
logger.error(error_msg)
342
+
cls._cache_failure(address_hash, street, postal_code, city, country, error_msg)
343
+
return None
344
+
345
+
@classmethod
346
+
def _rate_limit(cls):
347
+
"""Ensure we don't exceed Nominatim rate limits (1 req/sec)."""
348
+
import time
349
+
current_time = time.time()
350
+
elapsed = current_time - cls._last_request_time
351
+
352
+
if elapsed < cls.RATE_LIMIT_DELAY:
353
+
time.sleep(cls.RATE_LIMIT_DELAY - elapsed)
354
+
355
+
cls._last_request_time = time.time()
356
+
357
+
@classmethod
358
+
def _cache_failure(
359
+
cls,
360
+
address_hash: str,
361
+
street: str,
362
+
postal_code: str,
363
+
city: str,
364
+
country: str,
365
+
error_message: str
366
+
):
367
+
"""Cache a failed geocoding attempt to avoid repeated failures."""
368
+
from .models import GeocodeCache
369
+
370
+
GeocodeCache.objects.update_or_create(
371
+
address_hash=address_hash,
372
+
defaults={
373
+
'street': street,
374
+
'postal_code': postal_code,
375
+
'city': city,
376
+
'country': country,
377
+
'latitude': None,
378
+
'longitude': None,
379
+
'success': False,
380
+
'error_message': error_message,
381
+
}
382
+
)
383
+
```
384
+
385
+
**Step 4: Run test to verify it passes**
386
+
387
+
Run: `uv run python manage.py test letters.tests.AddressGeocoderTests -v`
388
+
Expected: PASS (2 tests)
389
+
390
+
**Step 5: Commit**
391
+
392
+
```bash
393
+
git add website/letters/services.py website/letters/tests.py
394
+
git commit -m "feat: add OSM Nominatim address geocoding service with caching"
395
+
```
396
+
397
+
---
398
+
399
+
## Task 3: Download and Prepare GeoJSON Data
400
+
401
+
**Files:**
402
+
- Modify: `website/letters/management/commands/fetch_wahlkreis_data.py` (already exists)
403
+
- Create: `website/letters/data/wahlkreise.geojson` (downloaded data)
404
+
405
+
**Step 1: Test existing download command**
406
+
407
+
Run: `uv run python manage.py fetch_wahlkreis_data --help`
408
+
Expected: Shows command help text
409
+
410
+
**Step 2: Download full Bundestag GeoJSON**
411
+
412
+
Run: `uv run python manage.py fetch_wahlkreis_data --output=website/letters/data/wahlkreise.geojson --force`
413
+
Expected: "Saved Wahlkreis data to website/letters/data/wahlkreise.geojson"
414
+
415
+
**Step 3: Verify GeoJSON structure**
416
+
417
+
Run: `uv run python -c "import json; data = json.load(open('website/letters/data/wahlkreise.geojson')); print(f'Loaded {len(data[\"features\"])} constituencies')"`
418
+
Expected: "Loaded 299 constituencies" (or similar)
419
+
420
+
**Step 4: Add GeoJSON to gitignore**
421
+
422
+
Add to `.gitignore`:
423
+
```
424
+
# Large GeoJSON data files
425
+
website/letters/data/*.geojson
426
+
!website/letters/data/wahlkreise_sample.geojson
427
+
```
428
+
429
+
**Step 5: Commit gitignore update**
430
+
431
+
```bash
432
+
git add .gitignore
433
+
git commit -m "chore: add GeoJSON files to gitignore"
434
+
```
435
+
436
+
**Step 6: Document download in README**
437
+
438
+
Add to README.md setup instructions:
439
+
```markdown
440
+
### Download Constituency Boundaries
441
+
442
+
Download the Bundestag constituency boundaries:
443
+
444
+
\`\`\`bash
445
+
uv run python manage.py fetch_wahlkreis_data
446
+
\`\`\`
447
+
448
+
This downloads ~2MB of GeoJSON data for accurate constituency matching.
449
+
```
450
+
451
+
**Step 7: Commit documentation**
452
+
453
+
```bash
454
+
git add README.md
455
+
git commit -m "docs: add constituency data download instructions"
456
+
```
457
+
458
+
---
459
+
460
+
## Task 4: WahlkreisLocator Service with Shapely
461
+
462
+
**Files:**
463
+
- Modify: `website/letters/services.py` (add WahlkreisLocator class)
464
+
- Test: `website/letters/tests.py`
465
+
466
+
**Step 1: Write the failing test**
467
+
468
+
Add to `website/letters/tests.py`:
469
+
470
+
```python
471
+
class WahlkreisLocatorTests(TestCase):
472
+
"""Test GeoJSON point-in-polygon constituency lookup."""
473
+
474
+
def setUp(self):
475
+
super().setUp()
476
+
# Create test parliament and constituencies
477
+
self.parliament = Parliament.objects.create(
478
+
name='Deutscher Bundestag',
479
+
level='FEDERAL',
480
+
legislative_body='Bundestag',
481
+
region='DE',
482
+
)
483
+
self.term = ParliamentTerm.objects.create(
484
+
parliament=self.parliament,
485
+
name='20. Wahlperiode',
486
+
start_date=date(2021, 10, 26),
487
+
)
488
+
# Berlin-Mitte constituency
489
+
self.constituency_mitte = Constituency.objects.create(
490
+
parliament_term=self.term,
491
+
name='Berlin-Mitte',
492
+
scope='FEDERAL_DISTRICT',
493
+
external_id='75', # Real Wahlkreis ID
494
+
metadata={'state': 'Berlin'},
495
+
)
496
+
497
+
def test_find_constituency_for_berlin_coordinates(self):
498
+
from .services import WahlkreisLocator
499
+
500
+
# Coordinates for Unter den Linden, Berlin-Mitte
501
+
latitude = 52.5170365
502
+
longitude = 13.3888599
503
+
504
+
result = WahlkreisLocator.find_constituency(latitude, longitude)
505
+
506
+
self.assertIsNotNone(result)
507
+
self.assertEqual(result.external_id, '75') # Berlin-Mitte
508
+
self.assertEqual(result.scope, 'FEDERAL_DISTRICT')
509
+
510
+
def test_returns_none_for_coordinates_outside_germany(self):
511
+
from .services import WahlkreisLocator
512
+
513
+
# Coordinates in Paris
514
+
latitude = 48.8566
515
+
longitude = 2.3522
516
+
517
+
result = WahlkreisLocator.find_constituency(latitude, longitude)
518
+
519
+
self.assertIsNone(result)
520
+
```
521
+
522
+
**Step 2: Run test to verify it fails**
523
+
524
+
Run: `uv run python manage.py test letters.tests.WahlkreisLocatorTests -v`
525
+
Expected: FAIL with "No module named 'WahlkreisLocator'"
526
+
527
+
**Step 3: Implement WahlkreisLocator service**
528
+
529
+
Add to `website/letters/services.py`:
530
+
531
+
```python
532
+
from pathlib import Path
533
+
from shapely.geometry import Point, shape
534
+
from typing import Optional, List, Dict, Any
535
+
536
+
537
+
class WahlkreisLocator:
538
+
"""Locate Bundestag constituency from lat/lng using GeoJSON boundaries."""
539
+
540
+
_geojson_data: Optional[Dict[str, Any]] = None
541
+
_geometries: Optional[List[tuple]] = None
542
+
543
+
GEOJSON_PATH = Path(__file__).parent / 'data' / 'wahlkreise.geojson'
544
+
545
+
@classmethod
546
+
def _load_geojson(cls):
547
+
"""Load GeoJSON data into memory (called once at startup)."""
548
+
if cls._geometries is not None:
549
+
return
550
+
551
+
if not cls.GEOJSON_PATH.exists():
552
+
logger.warning(f"GeoJSON file not found: {cls.GEOJSON_PATH}")
553
+
logger.warning("Run: python manage.py fetch_wahlkreis_data")
554
+
cls._geometries = []
555
+
return
556
+
557
+
try:
558
+
with open(cls.GEOJSON_PATH, 'r', encoding='utf-8') as f:
559
+
cls._geojson_data = json.load(f)
560
+
561
+
# Pre-process geometries for faster lookup
562
+
cls._geometries = []
563
+
for feature in cls._geojson_data.get('features', []):
564
+
geometry = shape(feature['geometry'])
565
+
properties = feature.get('properties', {})
566
+
567
+
# Extract Wahlkreis ID from properties
568
+
wahlkreis_id = properties.get('WKR_NR') or properties.get('id')
569
+
wahlkreis_name = properties.get('WKR_NAME') or properties.get('name')
570
+
571
+
if wahlkreis_id:
572
+
cls._geometries.append((
573
+
str(wahlkreis_id),
574
+
wahlkreis_name,
575
+
geometry
576
+
))
577
+
578
+
logger.info(f"Loaded {len(cls._geometries)} constituencies from GeoJSON")
579
+
580
+
except Exception as e:
581
+
logger.error(f"Failed to load GeoJSON: {e}")
582
+
cls._geometries = []
583
+
584
+
@classmethod
585
+
def find_constituency(
586
+
cls,
587
+
latitude: float,
588
+
longitude: float
589
+
) -> Optional[Constituency]:
590
+
"""
591
+
Find the Bundestag constituency containing the given coordinates.
592
+
593
+
Args:
594
+
latitude: Latitude in decimal degrees
595
+
longitude: Longitude in decimal degrees
596
+
597
+
Returns:
598
+
Constituency object or None if not found
599
+
"""
600
+
cls._load_geojson()
601
+
602
+
if not cls._geometries:
603
+
logger.warning("No GeoJSON data loaded")
604
+
return None
605
+
606
+
point = Point(longitude, latitude) # Note: shapely uses (x, y) = (lon, lat)
607
+
608
+
# Find which polygon contains this point
609
+
for wahlkreis_id, wahlkreis_name, geometry in cls._geometries:
610
+
if geometry.contains(point):
611
+
logger.debug(f"Found constituency: {wahlkreis_name} (ID: {wahlkreis_id})")
612
+
613
+
# Look up in database
614
+
constituency = Constituency.objects.filter(
615
+
external_id=wahlkreis_id,
616
+
scope='FEDERAL_DISTRICT'
617
+
).first()
618
+
619
+
if constituency:
620
+
return constituency
621
+
else:
622
+
logger.warning(
623
+
f"Constituency {wahlkreis_id} found in GeoJSON but not in database"
624
+
)
625
+
return None
626
+
627
+
logger.debug(f"No constituency found for coordinates ({latitude}, {longitude})")
628
+
return None
629
+
630
+
@classmethod
631
+
def clear_cache(cls):
632
+
"""Clear cached GeoJSON data (useful for testing)."""
633
+
cls._geojson_data = None
634
+
cls._geometries = None
635
+
```
636
+
637
+
**Step 4: Add shapely to requirements**
638
+
639
+
Check if shapely is in requirements:
640
+
Run: `grep shapely pyproject.toml || grep shapely requirements.txt`
641
+
642
+
If not found, add to pyproject.toml dependencies:
643
+
```toml
644
+
dependencies = [
645
+
"django>=5.0",
646
+
"shapely>=2.0",
647
+
# ... other deps
648
+
]
649
+
```
650
+
651
+
**Step 5: Install shapely**
652
+
653
+
Run: `uv sync`
654
+
Expected: "Resolved X packages in Yms"
655
+
656
+
**Step 6: Run test to verify it passes**
657
+
658
+
Run: `uv run python manage.py test letters.tests.WahlkreisLocatorTests -v`
659
+
Expected: PASS (2 tests)
660
+
661
+
**Step 7: Commit**
662
+
663
+
```bash
664
+
git add website/letters/services.py website/letters/tests.py pyproject.toml
665
+
git commit -m "feat: add GeoJSON point-in-polygon constituency lookup"
666
+
```
667
+
668
+
---
669
+
670
+
## Task 5: Integration - Update ConstituencyLocator
671
+
672
+
**Files:**
673
+
- Modify: `website/letters/services.py` (update ConstituencyLocator class)
674
+
- Test: `website/letters/tests.py`
675
+
676
+
**Step 1: Write integration test**
677
+
678
+
Add to `website/letters/tests.py`:
679
+
680
+
```python
681
+
class ConstituencyLocatorIntegrationTests(TestCase):
682
+
"""Test integrated address → constituency lookup."""
683
+
684
+
def setUp(self):
685
+
super().setUp()
686
+
self.parliament = Parliament.objects.create(
687
+
name='Deutscher Bundestag',
688
+
level='FEDERAL',
689
+
legislative_body='Bundestag',
690
+
region='DE',
691
+
)
692
+
self.term = ParliamentTerm.objects.create(
693
+
parliament=self.parliament,
694
+
name='20. Wahlperiode',
695
+
start_date=date(2021, 10, 26),
696
+
)
697
+
self.constituency_mitte = Constituency.objects.create(
698
+
parliament_term=self.term,
699
+
name='Berlin-Mitte',
700
+
scope='FEDERAL_DISTRICT',
701
+
external_id='75',
702
+
metadata={'state': 'Berlin'},
703
+
)
704
+
705
+
@patch('letters.services.AddressGeocoder.geocode')
706
+
def test_locate_uses_address_geocoding(self, mock_geocode):
707
+
from .services import ConstituencyLocator
708
+
709
+
# Mock geocoding to return Berlin-Mitte coordinates
710
+
mock_geocode.return_value = (52.5170365, 13.3888599)
711
+
712
+
result = ConstituencyLocator.locate_from_address(
713
+
street='Unter den Linden 77',
714
+
postal_code='10117',
715
+
city='Berlin'
716
+
)
717
+
718
+
self.assertIsNotNone(result.federal)
719
+
self.assertEqual(result.federal.external_id, '75')
720
+
721
+
# Verify geocoder was called
722
+
mock_geocode.assert_called_once_with(
723
+
street='Unter den Linden 77',
724
+
postal_code='10117',
725
+
city='Berlin',
726
+
country='DE'
727
+
)
728
+
729
+
def test_locate_falls_back_to_plz_prefix(self):
730
+
from .services import ConstituencyLocator
731
+
732
+
# Test with just PLZ (no full address)
733
+
result = ConstituencyLocator.locate('10117')
734
+
735
+
# Should return Berlin-based constituency using old heuristic
736
+
self.assertIsNotNone(result.federal)
737
+
```
738
+
739
+
**Step 2: Run test to verify it fails**
740
+
741
+
Run: `uv run python manage.py test letters.tests.ConstituencyLocatorIntegrationTests -v`
742
+
Expected: FAIL with "No method named 'locate_from_address'"
743
+
744
+
**Step 3: Add locate_from_address method**
745
+
746
+
Modify `ConstituencyLocator` class in `website/letters/services.py`:
747
+
748
+
```python
749
+
class ConstituencyLocator:
750
+
"""Enhanced mapping from addresses/postal codes to constituencies."""
751
+
752
+
# ... existing STATE_BY_PLZ_PREFIX dict ...
753
+
754
+
@classmethod
755
+
def locate_from_address(
756
+
cls,
757
+
street: str,
758
+
postal_code: str,
759
+
city: str,
760
+
country: str = 'DE'
761
+
) -> LocatedConstituencies:
762
+
"""
763
+
Locate constituency from full address using geocoding.
764
+
765
+
This is the preferred method for accurate constituency matching.
766
+
Falls back to PLZ prefix if geocoding fails.
767
+
"""
768
+
# Try accurate geocoding first
769
+
coordinates = AddressGeocoder.geocode(street, postal_code, city, country)
770
+
771
+
if coordinates:
772
+
latitude, longitude = coordinates
773
+
774
+
# Use GeoJSON lookup for federal constituency
775
+
federal_constituency = WahlkreisLocator.find_constituency(latitude, longitude)
776
+
777
+
if federal_constituency:
778
+
# Also try to determine state from the federal constituency
779
+
state_name = (federal_constituency.metadata or {}).get('state')
780
+
state_constituency = None
781
+
782
+
if state_name:
783
+
normalized_state = normalize_german_state(state_name)
784
+
state_constituency = cls._match_state(normalized_state)
785
+
786
+
return LocatedConstituencies(
787
+
federal=federal_constituency,
788
+
state=state_constituency,
789
+
local=None
790
+
)
791
+
792
+
# Fallback to PLZ prefix heuristic
793
+
logger.info(f"Falling back to PLZ prefix lookup for {postal_code}")
794
+
return cls.locate(postal_code)
795
+
796
+
@classmethod
797
+
def locate(cls, postal_code: str) -> LocatedConstituencies:
798
+
"""
799
+
Legacy PLZ prefix-based lookup.
800
+
801
+
Use locate_from_address() for accurate results.
802
+
This method kept for backwards compatibility and fallback.
803
+
"""
804
+
# ... existing implementation unchanged ...
805
+
```
806
+
807
+
**Step 4: Run test to verify it passes**
808
+
809
+
Run: `uv run python manage.py test letters.tests.ConstituencyLocatorIntegrationTests -v`
810
+
Expected: PASS (2 tests)
811
+
812
+
**Step 5: Commit**
813
+
814
+
```bash
815
+
git add website/letters/services.py website/letters/tests.py
816
+
git commit -m "feat: integrate address geocoding into ConstituencyLocator"
817
+
```
818
+
819
+
---
820
+
821
+
## Task 6: Update ConstituencySuggestionService to Use Address
822
+
823
+
**Files:**
824
+
- Modify: `website/letters/services.py` (update _resolve_location method)
825
+
- Test: `website/letters/tests.py`
826
+
827
+
**Step 1: Write test for address-based suggestion**
828
+
829
+
Add to `website/letters/tests.py`:
830
+
831
+
```python
832
+
class SuggestionServiceAddressTests(ParliamentFixtureMixin, TestCase):
833
+
"""Test suggestions with full address input."""
834
+
835
+
@patch('letters.services.AddressGeocoder.geocode')
836
+
def test_suggestions_with_full_address(self, mock_geocode):
837
+
from .services import ConstituencySuggestionService
838
+
839
+
# Mock geocoding
840
+
mock_geocode.return_value = (52.5170365, 13.3888599)
841
+
842
+
result = ConstituencySuggestionService.suggest_from_concern(
843
+
'Mehr Investitionen in den ÖPNV',
844
+
user_location={
845
+
'street': 'Unter den Linden 77',
846
+
'postal_code': '10117',
847
+
'city': 'Berlin',
848
+
}
849
+
)
850
+
851
+
self.assertIsNotNone(result['constituencies'])
852
+
self.assertTrue(len(result['representatives']) > 0)
853
+
```
854
+
855
+
**Step 2: Run test to verify it fails**
856
+
857
+
Run: `uv run python manage.py test letters.tests.SuggestionServiceAddressTests -v`
858
+
Expected: FAIL (address not being used)
859
+
860
+
**Step 3: Update _resolve_location to handle addresses**
861
+
862
+
Modify `ConstituencySuggestionService._resolve_location` in `website/letters/services.py`:
863
+
864
+
```python
865
+
@classmethod
866
+
def _resolve_location(cls, user_location: Dict[str, str]) -> LocationContext:
867
+
"""Resolve user location from various input formats."""
868
+
869
+
# Check if full address is provided
870
+
street = (user_location.get('street') or '').strip()
871
+
postal_code = (user_location.get('postal_code') or '').strip()
872
+
city = (user_location.get('city') or '').strip()
873
+
874
+
constituencies: List[Constituency] = []
875
+
876
+
# Priority 1: Explicitly provided constituency IDs
877
+
provided_constituencies = user_location.get('constituencies')
878
+
if provided_constituencies:
879
+
iterable = provided_constituencies if isinstance(provided_constituencies, (list, tuple, set)) else [provided_constituencies]
880
+
for item in iterable:
881
+
constituency = None
882
+
if isinstance(item, Constituency):
883
+
constituency = item
884
+
else:
885
+
try:
886
+
constituency_id = int(item)
887
+
except (TypeError, ValueError):
888
+
constituency_id = None
889
+
if constituency_id:
890
+
constituency = Constituency.objects.filter(id=constituency_id).first()
891
+
if constituency and all(c.id != constituency.id for c in constituencies):
892
+
constituencies.append(constituency)
893
+
894
+
# Priority 2: Full address geocoding
895
+
if not constituencies and city:
896
+
logger.info(f"Using address geocoding for: {city}")
897
+
located = ConstituencyLocator.locate_from_address(street, postal_code, city)
898
+
constituencies.extend(
899
+
constituency
900
+
for constituency in (located.local, located.state, located.federal)
901
+
if constituency
902
+
)
903
+
904
+
# Priority 3: PLZ-only fallback
905
+
elif not constituencies and postal_code:
906
+
logger.info(f"Using PLZ fallback for: {postal_code}")
907
+
located = ConstituencyLocator.locate(postal_code)
908
+
constituencies.extend(
909
+
constituency
910
+
for constituency in (located.local, located.state, located.federal)
911
+
if constituency
912
+
)
913
+
else:
914
+
located = LocatedConstituencies(None, None, None)
915
+
916
+
# Determine state
917
+
explicit_state = normalize_german_state(user_location.get('state')) if user_location.get('state') else None
918
+
inferred_state = None
919
+
for constituency in constituencies:
920
+
metadata_state = (constituency.metadata or {}).get('state') if constituency.metadata else None
921
+
if metadata_state:
922
+
inferred_state = normalize_german_state(metadata_state)
923
+
if inferred_state:
924
+
break
925
+
926
+
state = explicit_state or inferred_state
927
+
928
+
return LocationContext(
929
+
postal_code=postal_code or None,
930
+
state=state,
931
+
constituencies=constituencies,
932
+
)
933
+
```
934
+
935
+
**Step 4: Run test to verify it passes**
936
+
937
+
Run: `uv run python manage.py test letters.tests.SuggestionServiceAddressTests -v`
938
+
Expected: PASS
939
+
940
+
**Step 5: Commit**
941
+
942
+
```bash
943
+
git add website/letters/services.py website/letters/tests.py
944
+
git commit -m "feat: support full address in suggestion service"
945
+
```
946
+
947
+
---
948
+
949
+
## Task 7: Update Profile View to Collect Full Address
950
+
951
+
**Files:**
952
+
- Modify: `website/letters/forms.py` (update verification form)
953
+
- Modify: `website/letters/templates/letters/profile.html`
954
+
- Test: `website/letters/tests.py`
955
+
956
+
**Step 1: Write test for address collection**
957
+
958
+
Add to `website/letters/tests.py`:
959
+
960
+
```python
961
+
class ProfileAddressCollectionTests(TestCase):
962
+
"""Test profile form collects full address."""
963
+
964
+
def setUp(self):
965
+
self.user = User.objects.create_user(
966
+
username='testuser',
967
+
password='password123',
968
+
email='test@example.com'
969
+
)
970
+
971
+
def test_profile_form_has_address_fields(self):
972
+
from .forms import SelfDeclaredVerificationForm
973
+
974
+
form = SelfDeclaredVerificationForm()
975
+
976
+
self.assertIn('street', form.fields)
977
+
self.assertIn('postal_code', form.fields)
978
+
self.assertIn('city', form.fields)
979
+
```
980
+
981
+
**Step 2: Run test to verify it fails**
982
+
983
+
Run: `uv run python manage.py test letters.tests.ProfileAddressCollectionTests -v`
984
+
Expected: FAIL (fields don't exist)
985
+
986
+
**Step 3: Add address fields to form**
987
+
988
+
Modify `website/letters/forms.py` to add address fields to the verification form:
989
+
990
+
```python
991
+
class SelfDeclaredVerificationForm(forms.Form):
992
+
"""Form for self-declared constituency verification."""
993
+
994
+
street = forms.CharField(
995
+
max_length=255,
996
+
required=False,
997
+
label=_("Street and Number"),
998
+
help_text=_("Optional: Improves constituency matching accuracy"),
999
+
widget=forms.TextInput(attrs={
1000
+
'placeholder': _('Unter den Linden 77'),
1001
+
'class': 'form-control'
1002
+
})
1003
+
)
1004
+
1005
+
postal_code = forms.CharField(
1006
+
max_length=10,
1007
+
required=True,
1008
+
label=_("Postal Code"),
1009
+
widget=forms.TextInput(attrs={
1010
+
'placeholder': '10117',
1011
+
'class': 'form-control'
1012
+
})
1013
+
)
1014
+
1015
+
city = forms.CharField(
1016
+
max_length=100,
1017
+
required=True,
1018
+
label=_("City"),
1019
+
widget=forms.TextInput(attrs={
1020
+
'placeholder': 'Berlin',
1021
+
'class': 'form-control'
1022
+
})
1023
+
)
1024
+
1025
+
federal_constituency = forms.ModelChoiceField(
1026
+
queryset=Constituency.objects.filter(scope='FEDERAL_DISTRICT'),
1027
+
required=False,
1028
+
label=_("Federal Constituency (optional)"),
1029
+
help_text=_("Leave blank for automatic detection"),
1030
+
widget=forms.Select(attrs={'class': 'form-control'})
1031
+
)
1032
+
1033
+
state_constituency = forms.ModelChoiceField(
1034
+
queryset=Constituency.objects.filter(scope__in=['STATE_DISTRICT', 'STATE_LIST']),
1035
+
required=False,
1036
+
label=_("State Constituency (optional)"),
1037
+
help_text=_("Leave blank for automatic detection"),
1038
+
widget=forms.Select(attrs={'class': 'form-control'})
1039
+
)
1040
+
```
1041
+
1042
+
**Step 4: Update template to show address fields**
1043
+
1044
+
Modify `website/letters/templates/letters/profile.html` to show the new fields in the verification form section.
1045
+
1046
+
**Step 5: Update view to use address for verification**
1047
+
1048
+
Modify the `complete_verification` or equivalent view in `website/letters/views.py` to use the address fields:
1049
+
1050
+
```python
1051
+
def complete_verification(request):
1052
+
if request.method == 'POST':
1053
+
form = SelfDeclaredVerificationForm(request.POST)
1054
+
if form.is_valid():
1055
+
street = form.cleaned_data.get('street', '')
1056
+
postal_code = form.cleaned_data['postal_code']
1057
+
city = form.cleaned_data['city']
1058
+
1059
+
# Use address-based lookup if full address provided
1060
+
if city:
1061
+
located = ConstituencyLocator.locate_from_address(
1062
+
street, postal_code, city
1063
+
)
1064
+
else:
1065
+
located = ConstituencyLocator.locate(postal_code)
1066
+
1067
+
federal = form.cleaned_data.get('federal_constituency') or located.federal
1068
+
state = form.cleaned_data.get('state_constituency') or located.state
1069
+
1070
+
IdentityVerificationService.self_declare(
1071
+
request.user,
1072
+
federal_constituency=federal,
1073
+
state_constituency=state,
1074
+
)
1075
+
1076
+
messages.success(request, _('Your constituency has been saved.'))
1077
+
return redirect('profile')
1078
+
else:
1079
+
form = SelfDeclaredVerificationForm()
1080
+
1081
+
return render(request, 'letters/complete_verification.html', {'form': form})
1082
+
```
1083
+
1084
+
**Step 6: Run test to verify it passes**
1085
+
1086
+
Run: `uv run python manage.py test letters.tests.ProfileAddressCollectionTests -v`
1087
+
Expected: PASS
1088
+
1089
+
**Step 7: Test manually in browser**
1090
+
1091
+
1. Run dev server: `uv run python manage.py runserver`
1092
+
2. Navigate to /profile/verify/
1093
+
3. Verify address fields are visible
1094
+
4. Submit form with full address
1095
+
5. Check that constituency is correctly detected
1096
+
1097
+
**Step 8: Commit**
1098
+
1099
+
```bash
1100
+
git add website/letters/forms.py website/letters/templates/letters/profile.html website/letters/views.py website/letters/tests.py
1101
+
git commit -m "feat: collect full address for accurate constituency matching"
1102
+
```
1103
+
1104
+
---
1105
+
1106
+
## Task 8: Add Management Command to Test Matching
1107
+
1108
+
**Files:**
1109
+
- Create: `website/letters/management/commands/test_address_matching.py`
1110
+
1111
+
**Step 1: Create test command**
1112
+
1113
+
Create `website/letters/management/commands/test_address_matching.py`:
1114
+
1115
+
```python
1116
+
from django.core.management.base import BaseCommand
1117
+
from letters.services import AddressGeocoder, WahlkreisLocator, ConstituencyLocator
1118
+
1119
+
1120
+
class Command(BaseCommand):
1121
+
help = "Test address matching with sample German addresses"
1122
+
1123
+
TEST_ADDRESSES = [
1124
+
# Berlin
1125
+
("Unter den Linden 77", "10117", "Berlin"),
1126
+
("Pariser Platz 1", "10117", "Berlin"),
1127
+
1128
+
# Munich
1129
+
("Marienplatz 8", "80331", "München"),
1130
+
("Leopoldstraße 1", "80802", "München"),
1131
+
1132
+
# Hamburg
1133
+
("Rathausmarkt 1", "20095", "Hamburg"),
1134
+
("Jungfernstieg 1", "20095", "Hamburg"),
1135
+
1136
+
# Cologne
1137
+
("Rathausplatz 2", "50667", "Köln"),
1138
+
1139
+
# Frankfurt
1140
+
("Römerberg 27", "60311", "Frankfurt am Main"),
1141
+
]
1142
+
1143
+
def handle(self, *args, **options):
1144
+
self.stdout.write(self.style.SUCCESS("Testing Address Matching\n"))
1145
+
1146
+
for street, plz, city in self.TEST_ADDRESSES:
1147
+
self.stdout.write(f"\n{street}, {plz} {city}")
1148
+
self.stdout.write("-" * 60)
1149
+
1150
+
# Test geocoding
1151
+
coords = AddressGeocoder.geocode(street, plz, city)
1152
+
if coords:
1153
+
lat, lng = coords
1154
+
self.stdout.write(f" Coordinates: {lat:.6f}, {lng:.6f}")
1155
+
1156
+
# Test constituency lookup
1157
+
constituency = WahlkreisLocator.find_constituency(lat, lng)
1158
+
if constituency:
1159
+
self.stdout.write(self.style.SUCCESS(
1160
+
f" ✓ Constituency: {constituency.name} (ID: {constituency.external_id})"
1161
+
))
1162
+
else:
1163
+
self.stdout.write(self.style.WARNING(" ⚠ No constituency found"))
1164
+
else:
1165
+
self.stdout.write(self.style.ERROR(" ✗ Geocoding failed"))
1166
+
1167
+
# Small delay to respect rate limits
1168
+
import time
1169
+
time.sleep(1.1)
1170
+
1171
+
self.stdout.write("\n" + self.style.SUCCESS("Testing complete!"))
1172
+
```
1173
+
1174
+
**Step 2: Run test command**
1175
+
1176
+
Run: `uv run python manage.py test_address_matching`
1177
+
Expected: Shows results for 8 test addresses
1178
+
1179
+
**Step 3: Review results and fix any issues**
1180
+
1181
+
Check that:
1182
+
- All addresses geocode successfully
1183
+
- Constituencies are found for each address
1184
+
- Results match expected Wahlkreise
1185
+
1186
+
**Step 4: Commit**
1187
+
1188
+
```bash
1189
+
git add website/letters/management/commands/test_address_matching.py
1190
+
git commit -m "feat: add management command to test address matching"
1191
+
```
1192
+
1193
+
---
1194
+
1195
+
## Task 9: Performance Optimization and Monitoring
1196
+
1197
+
**Files:**
1198
+
- Modify: `website/letters/services.py` (add metrics/monitoring)
1199
+
- Create: `website/letters/middleware.py` (optional)
1200
+
1201
+
**Step 1: Add logging for matching performance**
1202
+
1203
+
Add to `WahlkreisLocator.find_constituency`:
1204
+
1205
+
```python
1206
+
import time
1207
+
1208
+
@classmethod
1209
+
def find_constituency(cls, latitude: float, longitude: float) -> Optional[Constituency]:
1210
+
start_time = time.time()
1211
+
1212
+
cls._load_geojson()
1213
+
1214
+
# ... existing implementation ...
1215
+
1216
+
elapsed = time.time() - start_time
1217
+
logger.info(f"Constituency lookup took {elapsed*1000:.1f}ms")
1218
+
1219
+
return result
1220
+
```
1221
+
1222
+
**Step 2: Add cache warming on startup**
1223
+
1224
+
Add Django app ready hook to pre-load GeoJSON:
1225
+
1226
+
Modify `website/letters/apps.py`:
1227
+
1228
+
```python
1229
+
from django.apps import AppConfig
1230
+
1231
+
1232
+
class LettersConfig(AppConfig):
1233
+
default_auto_field = 'django.db.models.BigAutoField'
1234
+
name = 'letters'
1235
+
1236
+
def ready(self):
1237
+
"""Pre-load GeoJSON data on startup."""
1238
+
from .services import WahlkreisLocator
1239
+
WahlkreisLocator._load_geojson()
1240
+
```
1241
+
1242
+
**Step 3: Test performance**
1243
+
1244
+
Run: `uv run python -m django shell`
1245
+
1246
+
```python
1247
+
from letters.services import WahlkreisLocator
1248
+
import time
1249
+
1250
+
# Test lookup performance
1251
+
start = time.time()
1252
+
result = WahlkreisLocator.find_constituency(52.5170365, 13.3888599)
1253
+
elapsed = time.time() - start
1254
+
1255
+
print(f"Lookup took {elapsed*1000:.1f}ms")
1256
+
print(f"Found: {result.name if result else 'None'}")
1257
+
```
1258
+
1259
+
Expected: < 50ms per lookup
1260
+
1261
+
**Step 4: Commit**
1262
+
1263
+
```bash
1264
+
git add website/letters/services.py website/letters/apps.py
1265
+
git commit -m "perf: optimize constituency lookup with startup cache warming"
1266
+
```
1267
+
1268
+
---
1269
+
1270
+
## Task 10: Documentation and README
1271
+
1272
+
**Files:**
1273
+
- Modify: `README.md`
1274
+
- Create: `docs/matching-algorithm.md`
1275
+
1276
+
**Step 1: Document matching algorithm**
1277
+
1278
+
Create `docs/matching-algorithm.md`:
1279
+
1280
+
```markdown
1281
+
# Constituency Matching Algorithm
1282
+
1283
+
## Overview
1284
+
1285
+
WriteThem.eu uses a two-stage process to match users to their correct Bundestag constituency:
1286
+
1287
+
1. **Address Geocoding**: Convert user's address to latitude/longitude coordinates
1288
+
2. **Point-in-Polygon Lookup**: Find which constituency polygon contains those coordinates
1289
+
1290
+
## Stage 1: Address Geocoding
1291
+
1292
+
We use OpenStreetMap's Nominatim API to convert addresses to coordinates.
1293
+
1294
+
### Process:
1295
+
1. User provides: Street, Postal Code, City
1296
+
2. System checks cache (GeocodeCache table) for previous results
1297
+
3. If not cached, query Nominatim API with rate limiting (1 req/sec)
1298
+
4. Cache result (success or failure) to minimize API calls
1299
+
5. Return (latitude, longitude) or None
1300
+
1301
+
### Fallback:
1302
+
If geocoding fails or user only provides postal code, fall back to PLZ prefix heuristic (maps first 2 digits to state).
1303
+
1304
+
## Stage 2: Point-in-Polygon Lookup
1305
+
1306
+
We use official Bundestag constituency boundaries (GeoJSON format) with shapely for geometric queries.
1307
+
1308
+
### Process:
1309
+
1. Load GeoJSON with 299 Bundestag constituencies on startup
1310
+
2. Create shapely Point from coordinates
1311
+
3. Check which constituency Polygon contains the point
1312
+
4. Look up Constituency object in database by external_id
1313
+
5. Return Constituency or None
1314
+
1315
+
### Performance:
1316
+
- GeoJSON loaded once at startup (~2MB in memory)
1317
+
- Lookup typically takes 10-50ms
1318
+
- No external API calls required
1319
+
1320
+
## Data Sources
1321
+
1322
+
- **Constituency Boundaries**: [dknx01/wahlkreissuche](https://github.com/dknx01/wahlkreissuche) (Open Data)
1323
+
- **Geocoding**: [OpenStreetMap Nominatim](https://nominatim.openstreetmap.org/) (Open Data)
1324
+
- **Representative Data**: [Abgeordnetenwatch API](https://www.abgeordnetenwatch.de/api)
1325
+
1326
+
## Accuracy
1327
+
1328
+
This approach provides constituency-accurate matching (exact Wahlkreis), significantly more precise than PLZ-based heuristics which only provide state-level accuracy.
1329
+
1330
+
### Known Limitations:
1331
+
- Requires valid German address
1332
+
- Dependent on OSM geocoding quality
1333
+
- Rate limited to 1 request/second (public API)
1334
+
```
1335
+
1336
+
**Step 2: Update README with setup instructions**
1337
+
1338
+
Add to `README.md`:
1339
+
1340
+
```markdown
1341
+
## Setup: Constituency Matching
1342
+
1343
+
WriteThem.eu uses accurate address-based constituency matching. Setup requires two steps:
1344
+
1345
+
### 1. Download Constituency Boundaries
1346
+
1347
+
```bash
1348
+
uv run python manage.py fetch_wahlkreis_data
1349
+
```
1350
+
1351
+
This downloads ~2MB of GeoJSON data containing official Bundestag constituency boundaries.
1352
+
1353
+
### 2. Test Matching
1354
+
1355
+
Test the matching system with sample addresses:
1356
+
1357
+
```bash
1358
+
uv run python manage.py test_address_matching
1359
+
```
1360
+
1361
+
You should see successful geocoding and constituency detection for major German cities.
1362
+
1363
+
### Configuration
1364
+
1365
+
Set in your environment or settings:
1366
+
1367
+
```python
1368
+
# Optional: Use self-hosted Nominatim (recommended for production)
1369
+
NOMINATIM_URL = 'https://your-nominatim-server.com/search'
1370
+
1371
+
# Optional: Custom GeoJSON path
1372
+
CONSTITUENCY_BOUNDARIES_PATH = '/path/to/wahlkreise.geojson'
1373
+
```
1374
+
1375
+
See `docs/matching-algorithm.md` for technical details.
1376
+
```
1377
+
1378
+
**Step 3: Commit**
1379
+
1380
+
```bash
1381
+
git add README.md docs/matching-algorithm.md
1382
+
git commit -m "docs: document constituency matching algorithm"
1383
+
```
1384
+
1385
+
---
1386
+
1387
+
## Plan Complete
1388
+
1389
+
**Total Implementation Time: ~5-8 hours** (experienced developer, TDD approach)
1390
+
1391
+
**Testing Checklist:**
1392
+
- [ ] All unit tests pass
1393
+
- [ ] Integration tests pass
1394
+
- [ ] Manual testing with 10+ real addresses
1395
+
- [ ] Performance < 100ms end-to-end
1396
+
- [ ] Geocoding cache reducing API calls
1397
+
1398
+
**Next Steps:**
1399
+
Run full test suite:
1400
+
```bash
1401
+
uv run python manage.py test letters
1402
+
```
1403
+
1404
+
Expected: All tests pass (20+ existing tests + ~15 new tests = 35+ total)
1405
+
1406
+
**Deployment Notes:**
1407
+
- Download GeoJSON as part of deployment process
1408
+
- Consider self-hosted Nominatim for production (no rate limits)
1409
+
- Monitor geocoding cache hit rate
1410
+
- Set up alerts for geocoding failures
1411
+
1412
+
---
1413
+
1414
+
This plan implements **Week 1-2, Track 1 (Days 1-5)** from the MVP roadmap. After completing this, proceed to Track 2 (UX Polish) tasks.
+1102
docs/plans/2025-10-14-refactor-test-commands.md
+1102
docs/plans/2025-10-14-refactor-test-commands.md
···
···
1
+
# Refactor Test Commands Implementation Plan
2
+
3
+
> **For Claude:** Use `${SUPERPOWERS_SKILLS_ROOT}/skills/collaboration/executing-plans/SKILL.md` to implement this plan task-by-task.
4
+
5
+
**Goal:** Replace test-like management commands with proper Django tests and create new query commands for interactive debugging.
6
+
7
+
**Architecture:** Extract testing logic from three management commands (`test_matching.py`, `test_constituency_suggestion.py`, `test_topic_mapping.py`) into proper Django test files, then create three new query commands (`query_wahlkreis`, `query_topics`, `query_representatives`) for interactive use, and finally delete the old test commands.
8
+
9
+
**Tech Stack:** Django 5.2, Python 3.13, uv, Django test framework
10
+
11
+
---
12
+
13
+
## Task 1: Create test_address_matching.py test file
14
+
15
+
**Files:**
16
+
- Create: `website/letters/tests/test_address_matching.py`
17
+
- Reference: `website/letters/management/commands/test_matching.py` (for test data and logic)
18
+
19
+
**Step 1: Write the test file structure with TEST_ADDRESSES fixture**
20
+
21
+
Create `website/letters/tests/test_address_matching.py`:
22
+
23
+
```python
24
+
# ABOUTME: Test address-based constituency matching with geocoding and point-in-polygon lookup.
25
+
# ABOUTME: Covers AddressGeocoder, WahlkreisLocator, and ConstituencyLocator services.
26
+
27
+
from django.test import TestCase
28
+
from unittest.mock import patch, MagicMock
29
+
from letters.services import AddressGeocoder, WahlkreisLocator, ConstituencyLocator
30
+
from letters.models import GeocodeCache, Representative
31
+
32
+
33
+
# Test addresses covering all German states
34
+
TEST_ADDRESSES = [
35
+
{
36
+
'name': 'Bundestag (Berlin)',
37
+
'street': 'Platz der Republik 1',
38
+
'postal_code': '11011',
39
+
'city': 'Berlin',
40
+
'expected_state': 'Berlin'
41
+
},
42
+
{
43
+
'name': 'Hamburg Rathaus',
44
+
'street': 'Rathausmarkt 1',
45
+
'postal_code': '20095',
46
+
'city': 'Hamburg',
47
+
'expected_state': 'Hamburg'
48
+
},
49
+
{
50
+
'name': 'Marienplatz München (Bavaria)',
51
+
'street': 'Marienplatz 1',
52
+
'postal_code': '80331',
53
+
'city': 'München',
54
+
'expected_state': 'Bayern'
55
+
},
56
+
{
57
+
'name': 'Kölner Dom (North Rhine-Westphalia)',
58
+
'street': 'Domkloster 4',
59
+
'postal_code': '50667',
60
+
'city': 'Köln',
61
+
'expected_state': 'Nordrhein-Westfalen'
62
+
},
63
+
{
64
+
'name': 'Brandenburger Tor (Berlin)',
65
+
'street': 'Pariser Platz',
66
+
'postal_code': '10117',
67
+
'city': 'Berlin',
68
+
'expected_state': 'Berlin'
69
+
},
70
+
]
71
+
72
+
73
+
class AddressGeocodingTests(TestCase):
74
+
"""Test address geocoding with OSM Nominatim."""
75
+
76
+
def setUp(self):
77
+
self.geocoder = AddressGeocoder()
78
+
79
+
def test_geocode_success_with_mocked_api(self):
80
+
"""Test successful geocoding with mocked Nominatim response."""
81
+
pass
82
+
83
+
def test_geocode_caches_results(self):
84
+
"""Test that geocoding results are cached in database."""
85
+
pass
86
+
87
+
def test_geocode_returns_cached_results(self):
88
+
"""Test that cached geocoding results are reused."""
89
+
pass
90
+
91
+
def test_geocode_handles_api_error(self):
92
+
"""Test graceful handling of Nominatim API errors."""
93
+
pass
94
+
95
+
96
+
class WahlkreisLocationTests(TestCase):
97
+
"""Test point-in-polygon constituency matching."""
98
+
99
+
def test_locate_bundestag_coordinates(self):
100
+
"""Test that Bundestag coordinates find correct Berlin constituency."""
101
+
pass
102
+
103
+
def test_locate_hamburg_coordinates(self):
104
+
"""Test that Hamburg coordinates find correct constituency."""
105
+
pass
106
+
107
+
def test_coordinates_outside_germany(self):
108
+
"""Test that coordinates outside Germany return None."""
109
+
pass
110
+
111
+
112
+
class FullAddressMatchingTests(TestCase):
113
+
"""Integration tests for full address → constituency → representatives pipeline."""
114
+
115
+
@patch('letters.services.AddressGeocoder.geocode')
116
+
def test_address_to_constituency_pipeline(self, mock_geocode):
117
+
"""Test full pipeline from address to constituency with mocked geocoding."""
118
+
pass
119
+
120
+
def test_plz_fallback_when_geocoding_fails(self):
121
+
"""Test PLZ prefix fallback when geocoding fails."""
122
+
pass
123
+
124
+
125
+
# End of file
126
+
```
127
+
128
+
**Step 2: Run test to verify structure loads**
129
+
130
+
Run: `cd website && uv run python manage.py test letters.tests.test_address_matching`
131
+
Expected: All tests should be discovered and skip (no implementations yet)
132
+
133
+
**Step 3: Commit test file structure**
134
+
135
+
```bash
136
+
git add website/letters/tests/test_address_matching.py
137
+
git commit -m "test: add test_address_matching.py structure with fixtures"
138
+
```
139
+
140
+
---
141
+
142
+
## Task 2: Implement address geocoding tests
143
+
144
+
**Files:**
145
+
- Modify: `website/letters/tests/test_address_matching.py`
146
+
147
+
**Step 1: Implement test_geocode_success_with_mocked_api**
148
+
149
+
In `AddressGeocodingTests` class, replace the pass statement:
150
+
151
+
```python
152
+
def test_geocode_success_with_mocked_api(self):
153
+
"""Test successful geocoding with mocked Nominatim response."""
154
+
with patch('requests.get') as mock_get:
155
+
# Mock successful Nominatim response
156
+
mock_response = MagicMock()
157
+
mock_response.status_code = 200
158
+
mock_response.json.return_value = [{
159
+
'lat': '52.5186',
160
+
'lon': '13.3761'
161
+
}]
162
+
mock_get.return_value = mock_response
163
+
164
+
lat, lon, success, error = self.geocoder.geocode(
165
+
'Platz der Republik 1',
166
+
'11011',
167
+
'Berlin'
168
+
)
169
+
170
+
self.assertTrue(success)
171
+
self.assertIsNone(error)
172
+
self.assertAlmostEqual(lat, 52.5186, places=4)
173
+
self.assertAlmostEqual(lon, 13.3761, places=4)
174
+
```
175
+
176
+
**Step 2: Implement test_geocode_caches_results**
177
+
178
+
```python
179
+
def test_geocode_caches_results(self):
180
+
"""Test that geocoding results are cached in database."""
181
+
with patch('requests.get') as mock_get:
182
+
mock_response = MagicMock()
183
+
mock_response.status_code = 200
184
+
mock_response.json.return_value = [{
185
+
'lat': '52.5186',
186
+
'lon': '13.3761'
187
+
}]
188
+
mock_get.return_value = mock_response
189
+
190
+
# First call should cache
191
+
self.geocoder.geocode('Platz der Republik 1', '11011', 'Berlin')
192
+
193
+
# Check cache entry exists
194
+
cache_key = self.geocoder._generate_cache_key(
195
+
'Platz der Republik 1', '11011', 'Berlin', 'DE'
196
+
)
197
+
cache_entry = GeocodeCache.objects.filter(address_hash=cache_key).first()
198
+
self.assertIsNotNone(cache_entry)
199
+
self.assertTrue(cache_entry.success)
200
+
```
201
+
202
+
**Step 3: Implement test_geocode_returns_cached_results**
203
+
204
+
```python
205
+
def test_geocode_returns_cached_results(self):
206
+
"""Test that cached geocoding results are reused."""
207
+
# Create cache entry
208
+
cache_key = self.geocoder._generate_cache_key(
209
+
'Test Street', '12345', 'Test City', 'DE'
210
+
)
211
+
GeocodeCache.objects.create(
212
+
address_hash=cache_key,
213
+
success=True,
214
+
latitude=52.0,
215
+
longitude=13.0
216
+
)
217
+
218
+
# Should return cached result without API call
219
+
with patch('requests.get') as mock_get:
220
+
lat, lon, success, error = self.geocoder.geocode(
221
+
'Test Street', '12345', 'Test City'
222
+
)
223
+
224
+
# Verify no API call was made
225
+
mock_get.assert_not_called()
226
+
227
+
# Verify cached results returned
228
+
self.assertTrue(success)
229
+
self.assertEqual(lat, 52.0)
230
+
self.assertEqual(lon, 13.0)
231
+
```
232
+
233
+
**Step 4: Implement test_geocode_handles_api_error**
234
+
235
+
```python
236
+
def test_geocode_handles_api_error(self):
237
+
"""Test graceful handling of Nominatim API errors."""
238
+
with patch('requests.get') as mock_get:
239
+
mock_get.side_effect = Exception("API Error")
240
+
241
+
lat, lon, success, error = self.geocoder.geocode(
242
+
'Invalid Street', '99999', 'Nowhere'
243
+
)
244
+
245
+
self.assertFalse(success)
246
+
self.assertIsNone(lat)
247
+
self.assertIsNone(lon)
248
+
self.assertIn('API Error', error)
249
+
```
250
+
251
+
**Step 5: Run tests to verify they pass**
252
+
253
+
Run: `cd website && uv run python manage.py test letters.tests.test_address_matching.AddressGeocodingTests -v`
254
+
Expected: 4 tests PASS
255
+
256
+
**Step 6: Commit geocoding tests**
257
+
258
+
```bash
259
+
git add website/letters/tests/test_address_matching.py
260
+
git commit -m "test: implement address geocoding tests with mocking"
261
+
```
262
+
263
+
---
264
+
265
+
## Task 3: Implement Wahlkreis location tests
266
+
267
+
**Files:**
268
+
- Modify: `website/letters/tests/test_address_matching.py`
269
+
270
+
**Step 1: Implement test_locate_bundestag_coordinates**
271
+
272
+
In `WahlkreisLocationTests` class:
273
+
274
+
```python
275
+
def test_locate_bundestag_coordinates(self):
276
+
"""Test that Bundestag coordinates find correct Berlin constituency."""
277
+
locator = WahlkreisLocator()
278
+
result = locator.locate(52.5186, 13.3761)
279
+
280
+
self.assertIsNotNone(result)
281
+
wkr_nr, wkr_name, land_name = result
282
+
self.assertIsInstance(wkr_nr, int)
283
+
self.assertIn('Berlin', land_name)
284
+
```
285
+
286
+
**Step 2: Implement test_locate_hamburg_coordinates**
287
+
288
+
```python
289
+
def test_locate_hamburg_coordinates(self):
290
+
"""Test that Hamburg coordinates find correct constituency."""
291
+
locator = WahlkreisLocator()
292
+
result = locator.locate(53.5511, 9.9937)
293
+
294
+
self.assertIsNotNone(result)
295
+
wkr_nr, wkr_name, land_name = result
296
+
self.assertIsInstance(wkr_nr, int)
297
+
self.assertIn('Hamburg', land_name)
298
+
```
299
+
300
+
**Step 3: Implement test_coordinates_outside_germany**
301
+
302
+
```python
303
+
def test_coordinates_outside_germany(self):
304
+
"""Test that coordinates outside Germany return None."""
305
+
locator = WahlkreisLocator()
306
+
307
+
# Paris coordinates
308
+
result = locator.locate(48.8566, 2.3522)
309
+
self.assertIsNone(result)
310
+
311
+
# London coordinates
312
+
result = locator.locate(51.5074, -0.1278)
313
+
self.assertIsNone(result)
314
+
```
315
+
316
+
**Step 4: Run tests to verify they pass**
317
+
318
+
Run: `cd website && uv run python manage.py test letters.tests.test_address_matching.WahlkreisLocationTests -v`
319
+
Expected: 3 tests PASS
320
+
321
+
**Step 5: Commit Wahlkreis location tests**
322
+
323
+
```bash
324
+
git add website/letters/tests/test_address_matching.py
325
+
git commit -m "test: implement Wahlkreis point-in-polygon location tests"
326
+
```
327
+
328
+
---
329
+
330
+
## Task 4: Implement full address matching integration tests
331
+
332
+
**Files:**
333
+
- Modify: `website/letters/tests/test_address_matching.py`
334
+
335
+
**Step 1: Implement test_address_to_constituency_pipeline**
336
+
337
+
In `FullAddressMatchingTests` class:
338
+
339
+
```python
340
+
@patch('letters.services.AddressGeocoder.geocode')
341
+
def test_address_to_constituency_pipeline(self, mock_geocode):
342
+
"""Test full pipeline from address to constituency with mocked geocoding."""
343
+
# Mock geocoding to return Bundestag coordinates
344
+
mock_geocode.return_value = (52.5186, 13.3761, True, None)
345
+
346
+
locator = ConstituencyLocator()
347
+
representatives = locator.locate(
348
+
street='Platz der Republik 1',
349
+
postal_code='11011',
350
+
city='Berlin'
351
+
)
352
+
353
+
# Should return representatives (even if list is empty due to no DB data)
354
+
self.assertIsInstance(representatives, list)
355
+
mock_geocode.assert_called_once()
356
+
```
357
+
358
+
**Step 2: Implement test_plz_fallback_when_geocoding_fails**
359
+
360
+
```python
361
+
def test_plz_fallback_when_geocoding_fails(self):
362
+
"""Test PLZ prefix fallback when geocoding fails."""
363
+
with patch('letters.services.AddressGeocoder.geocode') as mock_geocode:
364
+
# Mock geocoding failure
365
+
mock_geocode.return_value = (None, None, False, "Geocoding failed")
366
+
367
+
locator = ConstituencyLocator()
368
+
representatives = locator.locate(
369
+
postal_code='10115' # Berlin postal code
370
+
)
371
+
372
+
# Should still return list (using PLZ fallback)
373
+
self.assertIsInstance(representatives, list)
374
+
```
375
+
376
+
**Step 3: Run tests to verify they pass**
377
+
378
+
Run: `cd website && uv run python manage.py test letters.tests.test_address_matching.FullAddressMatchingTests -v`
379
+
Expected: 2 tests PASS
380
+
381
+
**Step 4: Run full test suite**
382
+
383
+
Run: `cd website && uv run python manage.py test letters.tests.test_address_matching -v`
384
+
Expected: All 9 tests PASS
385
+
386
+
**Step 5: Commit integration tests**
387
+
388
+
```bash
389
+
git add website/letters/tests/test_address_matching.py
390
+
git commit -m "test: implement full address matching integration tests"
391
+
```
392
+
393
+
---
394
+
395
+
## Task 5: Create test_topic_mapping.py test file
396
+
397
+
**Files:**
398
+
- Create: `website/letters/tests/test_topic_mapping.py`
399
+
- Reference: `website/letters/management/commands/test_topic_mapping.py` (for test data)
400
+
401
+
**Step 1: Write test file with topic matching tests**
402
+
403
+
Create `website/letters/tests/test_topic_mapping.py`:
404
+
405
+
```python
406
+
# ABOUTME: Test topic suggestion and matching based on letter content.
407
+
# ABOUTME: Covers TopicSuggestionService keyword matching and level suggestion logic.
408
+
409
+
from django.test import TestCase
410
+
from letters.services import TopicSuggestionService
411
+
from letters.models import TopicArea
412
+
413
+
414
+
class TopicMatchingTests(TestCase):
415
+
"""Test topic keyword matching and scoring."""
416
+
417
+
def test_transport_keywords_match_verkehr_topic(self):
418
+
"""Test that transport-related keywords match Verkehr topic."""
419
+
concern = "I want to see better train connections between cities"
420
+
topics = TopicSuggestionService.get_topic_suggestions(concern)
421
+
422
+
# Should find at least one topic
423
+
self.assertGreater(len(topics), 0)
424
+
425
+
# Top topic should be transport-related
426
+
top_topic = topics[0]
427
+
self.assertIn('score', top_topic)
428
+
self.assertGreater(top_topic['score'], 0)
429
+
430
+
def test_housing_keywords_match_wohnen_topic(self):
431
+
"""Test that housing keywords match Wohnen topic."""
432
+
concern = "We need more affordable housing and rent control"
433
+
topics = TopicSuggestionService.get_topic_suggestions(concern)
434
+
435
+
self.assertGreater(len(topics), 0)
436
+
437
+
def test_education_keywords_match_bildung_topic(self):
438
+
"""Test that education keywords match Bildung topic."""
439
+
concern = "Our school curriculum needs reform"
440
+
topics = TopicSuggestionService.get_topic_suggestions(concern)
441
+
442
+
self.assertGreater(len(topics), 0)
443
+
444
+
def test_climate_keywords_match_umwelt_topic(self):
445
+
"""Test that climate keywords match environment topic."""
446
+
concern = "Climate protection and CO2 emissions must be addressed"
447
+
topics = TopicSuggestionService.get_topic_suggestions(concern)
448
+
449
+
self.assertGreater(len(topics), 0)
450
+
451
+
def test_no_match_returns_empty_list(self):
452
+
"""Test that completely unrelated text returns empty list."""
453
+
concern = "xyzabc nonsense gibberish"
454
+
topics = TopicSuggestionService.get_topic_suggestions(concern)
455
+
456
+
# May return empty or very low scores
457
+
if topics:
458
+
self.assertLess(topics[0]['score'], 0.3)
459
+
460
+
461
+
class LevelSuggestionTests(TestCase):
462
+
"""Test government level suggestion logic."""
463
+
464
+
def test_federal_transport_suggests_federal_level(self):
465
+
"""Test that long-distance transport suggests federal level."""
466
+
result = TopicSuggestionService.suggest_representatives_for_concern(
467
+
"Deutsche Bahn is always late",
468
+
limit=5
469
+
)
470
+
471
+
self.assertIn('suggested_level', result)
472
+
self.assertIn('explanation', result)
473
+
# Federal issues should suggest Bundestag
474
+
self.assertIn('Bundestag', result['suggested_level'])
475
+
476
+
def test_local_bus_suggests_state_or_local(self):
477
+
"""Test that local transport suggests state/local level."""
478
+
result = TopicSuggestionService.suggest_representatives_for_concern(
479
+
"Better bus services in my town",
480
+
limit=5
481
+
)
482
+
483
+
self.assertIn('suggested_level', result)
484
+
# Local issues should not exclusively suggest federal
485
+
explanation = result['explanation'].lower()
486
+
self.assertTrue('state' in explanation or 'local' in explanation or 'land' in explanation)
487
+
488
+
489
+
# End of file
490
+
```
491
+
492
+
**Step 2: Run tests to verify they work**
493
+
494
+
Run: `cd website && uv run python manage.py test letters.tests.test_topic_mapping -v`
495
+
Expected: Tests PASS (some may be skipped if TopicArea data not loaded)
496
+
497
+
**Step 3: Commit topic mapping tests**
498
+
499
+
```bash
500
+
git add website/letters/tests/test_topic_mapping.py
501
+
git commit -m "test: add topic matching and level suggestion tests"
502
+
```
503
+
504
+
---
505
+
506
+
## Task 6: Create test_constituency_suggestions.py test file
507
+
508
+
**Files:**
509
+
- Create: `website/letters/tests/test_constituency_suggestions.py`
510
+
- Reference: `website/letters/management/commands/test_constituency_suggestion.py`
511
+
512
+
**Step 1: Write test file for constituency suggestion service**
513
+
514
+
Create `website/letters/tests/test_constituency_suggestions.py`:
515
+
516
+
```python
517
+
# ABOUTME: Test ConstituencySuggestionService combining topics and geography.
518
+
# ABOUTME: Integration tests for letter title/address to representative suggestions.
519
+
520
+
from django.test import TestCase
521
+
from unittest.mock import patch
522
+
from letters.services import ConstituencySuggestionService
523
+
524
+
525
+
class ConstituencySuggestionTests(TestCase):
526
+
"""Test constituency suggestion combining topic and address matching."""
527
+
528
+
@patch('letters.services.AddressGeocoder.geocode')
529
+
def test_suggest_with_title_and_address(self, mock_geocode):
530
+
"""Test suggestions work with both title and address."""
531
+
# Mock geocoding
532
+
mock_geocode.return_value = (52.5186, 13.3761, True, None)
533
+
534
+
result = ConstituencySuggestionService.suggest_from_concern(
535
+
concern="We need better train connections",
536
+
street="Platz der Republik 1",
537
+
postal_code="11011",
538
+
city="Berlin"
539
+
)
540
+
541
+
self.assertIn('matched_topics', result)
542
+
self.assertIn('suggested_level', result)
543
+
self.assertIn('explanation', result)
544
+
self.assertIn('representatives', result)
545
+
self.assertIn('constituencies', result)
546
+
547
+
def test_suggest_with_only_title(self):
548
+
"""Test suggestions work with only title (no address)."""
549
+
result = ConstituencySuggestionService.suggest_from_concern(
550
+
concern="Climate protection is important"
551
+
)
552
+
553
+
self.assertIn('matched_topics', result)
554
+
self.assertIn('suggested_level', result)
555
+
# Without address, should still suggest level and topics
556
+
self.assertIsNotNone(result['suggested_level'])
557
+
558
+
def test_suggest_with_only_postal_code(self):
559
+
"""Test suggestions work with only postal code."""
560
+
result = ConstituencySuggestionService.suggest_from_concern(
561
+
concern="Local infrastructure problems",
562
+
postal_code="10115"
563
+
)
564
+
565
+
self.assertIn('constituencies', result)
566
+
# Should use PLZ fallback
567
+
self.assertIsInstance(result['constituencies'], list)
568
+
569
+
570
+
# End of file
571
+
```
572
+
573
+
**Step 2: Run tests to verify they pass**
574
+
575
+
Run: `cd website && uv run python manage.py test letters.tests.test_constituency_suggestions -v`
576
+
Expected: 3 tests PASS
577
+
578
+
**Step 3: Commit constituency suggestion tests**
579
+
580
+
```bash
581
+
git add website/letters/tests/test_constituency_suggestions.py
582
+
git commit -m "test: add constituency suggestion integration tests"
583
+
```
584
+
585
+
---
586
+
587
+
## Task 7: Create query_wahlkreis management command
588
+
589
+
**Files:**
590
+
- Create: `website/letters/management/commands/query_wahlkreis.py`
591
+
592
+
**Step 1: Write query_wahlkreis command**
593
+
594
+
Create `website/letters/management/commands/query_wahlkreis.py`:
595
+
596
+
```python
597
+
# ABOUTME: Query management command to find constituency by address or postal code.
598
+
# ABOUTME: Interactive tool for testing address-based constituency matching.
599
+
600
+
from django.core.management.base import BaseCommand
601
+
from letters.services import AddressGeocoder, WahlkreisLocator, ConstituencyLocator
602
+
603
+
604
+
class Command(BaseCommand):
605
+
help = 'Find constituency (Wahlkreis) by address or postal code'
606
+
607
+
def add_arguments(self, parser):
608
+
parser.add_argument(
609
+
'--street',
610
+
type=str,
611
+
help='Street name and number'
612
+
)
613
+
parser.add_argument(
614
+
'--postal-code',
615
+
type=str,
616
+
help='Postal code (PLZ)',
617
+
required=True
618
+
)
619
+
parser.add_argument(
620
+
'--city',
621
+
type=str,
622
+
help='City name'
623
+
)
624
+
625
+
def handle(self, *args, **options):
626
+
street = options.get('street')
627
+
postal_code = options['postal_code']
628
+
city = options.get('city')
629
+
630
+
try:
631
+
# Try full address geocoding if all parts provided
632
+
if street and city:
633
+
geocoder = AddressGeocoder()
634
+
lat, lon, success, error = geocoder.geocode(street, postal_code, city)
635
+
636
+
if not success:
637
+
self.stdout.write(self.style.ERROR(f'Error: Could not geocode address: {error}'))
638
+
return
639
+
640
+
locator = WahlkreisLocator()
641
+
result = locator.locate(lat, lon)
642
+
643
+
if not result:
644
+
self.stdout.write('No constituency found for these coordinates')
645
+
return
646
+
647
+
wkr_nr, wkr_name, land_name = result
648
+
self.stdout.write(f'WK {wkr_nr:03d} - {wkr_name} ({land_name})')
649
+
650
+
# Fallback to PLZ prefix lookup
651
+
else:
652
+
from letters.constants import PLZ_TO_STATE
653
+
plz_prefix = postal_code[:2]
654
+
655
+
if plz_prefix in PLZ_TO_STATE:
656
+
state = PLZ_TO_STATE[plz_prefix]
657
+
self.stdout.write(f'State: {state} (from postal code prefix)')
658
+
else:
659
+
self.stdout.write('Error: Could not determine state from postal code')
660
+
661
+
except Exception as e:
662
+
self.stderr.write(self.style.ERROR(f'Error: {str(e)}'))
663
+
return
664
+
```
665
+
666
+
**Step 2: Test the command manually**
667
+
668
+
Run: `cd website && uv run python manage.py query_wahlkreis --street "Platz der Republik 1" --postal-code "11011" --city "Berlin"`
669
+
Expected: Output showing Berlin constituency
670
+
671
+
Run: `cd website && uv run python manage.py query_wahlkreis --postal-code "10115"`
672
+
Expected: Output showing "State: Berlin (from postal code prefix)"
673
+
674
+
**Step 3: Commit query_wahlkreis command**
675
+
676
+
```bash
677
+
git add website/letters/management/commands/query_wahlkreis.py
678
+
git commit -m "feat: add query_wahlkreis management command"
679
+
```
680
+
681
+
---
682
+
683
+
## Task 8: Create query_topics management command
684
+
685
+
**Files:**
686
+
- Create: `website/letters/management/commands/query_topics.py`
687
+
688
+
**Step 1: Write query_topics command**
689
+
690
+
Create `website/letters/management/commands/query_topics.py`:
691
+
692
+
```python
693
+
# ABOUTME: Query management command to find matching topics for letter text.
694
+
# ABOUTME: Interactive tool for testing topic keyword matching and scoring.
695
+
696
+
from django.core.management.base import BaseCommand
697
+
from letters.services import TopicSuggestionService
698
+
699
+
700
+
class Command(BaseCommand):
701
+
help = 'Find matching topics for a letter title or text'
702
+
703
+
def add_arguments(self, parser):
704
+
parser.add_argument(
705
+
'--text',
706
+
type=str,
707
+
required=True,
708
+
help='Letter title or text to analyze'
709
+
)
710
+
parser.add_argument(
711
+
'--limit',
712
+
type=int,
713
+
default=5,
714
+
help='Maximum number of topics to return (default: 5)'
715
+
)
716
+
717
+
def handle(self, *args, **options):
718
+
text = options['text']
719
+
limit = options['limit']
720
+
721
+
try:
722
+
topics = TopicSuggestionService.get_topic_suggestions(text)
723
+
724
+
if not topics:
725
+
self.stdout.write('No matching topics found')
726
+
return
727
+
728
+
# Limit results
729
+
topics = topics[:limit]
730
+
731
+
for topic in topics:
732
+
score = topic.get('match_score', topic.get('score', 0))
733
+
self.stdout.write(
734
+
f"{topic['name']} ({topic['level']}, Score: {score:.2f})"
735
+
)
736
+
if 'description' in topic and topic['description']:
737
+
self.stdout.write(f" {topic['description']}")
738
+
739
+
except Exception as e:
740
+
self.stderr.write(self.style.ERROR(f'Error: {str(e)}'))
741
+
return
742
+
```
743
+
744
+
**Step 2: Test the command manually**
745
+
746
+
Run: `cd website && uv run python manage.py query_topics --text "We need better train connections"`
747
+
Expected: Output showing transport-related topics with scores
748
+
749
+
Run: `cd website && uv run python manage.py query_topics --text "affordable housing" --limit 3`
750
+
Expected: Output showing top 3 housing-related topics
751
+
752
+
**Step 3: Commit query_topics command**
753
+
754
+
```bash
755
+
git add website/letters/management/commands/query_topics.py
756
+
git commit -m "feat: add query_topics management command"
757
+
```
758
+
759
+
---
760
+
761
+
## Task 9: Create query_representatives management command
762
+
763
+
**Files:**
764
+
- Create: `website/letters/management/commands/query_representatives.py`
765
+
766
+
**Step 1: Write query_representatives command**
767
+
768
+
Create `website/letters/management/commands/query_representatives.py`:
769
+
770
+
```python
771
+
# ABOUTME: Query management command to find representatives by address and/or topics.
772
+
# ABOUTME: Interactive tool for testing representative suggestion logic.
773
+
774
+
from django.core.management.base import BaseCommand
775
+
from letters.services import ConstituencyLocator, TopicSuggestionService, ConstituencySuggestionService
776
+
777
+
778
+
class Command(BaseCommand):
779
+
help = 'Find representatives by address and/or topics'
780
+
781
+
def add_arguments(self, parser):
782
+
# Address arguments
783
+
parser.add_argument(
784
+
'--street',
785
+
type=str,
786
+
help='Street name and number'
787
+
)
788
+
parser.add_argument(
789
+
'--postal-code',
790
+
type=str,
791
+
help='Postal code (PLZ)'
792
+
)
793
+
parser.add_argument(
794
+
'--city',
795
+
type=str,
796
+
help='City name'
797
+
)
798
+
799
+
# Topic arguments
800
+
parser.add_argument(
801
+
'--topics',
802
+
type=str,
803
+
help='Comma-separated topic keywords (e.g., "Verkehr,Infrastruktur")'
804
+
)
805
+
806
+
parser.add_argument(
807
+
'--limit',
808
+
type=int,
809
+
default=10,
810
+
help='Maximum number of representatives to return (default: 10)'
811
+
)
812
+
813
+
def handle(self, *args, **options):
814
+
street = options.get('street')
815
+
postal_code = options.get('postal_code')
816
+
city = options.get('city')
817
+
topics_str = options.get('topics')
818
+
limit = options['limit']
819
+
820
+
try:
821
+
# Use constituency locator if address provided
822
+
if postal_code or (street and city):
823
+
locator = ConstituencyLocator()
824
+
representatives = locator.locate(
825
+
street=street,
826
+
postal_code=postal_code,
827
+
city=city
828
+
)
829
+
830
+
if not representatives:
831
+
self.stdout.write('No representatives found for this location')
832
+
return
833
+
834
+
# Filter by topics if provided
835
+
if topics_str:
836
+
topic_keywords = [t.strip() for t in topics_str.split(',')]
837
+
# Simple keyword filter on representative focus areas
838
+
filtered_reps = []
839
+
for rep in representatives:
840
+
# Check if any committee or focus area matches
841
+
rep_text = ' '.join([
842
+
rep.full_name,
843
+
' '.join([c.name for c in rep.committees.all()]),
844
+
]).lower()
845
+
846
+
if any(keyword.lower() in rep_text for keyword in topic_keywords):
847
+
filtered_reps.append(rep)
848
+
849
+
representatives = filtered_reps if filtered_reps else representatives
850
+
851
+
# Display results
852
+
for rep in representatives[:limit]:
853
+
constituency = rep.primary_constituency
854
+
constituency_label = constituency.name if constituency else rep.parliament.name
855
+
self.stdout.write(f'{rep.full_name} ({rep.party}) - {constituency_label}')
856
+
857
+
# Show committees
858
+
committees = list(rep.committees.all()[:3])
859
+
if committees:
860
+
committee_names = ', '.join([c.name for c in committees])
861
+
self.stdout.write(f' Committees: {committee_names}')
862
+
863
+
# Use topic-based search if only topics provided
864
+
elif topics_str:
865
+
self.stdout.write('Topic-based representative search not yet implemented')
866
+
self.stdout.write('Please provide at least a postal code for location-based search')
867
+
868
+
else:
869
+
self.stderr.write(self.style.ERROR(
870
+
'Error: Please provide either an address (--postal-code required) or --topics'
871
+
))
872
+
873
+
except Exception as e:
874
+
self.stderr.write(self.style.ERROR(f'Error: {str(e)}'))
875
+
return
876
+
```
877
+
878
+
**Step 2: Test the command manually**
879
+
880
+
Run: `cd website && uv run python manage.py query_representatives --postal-code "11011"`
881
+
Expected: Output showing Berlin representatives
882
+
883
+
Run: `cd website && uv run python manage.py query_representatives --street "Platz der Republik 1" --postal-code "11011" --city "Berlin" --limit 5`
884
+
Expected: Output showing top 5 representatives for that location
885
+
886
+
**Step 3: Commit query_representatives command**
887
+
888
+
```bash
889
+
git add website/letters/management/commands/query_representatives.py
890
+
git commit -m "feat: add query_representatives management command"
891
+
```
892
+
893
+
---
894
+
895
+
## Task 10: Run full test suite and verify everything works
896
+
897
+
**Files:**
898
+
- All test files
899
+
900
+
**Step 1: Run complete test suite**
901
+
902
+
Run: `cd website && uv run python manage.py test`
903
+
Expected: All tests PASS (including new and existing tests)
904
+
905
+
**Step 2: Test all three query commands manually**
906
+
907
+
Run: `cd website && uv run python manage.py query_wahlkreis --street "Platz der Republik 1" --postal-code "11011" --city "Berlin"`
908
+
Expected: Correct constituency output
909
+
910
+
Run: `cd website && uv run python manage.py query_topics --text "climate change and renewable energy"`
911
+
Expected: Environment-related topics
912
+
913
+
Run: `cd website && uv run python manage.py query_representatives --postal-code "10115"`
914
+
Expected: Berlin representatives
915
+
916
+
**Step 3: Commit if any fixes needed**
917
+
918
+
If any issues found and fixed:
919
+
```bash
920
+
git add .
921
+
git commit -m "fix: address test suite issues"
922
+
```
923
+
924
+
---
925
+
926
+
## Task 11: Delete test_matching.py command
927
+
928
+
**Files:**
929
+
- Delete: `website/letters/management/commands/test_matching.py`
930
+
931
+
**Step 1: Verify tests cover all test_matching.py functionality**
932
+
933
+
Compare `test_matching.py` with `test_address_matching.py` to ensure all test cases are covered.
934
+
935
+
**Step 2: Delete test_matching.py**
936
+
937
+
Run: `rm website/letters/management/commands/test_matching.py`
938
+
939
+
**Step 3: Run tests to verify nothing broke**
940
+
941
+
Run: `cd website && uv run python manage.py test`
942
+
Expected: All tests still PASS
943
+
944
+
**Step 4: Commit deletion**
945
+
946
+
```bash
947
+
git add website/letters/management/commands/test_matching.py
948
+
git commit -m "refactor: remove test_matching command (moved to proper tests)"
949
+
```
950
+
951
+
---
952
+
953
+
## Task 12: Delete test_constituency_suggestion.py command
954
+
955
+
**Files:**
956
+
- Delete: `website/letters/management/commands/test_constituency_suggestion.py`
957
+
958
+
**Step 1: Verify tests cover functionality**
959
+
960
+
Compare with `test_constituency_suggestions.py`.
961
+
962
+
**Step 2: Delete test_constituency_suggestion.py**
963
+
964
+
Run: `rm website/letters/management/commands/test_constituency_suggestion.py`
965
+
966
+
**Step 3: Run tests to verify nothing broke**
967
+
968
+
Run: `cd website && uv run python manage.py test`
969
+
Expected: All tests PASS
970
+
971
+
**Step 4: Commit deletion**
972
+
973
+
```bash
974
+
git add website/letters/management/commands/test_constituency_suggestion.py
975
+
git commit -m "refactor: remove test_constituency_suggestion command (moved to proper tests)"
976
+
```
977
+
978
+
---
979
+
980
+
## Task 13: Delete test_topic_mapping.py command
981
+
982
+
**Files:**
983
+
- Delete: `website/letters/management/commands/test_topic_mapping.py`
984
+
985
+
**Step 1: Verify tests cover functionality**
986
+
987
+
Compare with `test_topic_mapping.py`.
988
+
989
+
**Step 2: Delete test_topic_mapping.py**
990
+
991
+
Run: `rm website/letters/management/commands/test_topic_mapping.py`
992
+
993
+
**Step 3: Run tests to verify nothing broke**
994
+
995
+
Run: `cd website && uv run python manage.py test`
996
+
Expected: All tests PASS
997
+
998
+
**Step 4: Commit deletion**
999
+
1000
+
```bash
1001
+
git add website/letters/management/commands/test_topic_mapping.py
1002
+
git commit -m "refactor: remove test_topic_mapping command (moved to proper tests)"
1003
+
```
1004
+
1005
+
---
1006
+
1007
+
## Task 14: Update documentation
1008
+
1009
+
**Files:**
1010
+
- Modify: `README.md` (if it mentions test commands)
1011
+
- Modify: `docs/matching-algorithm.md` (update command references)
1012
+
1013
+
**Step 1: Check if README mentions test commands**
1014
+
1015
+
Run: `grep -n "test_matching\|test_constituency\|test_topic" README.md`
1016
+
1017
+
If found, update to reference new query commands and proper test suite.
1018
+
1019
+
**Step 2: Update docs/matching-algorithm.md**
1020
+
1021
+
In `docs/matching-algorithm.md`, find section "Management Commands" (around line 70) and update:
1022
+
1023
+
```markdown
1024
+
### Management Commands
1025
+
1026
+
- **fetch_wahlkreis_data**: Downloads official Bundestag constituency boundaries
1027
+
- **query_wahlkreis**: Query constituency by address or postal code
1028
+
- **query_topics**: Find matching topics for letter text
1029
+
- **query_representatives**: Find representatives by address and/or topics
1030
+
1031
+
### Testing
1032
+
1033
+
Run the test suite:
1034
+
```bash
1035
+
python manage.py test letters.tests.test_address_matching
1036
+
python manage.py test letters.tests.test_topic_mapping
1037
+
python manage.py test letters.tests.test_constituency_suggestions
1038
+
```
1039
+
```
1040
+
1041
+
**Step 3: Commit documentation updates**
1042
+
1043
+
```bash
1044
+
git add README.md docs/matching-algorithm.md
1045
+
git commit -m "docs: update command and testing references"
1046
+
```
1047
+
1048
+
---
1049
+
1050
+
## Task 15: Final verification and summary
1051
+
1052
+
**Files:**
1053
+
- All modified files
1054
+
1055
+
**Step 1: Run complete test suite one final time**
1056
+
1057
+
Run: `cd website && uv run python manage.py test -v`
1058
+
Expected: All tests PASS with detailed output
1059
+
1060
+
**Step 2: Verify query commands work**
1061
+
1062
+
Test each command with various inputs to ensure they work correctly.
1063
+
1064
+
**Step 3: Create summary of changes**
1065
+
1066
+
Review all commits:
1067
+
```bash
1068
+
git log --oneline
1069
+
```
1070
+
1071
+
**Step 4: Final commit if needed**
1072
+
1073
+
If any final cleanup needed:
1074
+
```bash
1075
+
git add .
1076
+
git commit -m "chore: final cleanup for test command refactoring"
1077
+
```
1078
+
1079
+
---
1080
+
1081
+
## Summary
1082
+
1083
+
**What was accomplished:**
1084
+
1. Created three new test files with comprehensive test coverage
1085
+
2. Created three new query management commands for interactive debugging
1086
+
3. Deleted three old test-like management commands
1087
+
4. Updated documentation to reflect new structure
1088
+
1089
+
**New query commands:**
1090
+
- `query_wahlkreis` - Find constituency by address/PLZ
1091
+
- `query_topics` - Find matching topics for text
1092
+
- `query_representatives` - Find representatives by location/topics
1093
+
1094
+
**New test files:**
1095
+
- `letters/tests/test_address_matching.py` - Address geocoding and matching
1096
+
- `letters/tests/test_topic_mapping.py` - Topic keyword matching
1097
+
- `letters/tests/test_constituency_suggestions.py` - Integration tests
1098
+
1099
+
**Testing strategy:**
1100
+
- Mocked external API calls (Nominatim) to avoid rate limits
1101
+
- Integration tests use real services where possible
1102
+
- All edge cases covered (failures, fallbacks, empty results)
+60
docs/plans/matching.md
+60
docs/plans/matching.md
···
···
1
+
# Recipient Matching Vision
2
+
3
+
## Goal
4
+
Ensure every letter reaches the most relevant representative by combining precise constituency mapping with topic-awareness and reliable representative metadata.
5
+
6
+
## Core Pillars
7
+
8
+
1. **Constituency Precision**
9
+
- Replace postal-prefix heuristics with official boundary data:
10
+
- Bundestag Wahlkreise (Bundeswahlleiter / BKG GeoJSON)
11
+
- Landtag electoral districts via state open-data portals or OParl feeds
12
+
- EU parliament treated as nationwide constituency
13
+
- Normalise mandate modes:
14
+
- Direktmandat → voters in that Wahlkreis
15
+
- Landesliste → voters in the state
16
+
- Bundes/EU list → national constituencies
17
+
- Centralise the logic in a “constituency router” so each parliament’s data source is pluggable.
18
+
19
+
2. **Topic Understanding**
20
+
- Analyse title + body to classify concerns into a canonical taxonomy (reuse committee topics, extend as needed).
21
+
- Infer the responsible level (EU / Bund / Land / Kommune) from topic metadata.
22
+
- Keep the topic model extensible (keyword heuristics today, embeddings or classifiers tomorrow).
23
+
24
+
3. **Rich Representative Profiles**
25
+
- Build a `RepresentativeProfile` table to store per-source enrichments:
26
+
- Source (ABGEORDNETENWATCH, BUNDESTAG, LANDTAG_*)
27
+
- Normalised fields: focus areas, biography, external links, responsiveness
28
+
- Raw metadata + sync timestamps
29
+
- Importers:
30
+
- Abgeordnetenwatch: `/politicians/{id}` (issues, responsiveness, social links)
31
+
- Bundestag: official vita JSON (`mdbId`) for biography + spokesperson roles
32
+
- Landtage: state-specific data feeds (OParl, CSV, or one-off scrapers)
33
+
- Profiles coexist; the merging service resolves conflicts and picks the best available data.
34
+
35
+
## Matching Pipeline
36
+
1. **Constituency filter**: Use the router and mandate rules to determine eligible reps.
37
+
2. **Topic filter**: Narrow to the inferred level and portfolio.
38
+
3. **Scoring**: Blend signals—constituency proximity, topic match (committee → topic), activity (votes, questions), responsiveness stats, optional user preferences.
39
+
4. **Explanation**: Provide human-readable reasons (“Direct MP for WK 123; sits on Verkehrsausschuss; answered 90% of Abgeordnetenwatch questions”).
40
+
41
+
## Data Sources Reference
42
+
43
+
| Use Case | Federal | State | EU |
44
+
|-------------------------|-------------------------------------|---------------------------------------------|--------------------------------|
45
+
| Mandates & committees | Abgeordnetenwatch API | Abgeordnetenwatch, OParl, Landtag portals | EU Parliament REST API |
46
+
| Constituency boundaries | Bundeswahlleiter GeoJSON, BKG | Landeswahlleitungen, state GIS datasets | Whole-of-Germany (single geom) |
47
+
| Biography / focus | Bundestag vita JSON, Abgeordnetenwatch issues | Landtag bios (open data) | Europarl member profiles |
48
+
49
+
## Implementation Notes
50
+
- Expose `sync_representative_profiles` commands per source; schedule separately from mandate sync.
51
+
- Track `source_version`/`hash` to avoid redundant imports.
52
+
- View layer consumes a `RepresentativeProfileService` that aggregates focus areas, biography, links, responsiveness.
53
+
- Keep a roadmap for future sources (party press, DIP21 votes, Europarl “files”).
54
+
55
+
## Next Steps
56
+
- Implement `RepresentativeProfile` model + importers for Abgeordnetenwatch and Bundestag.
57
+
- Integrate boundary datasets and swap the PLZ router.
58
+
- Wire the matching pipeline into the letter form suggestions and automated routing.
59
+
- Add logging/monitoring for profile freshness and matching success.
60
+
+221
docs/plans/mvp.md
+221
docs/plans/mvp.md
···
···
1
+
# MVP Vision
2
+
3
+
## Mission
4
+
Empower citizens to participate in democracy by writing impactful open letters to the representatives best positioned to act, and by allowing others to rally behind those letters with signatures. Verified identities add credibility; when a letter clears a signature threshold, we commit to printing, signing, and delivering it to the relevant office.
5
+
6
+
## Core Feature Set
7
+
1. **Accounts & Profiles**
8
+
- Email/password registration & login.
9
+
- Profile page showing authored letters, signed letters, and verification status.
10
+
2. **Representative Directory**
11
+
- EU, Bundestag, Landtag members with photo, party, mandate mode, committee roles, Abgeordnetenwatch links.
12
+
- Exposed via detail view and reusable UI card.
13
+
3. **Letter Authoring & Publishing**
14
+
- Draft open letters, auto-suggest recipients based on title + PLZ.
15
+
- Auto-sign on publish; allow editing until first signature.
16
+
- Letter detail page shows full content, representative card, signature stats.
17
+
4. **Recommendation Engine**
18
+
- PLZ → constituency router (direct/state/federal) using official boundary data.
19
+
- Topic analysis highlighting likely responsible level and committee working areas.
20
+
- Explain why a representative is recommended, surface relevant tags, show similar letters.
21
+
5. **Signature Flow**
22
+
- One-click signing for logged in users; prompt login otherwise.
23
+
- Badges for verified vs unverified signatures, count constituents distinctly.
24
+
- Social sharing (link copy, optional Twitter/Bluesky share).
25
+
6. **Identity Verification (Optional)**
26
+
- Integrate one third-party provider (e.g., Verimi or yes®) via OAuth2/OIDC to pull verified name/address.
27
+
- Store attestation + expiry; map address to constituency for direct mandates.
28
+
- Users without verification can still sign, flagged as “unverified.”
29
+
7. **Signature Threshold & Fulfilment**
30
+
- Configurable threshold per letter or representative type.
31
+
- Admin view showing letters reaching milestones, export letter + supporters for printing/mailing.
32
+
8. **Admin & Moderation**
33
+
- Admin dashboard to inspect letters, representatives, signatures, verification status, and signature thresholds.
34
+
- Ability to disable inappropriate letters, resend sync, run exports.
35
+
9. **Landing & Discovery**
36
+
- Public homepage summarising mission, stats, featured letters.
37
+
- Browse letters and representatives without login.
38
+
10. **Documentation & Transparency**
39
+
- Public “How it works” page, privacy policy, terms.
40
+
- README covering setup, architecture, deployment.
41
+
42
+
## 1-Month Sprint to 39C3 (December 2024)
43
+
44
+
### **Week 1-2: Core Functionality** (Days 1-10)
45
+
46
+
#### Track 1: Accurate Constituency Matching (Days 1-5) ⚠️ CRITICAL
47
+
**Day 1: OSM Nominatim Integration**
48
+
- [ ] Set up OSM Nominatim API client (requests-based, with rate limiting)
49
+
- [ ] Add address geocoding service (`AddressGeocoder`)
50
+
- [ ] Cache geocoding results in database to minimize API calls
51
+
- [ ] Write tests for address → lat/lng conversion
52
+
53
+
**Day 2: GeoJSON Point-in-Polygon Lookup**
54
+
- [ ] Download full Bundestag Wahlkreis GeoJSON (via existing management command)
55
+
- [ ] Build `WahlkreisLocator` using shapely for point-in-polygon
56
+
- [ ] Load GeoJSON into memory at startup (or cache in Redis)
57
+
- [ ] Test coordinate → Wahlkreis lookup with sample points
58
+
59
+
**Day 3: Integration & Service Layer**
60
+
- [ ] Replace `ConstituencyLocator` with new address-based lookup
61
+
- [ ] Update `LocationContext` to accept full addresses
62
+
- [ ] Maintain PLZ prefix fallback for partial data
63
+
- [ ] Add comprehensive error handling and logging
64
+
65
+
**Day 4: Representative Matching Validation**
66
+
- [ ] Test matching with 20 real German addresses
67
+
- [ ] Verify direct representatives are correctly suggested
68
+
- [ ] Test topic + geography combined matching
69
+
- [ ] Document matching algorithm for transparency
70
+
71
+
**Day 5: Performance & Edge Cases**
72
+
- [ ] Add caching layer for expensive operations
73
+
- [ ] Handle border constituencies and ambiguous addresses
74
+
- [ ] Performance test with 100+ concurrent requests
75
+
- [ ] Add monitoring/logging for matching accuracy
76
+
77
+
#### Track 2: UX Polish (Days 3-8)
78
+
79
+
**Day 3-4: Gov.uk-Inspired Branding**
80
+
- [ ] Define color palette (inspired by gov.uk: blues, blacks, whites)
81
+
- [ ] Choose typography (gov.uk uses: Transport/Arial for headings, system fonts for body)
82
+
- [ ] Create CSS design system with variables
83
+
- [ ] Update base template with new styles
84
+
- [ ] Design simple wordmark/logo
85
+
86
+
**Day 5-6: Letter List Improvements**
87
+
- [ ] Add sorting controls (newest, most signatures, most verified)
88
+
- [ ] Add TopicArea filtering (multi-select chips)
89
+
- [ ] Improve letter card design (hierarchy, spacing, affordances)
90
+
- [ ] Add empty states with helpful CTAs
91
+
- [ ] Mobile responsive improvements
92
+
93
+
**Day 6-7: Letter Authoring Flow**
94
+
- [ ] Add character counter (500 char minimum)
95
+
- [ ] Add prominent immutability warning before publish
96
+
- [ ] Show representative suggestion reasoning
97
+
- [ ] Add preview step before publishing
98
+
- [ ] Improve auto-signature confirmation messaging
99
+
100
+
**Day 7-8: Letter Detail & Sharing**
101
+
- [ ] Add prominent "Copy link" button with visual feedback
102
+
- [ ] Add social share buttons (Twitter, Bluesky with pre-filled text)
103
+
- [ ] Clarify signature removal instructions
104
+
- [ ] Improve verified/unverified signature badges
105
+
- [ ] Polish report button and modal
106
+
107
+
#### Track 3: Localization Foundation (Days 6-8)
108
+
109
+
**Day 6-7: Django i18n Setup**
110
+
- [ ] Wrap all strings in `gettext()` / `_()` calls
111
+
- [ ] Generate German .po files
112
+
- [ ] Add language switcher infrastructure (even if only DE works)
113
+
- [ ] Document translation workflow
114
+
115
+
**Day 8: Content Audit**
116
+
- [ ] Audit templates for hardcoded strings
117
+
- [ ] Review German tone/voice consistency
118
+
- [ ] Ensure error messages are clear and helpful
119
+
- [ ] Proofread all user-facing content
120
+
121
+
#### Track 4: Automated Testing (Days 8-10)
122
+
123
+
**Day 8: Integration Tests**
124
+
- [ ] Test full flow: Register → Create Letter → Suggestions → Publish → Sign
125
+
- [ ] Test with 10 real German addresses
126
+
- [ ] Test with 5 different topics
127
+
- [ ] Test email flows (registration, password reset)
128
+
129
+
**Day 9: Matching Tests**
130
+
- [ ] Unit tests for geocoding service
131
+
- [ ] Unit tests for GeoJSON lookup
132
+
- [ ] Integration tests for address → representative matching
133
+
- [ ] Test edge cases (border areas, ambiguous addresses)
134
+
135
+
**Day 10: System Tests**
136
+
- [ ] Browser automation tests (Playwright/Selenium)
137
+
- [ ] Mobile responsive tests
138
+
- [ ] Performance tests (response times, concurrent users)
139
+
- [ ] Create bug fix punch list
140
+
141
+
### **Week 3-4: Deployment & Polish** (Days 11-20)
142
+
143
+
#### Track 5: Production Deployment (Days 11-14)
144
+
145
+
**Day 11-12: VPS Setup**
146
+
- [ ] Provision VPS with cloud-init template
147
+
- [ ] Configure Gunicorn + Nginx
148
+
- [ ] Set up SSL/TLS certificates (Let's Encrypt)
149
+
- [ ] Configure static file serving
150
+
151
+
**Day 13: Production Configuration**
152
+
- [ ] Environment-based settings (secrets, database)
153
+
- [ ] Configure email backend (SMTP/SendGrid/SES)
154
+
- [ ] Set up error tracking (Sentry/Rollbar)
155
+
- [ ] Configure logging (structured logs)
156
+
157
+
**Day 14: Deployment Automation**
158
+
- [ ] Create deployment script (simple rsync/git pull based)
159
+
- [ ] Test rollback procedure
160
+
- [ ] Document deployment process
161
+
- [ ] Set up basic monitoring/health checks
162
+
163
+
#### Track 6: Content & Documentation (Days 15-17)
164
+
165
+
**Day 15-16: Landing & How It Works**
166
+
- [ ] Create compelling homepage (mission, stats, CTA)
167
+
- [ ] Write "How It Works" page (transparency about matching)
168
+
- [ ] Create FAQ section
169
+
- [ ] Add example letters / testimonials
170
+
171
+
**Day 17: Legal & Privacy**
172
+
- [ ] Write basic Privacy Policy (GDPR-compliant)
173
+
- [ ] Write Terms of Service
174
+
- [ ] Add cookie consent if needed
175
+
- [ ] Create Impressum (legal requirement in Germany)
176
+
177
+
#### Track 7: Final Testing & Launch Prep (Days 18-20)
178
+
179
+
**Day 18: User Acceptance Testing**
180
+
- [ ] Run through entire flow with fresh eyes
181
+
- [ ] Test on multiple devices and browsers
182
+
- [ ] Verify all links and forms work
183
+
- [ ] Check for typos and formatting issues
184
+
185
+
**Day 19: Performance & Security Audit**
186
+
- [ ] Load testing (how many concurrent users can it handle?)
187
+
- [ ] Security review (XSS, CSRF, SQL injection protections)
188
+
- [ ] Check all forms have proper validation
189
+
- [ ] Review admin permissions
190
+
191
+
**Day 20: Launch Preparation**
192
+
- [ ] Create launch checklist
193
+
- [ ] Prepare 39C3 demo script
194
+
- [ ] Set up analytics/monitoring dashboards
195
+
- [ ] Plan initial outreach (Twitter, mailing lists, etc.)
196
+
197
+
## Completed Features (From Previous Work)
198
+
- [x] Account Management (registration, login, password reset, deletion)
199
+
- [x] Double opt-in email verification
200
+
- [x] TopicArea taxonomy with keyword matching
201
+
- [x] Representative metadata sync (photos, committees, focus areas)
202
+
- [x] Committee-to-topic automatic mapping
203
+
- [x] Self-declared constituency verification
204
+
- [x] HTMX-based representative suggestions
205
+
- [x] Basic letter authoring and signing flow
206
+
207
+
## Explicitly Deferred (Post-39C3)
208
+
- Third-party identity verification (Verimi, yes®)
209
+
- Analytics/feedback systems (basic monitoring only for MVP)
210
+
- EU Parliament & Landtag levels (Bundestag only for MVP)
211
+
- Draft auto-save functionality
212
+
- Advanced admin moderation tools
213
+
- Multiple language support (German only for MVP, i18n structure ready)
214
+
215
+
## Out of Scope for MVP
216
+
- Local municipality reps, party-wide campaigns.
217
+
- In-browser letter editing with collaboration.
218
+
- Advanced analytics or CRM tooling.
219
+
- Multiple identity providers (beyond initial integration).
220
+
- Expert matching based on representative metadata keywords
221
+
- Biography providers, display on representative detail view and extract keywords for matching
+65
docs/plans/verification.md
+65
docs/plans/verification.md
···
···
1
+
# Identity Verification Vision
2
+
3
+
## Objective
4
+
Guarantee that only real, uniquely verified individuals can sign letters and that their constituency is proven—ideally by reusing identities users already hold (e.g., Verimi, BundID, bank login) rather than forcing a new verification flow.
5
+
6
+
## Core Requirements
7
+
- **Proof of personhood:** Ensure each signature is tied to a real individual (no throwaway accounts).
8
+
- **Proof of constituency:** Capture verified address (street, PLZ, city) and map it to the correct Wahlkreis / Landtag district.
9
+
- **Reusable identity:** Prefer providers where users can consent to sharing existing verified attributes (lower friction than video calls).
10
+
- **Evidence retention:** Store cryptographically signed responses or verification references so we can prove the verification later.
11
+
- **Expiry / refresh:** Verification should have a validity window (e.g., re-check every 6–12 months or when user updates address).
12
+
13
+
## Recommended Providers (Germany)
14
+
15
+
### Identity Wallets (best reuse experience)
16
+
- **Verimi** (OAuth2/OIDC)
17
+
- Users already have a Verimi wallet → grant consent → we receive name + address.
18
+
- Supports multiple underlying methods (eID, VideoIdent, bank sources).
19
+
- **BundID / BundesIdent** (official government ID)
20
+
- OIDC-based access to Personalausweis attributes via government portal.
21
+
- Gold standard for address proof; onboarding limited to approved use cases.
22
+
- **yes® (yes.com)**
23
+
- Bank login to participating institutions; returns bank-verified identity/address via OpenID Connect.
24
+
- No new verification, just consent.
25
+
- **Signicat Identity Platform**
26
+
- Aggregator: supports Verimi, yes®, BankID, eIDAS. Useful if expansion beyond Germany is planned.
27
+
- **Nect Ident**
28
+
- After an initial automated verification, users can re-share their identity from a wallet.
29
+
30
+
### Alternative Methods
31
+
- **BankIdent / PSD2 providers** (WebID BankIdent, yes®, Deutsche Bank BankIdent)
32
+
- Users log into their bank; returns name/address. High trust, no video.
33
+
- **eID solutions (AUTHADA, D-Trust)**
34
+
- NFC-based Personalausweis reading; some provide reusable tokens after first use.
35
+
- **VideoIdent (IDnow, WebID VideoIdent, POSTIDENT)**
36
+
- Higher friction; use as fallback when wallet/bank options fail.
37
+
38
+
## Integration Architecture
39
+
1. **Abstraction layer:** Implement a `VerificationProvider` interface with methods like `start_verification(user)` and `handle_callback(payload)`.
40
+
2. **Provider adapters:** Build adapters for Verimi, yes®, BundID, etc., each handling OAuth2/OIDC flows, token validation, and attribute extraction.
41
+
3. **Verification storage:** Extend `IdentityVerification` to store provider name, verification reference, timestamp, address, and provider response hash/signature.
42
+
4. **Constituency mapping:** After receiving address data, run it through the constituency router (GeoJSON-based once available) to attach the exact direct-mandate seat/state.
43
+
5. **Expiry handling:** Add `expires_at`—prompt users to re-verify when outdated or on address change.
44
+
6. **Audit trail:** Log provider responses; maintain a verification history per user.
45
+
7. **Fallback/manual path:** Offer manual verification (moderator-reviewed documents) only if all automated providers fail, clearly flagging such signatures.
46
+
47
+
## User Flow Blueprint
48
+
1. User chooses “Verify identity”.
49
+
2. We present available providers (Verimi, yes®, BundID…).
50
+
3. User authenticates/consents with chosen provider.
51
+
4. Provider redirects back / sends webhook with verification result + attributes.
52
+
5. We validate response, persist identity data, and map PLZ → constituency.
53
+
6. Signatures now display “Verified constituent” (and reinforce direct mandates with proof).
54
+
55
+
## Implementation Priorities
56
+
- Start with a wallet provider (Verimi or yes®) for minimal friction.
57
+
- Add BundID for maximum trust where accessible.
58
+
- Abstract architecture so adding Landtag-specific providers later is straightforward.
59
+
- Ensure we can reuse the same verification across multiple letters until it expires.
60
+
61
+
## Outstanding Questions
62
+
- Do we need different assurance levels for general signatures vs. direct-mandate proof? (e.g., allow bank login for signing, but require eID for constituency-critical interactions?)
63
+
- How to handle users without access to any supported provider? (Manual override / postal verification?)
64
+
- Data protection & consent: store only what’s necessary (likely name + address); ensure GDPR-compliant retention policies.
65
+
+1
pyproject.toml
+1
pyproject.toml
+11
uv.lock
+11
uv.lock
···
157
]
158
159
[[package]]
160
name = "requests"
161
version = "2.32.5"
162
source = { registry = "https://pypi.org/simple" }
···
260
dependencies = [
261
{ name = "django" },
262
{ name = "geopy" },
263
{ name = "requests" },
264
{ name = "shapely" },
265
{ name = "tqdm" },
···
269
requires-dist = [
270
{ name = "django", specifier = ">=5.2.6" },
271
{ name = "geopy", specifier = ">=2.4.0" },
272
{ name = "requests", specifier = ">=2.31.0" },
273
{ name = "shapely", specifier = ">=2.1.0" },
274
{ name = "tqdm", specifier = ">=4.67.1" },
···
157
]
158
159
[[package]]
160
+
name = "pyshp"
161
+
version = "3.0.2.post1"
162
+
source = { registry = "https://pypi.org/simple" }
163
+
sdist = { url = "https://files.pythonhosted.org/packages/7f/fb/07f057ff01229c575831766b44bd249aefa086146cf5bce52e172d77cf4e/pyshp-3.0.2.post1.tar.gz", hash = "sha256:18e34a66759b6d34a6f535978c76dad518200f23a727d9e22af8e8535c0245b9", size = 2192180, upload-time = "2025-10-10T16:04:58.529Z" }
164
+
wheels = [
165
+
{ url = "https://files.pythonhosted.org/packages/51/92/a8ad817864a560b96ac1c817f9c56bb7eacc1a7d522e2d39afe9e9c77d7b/pyshp-3.0.2.post1-py3-none-any.whl", hash = "sha256:b0aec66bc55f7cd3a846f6b02c5a9eec1fc1d2cff16ccfcf6493a6773c7eb602", size = 58298, upload-time = "2025-10-10T16:04:57.151Z" },
166
+
]
167
+
168
+
[[package]]
169
name = "requests"
170
version = "2.32.5"
171
source = { registry = "https://pypi.org/simple" }
···
269
dependencies = [
270
{ name = "django" },
271
{ name = "geopy" },
272
+
{ name = "pyshp" },
273
{ name = "requests" },
274
{ name = "shapely" },
275
{ name = "tqdm" },
···
279
requires-dist = [
280
{ name = "django", specifier = ">=5.2.6" },
281
{ name = "geopy", specifier = ">=2.4.0" },
282
+
{ name = "pyshp", specifier = ">=2.3.1" },
283
{ name = "requests", specifier = ">=2.31.0" },
284
{ name = "shapely", specifier = ">=2.1.0" },
285
{ name = "tqdm", specifier = ">=4.67.1" },
-60
vision/matching.md
-60
vision/matching.md
···
1
-
# Recipient Matching Vision
2
-
3
-
## Goal
4
-
Ensure every letter reaches the most relevant representative by combining precise constituency mapping with topic-awareness and reliable representative metadata.
5
-
6
-
## Core Pillars
7
-
8
-
1. **Constituency Precision**
9
-
- Replace postal-prefix heuristics with official boundary data:
10
-
- Bundestag Wahlkreise (Bundeswahlleiter / BKG GeoJSON)
11
-
- Landtag electoral districts via state open-data portals or OParl feeds
12
-
- EU parliament treated as nationwide constituency
13
-
- Normalise mandate modes:
14
-
- Direktmandat → voters in that Wahlkreis
15
-
- Landesliste → voters in the state
16
-
- Bundes/EU list → national constituencies
17
-
- Centralise the logic in a “constituency router” so each parliament’s data source is pluggable.
18
-
19
-
2. **Topic Understanding**
20
-
- Analyse title + body to classify concerns into a canonical taxonomy (reuse committee topics, extend as needed).
21
-
- Infer the responsible level (EU / Bund / Land / Kommune) from topic metadata.
22
-
- Keep the topic model extensible (keyword heuristics today, embeddings or classifiers tomorrow).
23
-
24
-
3. **Rich Representative Profiles**
25
-
- Build a `RepresentativeProfile` table to store per-source enrichments:
26
-
- Source (ABGEORDNETENWATCH, BUNDESTAG, LANDTAG_*)
27
-
- Normalised fields: focus areas, biography, external links, responsiveness
28
-
- Raw metadata + sync timestamps
29
-
- Importers:
30
-
- Abgeordnetenwatch: `/politicians/{id}` (issues, responsiveness, social links)
31
-
- Bundestag: official vita JSON (`mdbId`) for biography + spokesperson roles
32
-
- Landtage: state-specific data feeds (OParl, CSV, or one-off scrapers)
33
-
- Profiles coexist; the merging service resolves conflicts and picks the best available data.
34
-
35
-
## Matching Pipeline
36
-
1. **Constituency filter**: Use the router and mandate rules to determine eligible reps.
37
-
2. **Topic filter**: Narrow to the inferred level and portfolio.
38
-
3. **Scoring**: Blend signals—constituency proximity, topic match (committee → topic), activity (votes, questions), responsiveness stats, optional user preferences.
39
-
4. **Explanation**: Provide human-readable reasons (“Direct MP for WK 123; sits on Verkehrsausschuss; answered 90% of Abgeordnetenwatch questions”).
40
-
41
-
## Data Sources Reference
42
-
43
-
| Use Case | Federal | State | EU |
44
-
|-------------------------|-------------------------------------|---------------------------------------------|--------------------------------|
45
-
| Mandates & committees | Abgeordnetenwatch API | Abgeordnetenwatch, OParl, Landtag portals | EU Parliament REST API |
46
-
| Constituency boundaries | Bundeswahlleiter GeoJSON, BKG | Landeswahlleitungen, state GIS datasets | Whole-of-Germany (single geom) |
47
-
| Biography / focus | Bundestag vita JSON, Abgeordnetenwatch issues | Landtag bios (open data) | Europarl member profiles |
48
-
49
-
## Implementation Notes
50
-
- Expose `sync_representative_profiles` commands per source; schedule separately from mandate sync.
51
-
- Track `source_version`/`hash` to avoid redundant imports.
52
-
- View layer consumes a `RepresentativeProfileService` that aggregates focus areas, biography, links, responsiveness.
53
-
- Keep a roadmap for future sources (party press, DIP21 votes, Europarl “files”).
54
-
55
-
## Next Steps
56
-
- Implement `RepresentativeProfile` model + importers for Abgeordnetenwatch and Bundestag.
57
-
- Integrate boundary datasets and swap the PLZ router.
58
-
- Wire the matching pipeline into the letter form suggestions and automated routing.
59
-
- Add logging/monitoring for profile freshness and matching success.
60
-
···
-89
vision/mvp.md
-89
vision/mvp.md
···
1
-
# MVP Vision
2
-
3
-
## Mission
4
-
Empower citizens to participate in democracy by writing impactful open letters to the representatives best positioned to act, and by allowing others to rally behind those letters with signatures. Verified identities add credibility; when a letter clears a signature threshold, we commit to printing, signing, and delivering it to the relevant office.
5
-
6
-
## Core Feature Set
7
-
1. **Accounts & Profiles**
8
-
- Email/password registration & login.
9
-
- Profile page showing authored letters, signed letters, and verification status.
10
-
2. **Representative Directory**
11
-
- EU, Bundestag, Landtag members with photo, party, mandate mode, committee roles, Abgeordnetenwatch links.
12
-
- Exposed via detail view and reusable UI card.
13
-
3. **Letter Authoring & Publishing**
14
-
- Draft open letters, auto-suggest recipients based on title + PLZ.
15
-
- Auto-sign on publish; allow editing until first signature.
16
-
- Letter detail page shows full content, representative card, signature stats.
17
-
4. **Recommendation Engine**
18
-
- PLZ → constituency router (direct/state/federal) using official boundary data.
19
-
- Topic analysis highlighting likely responsible level and committee working areas.
20
-
- Explain why a representative is recommended, surface relevant tags, show similar letters.
21
-
5. **Signature Flow**
22
-
- One-click signing for logged in users; prompt login otherwise.
23
-
- Badges for verified vs unverified signatures, count constituents distinctly.
24
-
- Social sharing (link copy, optional Twitter/Bluesky share).
25
-
6. **Identity Verification (Optional)**
26
-
- Integrate one third-party provider (e.g., Verimi or yes®) via OAuth2/OIDC to pull verified name/address.
27
-
- Store attestation + expiry; map address to constituency for direct mandates.
28
-
- Users without verification can still sign, flagged as “unverified.”
29
-
7. **Signature Threshold & Fulfilment**
30
-
- Configurable threshold per letter or representative type.
31
-
- Admin view showing letters reaching milestones, export letter + supporters for printing/mailing.
32
-
8. **Admin & Moderation**
33
-
- Admin dashboard to inspect letters, representatives, signatures, verification status, and signature thresholds.
34
-
- Ability to disable inappropriate letters, resend sync, run exports.
35
-
9. **Landing & Discovery**
36
-
- Public homepage summarising mission, stats, featured letters.
37
-
- Browse letters and representatives without login.
38
-
10. **Documentation & Transparency**
39
-
- Public “How it works” page, privacy policy, terms.
40
-
- README covering setup, architecture, deployment.
41
-
42
-
## MVP To-Do List
43
-
1. **Constituency & Matching Foundations**
44
-
- [ ] Replace PLZ prefix heuristic with Wahlkreis GeoJSON (Bundestag) + state-level boundaries; build router service.
45
-
- [x] Expand `TopicArea` taxonomy, add NLP/keyword scoring, and present explanations.
46
-
- [x] Enrich representative metadata with committee focus, responsiveness, photos.
47
-
- [x] Scope recommendation engine to relevant parliaments using constituency + topic competence.
48
-
2. **Account Management**
49
-
- [ ] Add account deletion option (removes signatures but keeps letters)
50
-
- [ ] Add double opt-in for account creation
51
-
3. **UX**
52
-
- [ ] Add letter list sorting by signatures / verified signatures / age
53
-
- [ ] Add filtering based on TopicArea keywords
54
-
- [ ] Remove Kompetenzen info page
55
-
- [ ] Rudimentary branding - color scheme, bootstrap
56
-
3. **Identity Verification Integration**
57
-
- [ ] Build provider abstraction and connect to first reusable ID service.
58
-
- [ ] Persist provider response (hash/ID, address) with expiry handling; skip manual verification path.
59
-
- [ ] Determine if providers exist offering login-provider functionality
60
-
- [x] Support self-declared constituency verification with profile management UI.
61
-
3. **Letter Authoring UX**
62
-
- [x] Polish HTMX suggestions and representative cards for consistency.
63
-
- [ ] Allow draft auto-save and clearer edit states pre-signature.
64
-
- [ ] Add share buttons and clearer “copy link” prompt on letter detail.
65
-
- [ ] Add minimum letter length of 500 characters.
66
-
- [ ] Make it very clear that letters cannot be changed after publication
67
-
- [ ] Make it very clear that you can remove your signature from a letter, but not the letter itself
68
-
6. **Localization & Accessibility**
69
-
- [ ] Complete en/de translation coverage for all templates and forms.
70
-
- [ ] Ensure forms, buttons, and suggestions meet accessibility best practices.
71
-
7. **Deployment Readiness**
72
-
- [ ] Production config (secrets, logging, error tracking, email backend).
73
-
- [ ] Deploy to VPS - static media, unicorn, nginx, docker
74
-
- [ ] Health checks with Tinylytics
75
-
- [ ] Add caching
76
-
8. **Feedback & Analytics**
77
-
- [ ] Add feedback/contact channel for users.
78
-
- [ ] Add simple analytics app. Middleware that keeps track of impressions -> build this so it can easily be moved into a separate repo
79
-
9. **Testing & QA**
80
-
- [ ] Expand automated test coverage (matching, verification, export workflow).
81
-
- [ ] QA checklist for matching accuracy, verification flow, admin exports.
82
-
83
-
## Out of Scope for MVP
84
-
- Local municipality reps, party-wide campaigns.
85
-
- In-browser letter editing with collaboration.
86
-
- Advanced analytics or CRM tooling.
87
-
- Multiple identity providers (beyond initial integration).
88
-
- Expert matching based on representative metadata keywords
89
-
- Biography providers, display on representative detail view and extract keywords for matching
···
-65
vision/verification.md
-65
vision/verification.md
···
1
-
# Identity Verification Vision
2
-
3
-
## Objective
4
-
Guarantee that only real, uniquely verified individuals can sign letters and that their constituency is proven—ideally by reusing identities users already hold (e.g., Verimi, BundID, bank login) rather than forcing a new verification flow.
5
-
6
-
## Core Requirements
7
-
- **Proof of personhood:** Ensure each signature is tied to a real individual (no throwaway accounts).
8
-
- **Proof of constituency:** Capture verified address (street, PLZ, city) and map it to the correct Wahlkreis / Landtag district.
9
-
- **Reusable identity:** Prefer providers where users can consent to sharing existing verified attributes (lower friction than video calls).
10
-
- **Evidence retention:** Store cryptographically signed responses or verification references so we can prove the verification later.
11
-
- **Expiry / refresh:** Verification should have a validity window (e.g., re-check every 6–12 months or when user updates address).
12
-
13
-
## Recommended Providers (Germany)
14
-
15
-
### Identity Wallets (best reuse experience)
16
-
- **Verimi** (OAuth2/OIDC)
17
-
- Users already have a Verimi wallet → grant consent → we receive name + address.
18
-
- Supports multiple underlying methods (eID, VideoIdent, bank sources).
19
-
- **BundID / BundesIdent** (official government ID)
20
-
- OIDC-based access to Personalausweis attributes via government portal.
21
-
- Gold standard for address proof; onboarding limited to approved use cases.
22
-
- **yes® (yes.com)**
23
-
- Bank login to participating institutions; returns bank-verified identity/address via OpenID Connect.
24
-
- No new verification, just consent.
25
-
- **Signicat Identity Platform**
26
-
- Aggregator: supports Verimi, yes®, BankID, eIDAS. Useful if expansion beyond Germany is planned.
27
-
- **Nect Ident**
28
-
- After an initial automated verification, users can re-share their identity from a wallet.
29
-
30
-
### Alternative Methods
31
-
- **BankIdent / PSD2 providers** (WebID BankIdent, yes®, Deutsche Bank BankIdent)
32
-
- Users log into their bank; returns name/address. High trust, no video.
33
-
- **eID solutions (AUTHADA, D-Trust)**
34
-
- NFC-based Personalausweis reading; some provide reusable tokens after first use.
35
-
- **VideoIdent (IDnow, WebID VideoIdent, POSTIDENT)**
36
-
- Higher friction; use as fallback when wallet/bank options fail.
37
-
38
-
## Integration Architecture
39
-
1. **Abstraction layer:** Implement a `VerificationProvider` interface with methods like `start_verification(user)` and `handle_callback(payload)`.
40
-
2. **Provider adapters:** Build adapters for Verimi, yes®, BundID, etc., each handling OAuth2/OIDC flows, token validation, and attribute extraction.
41
-
3. **Verification storage:** Extend `IdentityVerification` to store provider name, verification reference, timestamp, address, and provider response hash/signature.
42
-
4. **Constituency mapping:** After receiving address data, run it through the constituency router (GeoJSON-based once available) to attach the exact direct-mandate seat/state.
43
-
5. **Expiry handling:** Add `expires_at`—prompt users to re-verify when outdated or on address change.
44
-
6. **Audit trail:** Log provider responses; maintain a verification history per user.
45
-
7. **Fallback/manual path:** Offer manual verification (moderator-reviewed documents) only if all automated providers fail, clearly flagging such signatures.
46
-
47
-
## User Flow Blueprint
48
-
1. User chooses “Verify identity”.
49
-
2. We present available providers (Verimi, yes®, BundID…).
50
-
3. User authenticates/consents with chosen provider.
51
-
4. Provider redirects back / sends webhook with verification result + attributes.
52
-
5. We validate response, persist identity data, and map PLZ → constituency.
53
-
6. Signatures now display “Verified constituent” (and reinforce direct mandates with proof).
54
-
55
-
## Implementation Priorities
56
-
- Start with a wallet provider (Verimi or yes®) for minimal friction.
57
-
- Add BundID for maximum trust where accessible.
58
-
- Abstract architecture so adding Landtag-specific providers later is straightforward.
59
-
- Ensure we can reuse the same verification across multiple letters until it expires.
60
-
61
-
## Outstanding Questions
62
-
- Do we need different assurance levels for general signatures vs. direct-mandate proof? (e.g., allow bank login for signing, but require eID for constituency-critical interactions?)
63
-
- How to handle users without access to any supported provider? (Manual override / postal verification?)
64
-
- Data protection & consent: store only what’s necessary (likely name + address); ensure GDPR-compliant retention policies.
65
-
···
+18
website/letters/apps.py
+18
website/letters/apps.py
···
4
class LettersConfig(AppConfig):
5
default_auto_field = 'django.db.models.BigAutoField'
6
name = 'letters'
7
+
8
+
def ready(self):
9
+
"""Pre-load GeoJSON data on startup for improved performance."""
10
+
try:
11
+
from .services import WahlkreisLocator
12
+
import logging
13
+
14
+
logger = logging.getLogger(__name__)
15
+
logger.info("Warming WahlkreisLocator cache on startup...")
16
+
17
+
# Initialize to load and cache GeoJSON data
18
+
WahlkreisLocator()
19
+
20
+
logger.info("WahlkreisLocator cache warmed successfully")
21
+
except Exception as e:
22
+
import logging
23
+
logger = logging.getLogger(__name__)
24
+
logger.warning(f"Failed to warm WahlkreisLocator cache: {e}")
+65
-1
website/letters/forms.py
+65
-1
website/letters/forms.py
···
5
from django.utils.translation import gettext_lazy as _
6
7
from .constants import normalize_german_state
8
+
from .models import Letter, Representative, Signature, Report, Tag, Constituency, IdentityVerification
9
10
11
class UserRegisterForm(UserCreationForm):
···
241
)
242
243
return cleaned_data
244
+
245
+
246
+
class IdentityVerificationForm(forms.Form):
247
+
"""Form for collecting full address for identity verification."""
248
+
249
+
street_address = forms.CharField(
250
+
max_length=255,
251
+
required=False,
252
+
label=_('Straße und Hausnummer'),
253
+
widget=forms.TextInput(attrs={
254
+
'class': 'form-control',
255
+
'placeholder': _('z.B. Unter den Linden 77')
256
+
})
257
+
)
258
+
postal_code = forms.CharField(
259
+
max_length=20,
260
+
required=False,
261
+
label=_('Postleitzahl'),
262
+
widget=forms.TextInput(attrs={
263
+
'class': 'form-control',
264
+
'placeholder': _('z.B. 10117')
265
+
})
266
+
)
267
+
city = forms.CharField(
268
+
max_length=100,
269
+
required=False,
270
+
label=_('Stadt'),
271
+
widget=forms.TextInput(attrs={
272
+
'class': 'form-control',
273
+
'placeholder': _('z.B. Berlin')
274
+
})
275
+
)
276
+
277
+
def __init__(self, *args, **kwargs):
278
+
self.user = kwargs.pop('user', None)
279
+
super().__init__(*args, **kwargs)
280
+
281
+
# Pre-fill with existing address if available
282
+
if self.user and hasattr(self.user, 'identity_verification'):
283
+
verification = getattr(self.user, 'identity_verification', None)
284
+
if verification:
285
+
if verification.street_address:
286
+
self.fields['street_address'].initial = verification.street_address
287
+
if verification.postal_code:
288
+
self.fields['postal_code'].initial = verification.postal_code
289
+
if verification.city:
290
+
self.fields['city'].initial = verification.city
291
+
292
+
def clean(self):
293
+
cleaned_data = super().clean()
294
+
street_address = cleaned_data.get('street_address')
295
+
postal_code = cleaned_data.get('postal_code')
296
+
city = cleaned_data.get('city')
297
+
298
+
# Check if any field is provided
299
+
has_any = any([street_address, postal_code, city])
300
+
has_all = all([street_address, postal_code, city])
301
+
302
+
if has_any and not has_all:
303
+
raise forms.ValidationError(
304
+
_('Bitte geben Sie eine vollständige Adresse ein (Straße, PLZ und Stadt) oder lassen Sie alle Felder leer.')
305
+
)
306
+
307
+
return cleaned_data
+80
-10
website/letters/management/commands/fetch_wahlkreis_data.py
+80
-10
website/letters/management/commands/fetch_wahlkreis_data.py
···
1
import io
2
import json
3
import zipfile
4
from pathlib import Path
5
from typing import Optional
···
8
from django.conf import settings
9
from django.core.management.base import BaseCommand, CommandError
10
11
DEFAULT_WAHLKREIS_URL = (
12
-
"https://raw.githubusercontent.com/dknx01/wahlkreissuche/main/data/wahlkreise.geojson"
13
)
14
15
16
class Command(BaseCommand):
17
-
"""Download and store Wahlkreis geodata for constituency lookups."""
18
19
help = (
20
-
"Fetch Bundestag constituency (Wahlkreis) boundary data and store it as GeoJSON "
21
-
"for shapely-based lookup."
22
)
23
24
def add_arguments(self, parser):
···
68
content_type = response.headers.get("Content-Type", "")
69
data_bytes = response.content
70
71
if url.lower().endswith(".zip") or "zip" in content_type:
72
-
geojson_bytes = self._extract_from_zip(data_bytes, zip_member)
73
else:
74
-
geojson_bytes = data_bytes
75
76
try:
77
-
geojson_text = geojson_bytes.decode("utf-8")
78
-
json.loads(geojson_text)
79
-
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
80
-
raise CommandError("Downloaded data is not valid UTF-8 GeoJSON") from exc
81
82
output_path.parent.mkdir(parents=True, exist_ok=True)
83
output_path.write_text(geojson_text, encoding="utf-8")
84
85
self.stdout.write(self.style.SUCCESS(f"Saved Wahlkreis data to {output_path}"))
86
87
def _extract_from_zip(self, data: bytes, member: Optional[str]) -> bytes:
88
with zipfile.ZipFile(io.BytesIO(data)) as archive:
···
1
import io
2
import json
3
+
import tempfile
4
import zipfile
5
from pathlib import Path
6
from typing import Optional
···
9
from django.conf import settings
10
from django.core.management.base import BaseCommand, CommandError
11
12
+
# Official Bundeswahlleiterin Shapefile URL (2025 election)
13
DEFAULT_WAHLKREIS_URL = (
14
+
"https://www.bundeswahlleiterin.de/dam/jcr/a3b60aa9-8fa5-4223-9fb4-0a3a3cebd7d1/"
15
+
"btw25_geometrie_wahlkreise_vg250_shp_geo.zip"
16
)
17
18
19
class Command(BaseCommand):
20
+
"""Download and convert Wahlkreis geodata for constituency lookups."""
21
22
help = (
23
+
"Fetch Bundestag constituency (Wahlkreis) boundary data from bundeswahlleiterin.de, "
24
+
"convert from Shapefile to GeoJSON if needed, and store for shapely-based lookup. "
25
+
"The GeoJSON file is cached locally and Shapefile components are not kept."
26
)
27
28
def add_arguments(self, parser):
···
72
content_type = response.headers.get("Content-Type", "")
73
data_bytes = response.content
74
75
+
# Check if this is a ZIP file
76
if url.lower().endswith(".zip") or "zip" in content_type:
77
+
# Check if it contains a .shp file (Shapefile format)
78
+
if self._zip_contains_shapefile(data_bytes):
79
+
self.stdout.write("Detected Shapefile in ZIP, converting to GeoJSON...")
80
+
geojson_text = self._convert_shapefile_to_geojson(data_bytes)
81
+
else:
82
+
# Extract GeoJSON directly from ZIP
83
+
geojson_bytes = self._extract_from_zip(data_bytes, zip_member)
84
+
geojson_text = geojson_bytes.decode("utf-8")
85
else:
86
+
geojson_text = data_bytes.decode("utf-8")
87
88
+
# Validate GeoJSON
89
try:
90
+
geojson_data = json.loads(geojson_text)
91
+
feature_count = len(geojson_data.get("features", []))
92
+
self.stdout.write(f"Validated GeoJSON with {feature_count} features")
93
+
except json.JSONDecodeError as exc:
94
+
raise CommandError("Downloaded data is not valid GeoJSON") from exc
95
96
output_path.parent.mkdir(parents=True, exist_ok=True)
97
output_path.write_text(geojson_text, encoding="utf-8")
98
99
self.stdout.write(self.style.SUCCESS(f"Saved Wahlkreis data to {output_path}"))
100
+
101
+
def _zip_contains_shapefile(self, data: bytes) -> bool:
102
+
"""Check if ZIP contains Shapefile components (.shp)."""
103
+
try:
104
+
with zipfile.ZipFile(io.BytesIO(data)) as archive:
105
+
return any(name.lower().endswith(".shp") for name in archive.namelist())
106
+
except zipfile.BadZipFile:
107
+
return False
108
+
109
+
def _convert_shapefile_to_geojson(self, data: bytes) -> str:
110
+
"""Convert Shapefile in ZIP to GeoJSON using pyshp."""
111
+
try:
112
+
import shapefile # pyshp library
113
+
except ImportError:
114
+
raise CommandError(
115
+
"pyshp library is required to convert Shapefiles. "
116
+
"Install with: pip install pyshp"
117
+
)
118
+
119
+
with tempfile.TemporaryDirectory() as tmpdir:
120
+
tmpdir_path = Path(tmpdir)
121
+
122
+
# Extract all Shapefile components to temp directory
123
+
with zipfile.ZipFile(io.BytesIO(data)) as archive:
124
+
shp_files = [name for name in archive.namelist() if name.lower().endswith(".shp")]
125
+
if not shp_files:
126
+
raise CommandError("No .shp file found in ZIP archive")
127
+
128
+
shp_file = shp_files[0]
129
+
base_name = Path(shp_file).stem
130
+
131
+
# Extract all related files (.shp, .shx, .dbf, .prj, etc.)
132
+
for member in archive.namelist():
133
+
if Path(member).stem == base_name:
134
+
archive.extract(member, tmpdir_path)
135
+
136
+
# Convert using pyshp
137
+
shp_path = tmpdir_path / shp_file
138
+
sf = shapefile.Reader(str(shp_path))
139
+
140
+
# Convert to GeoJSON
141
+
features = []
142
+
for shape_rec in sf.shapeRecords():
143
+
feature = {
144
+
"type": "Feature",
145
+
"geometry": shape_rec.shape.__geo_interface__,
146
+
"properties": shape_rec.record.as_dict()
147
+
}
148
+
features.append(feature)
149
+
150
+
geojson = {
151
+
"type": "FeatureCollection",
152
+
"features": features
153
+
}
154
+
155
+
return json.dumps(geojson, ensure_ascii=False, indent=None)
156
157
def _extract_from_zip(self, data: bytes, member: Optional[str]) -> bytes:
158
with zipfile.ZipFile(io.BytesIO(data)) as archive:
+105
website/letters/management/commands/query_representatives.py
+105
website/letters/management/commands/query_representatives.py
···
···
1
+
# ABOUTME: Query management command to find representatives by address and/or topics.
2
+
# ABOUTME: Interactive tool for testing representative suggestion logic.
3
+
4
+
from django.core.management.base import BaseCommand
5
+
from letters.services import ConstituencyLocator, TopicSuggestionService, ConstituencySuggestionService
6
+
7
+
8
+
class Command(BaseCommand):
9
+
help = 'Find representatives by address and/or topics'
10
+
11
+
def add_arguments(self, parser):
12
+
# Address arguments
13
+
parser.add_argument(
14
+
'--street',
15
+
type=str,
16
+
help='Street name and number'
17
+
)
18
+
parser.add_argument(
19
+
'--postal-code',
20
+
type=str,
21
+
help='Postal code (PLZ)'
22
+
)
23
+
parser.add_argument(
24
+
'--city',
25
+
type=str,
26
+
help='City name'
27
+
)
28
+
29
+
# Topic arguments
30
+
parser.add_argument(
31
+
'--topics',
32
+
type=str,
33
+
help='Comma-separated topic keywords (e.g., "Verkehr,Infrastruktur")'
34
+
)
35
+
36
+
parser.add_argument(
37
+
'--limit',
38
+
type=int,
39
+
default=10,
40
+
help='Maximum number of representatives to return (default: 10)'
41
+
)
42
+
43
+
def handle(self, *args, **options):
44
+
street = options.get('street')
45
+
postal_code = options.get('postal_code')
46
+
city = options.get('city')
47
+
topics_str = options.get('topics')
48
+
limit = options['limit']
49
+
50
+
try:
51
+
# Use constituency locator if address provided
52
+
if postal_code or (street and city):
53
+
locator = ConstituencyLocator()
54
+
representatives = locator.locate(
55
+
street=street,
56
+
postal_code=postal_code,
57
+
city=city
58
+
)
59
+
60
+
if not representatives:
61
+
self.stdout.write('No representatives found for this location')
62
+
return
63
+
64
+
# Filter by topics if provided
65
+
if topics_str:
66
+
topic_keywords = [t.strip() for t in topics_str.split(',')]
67
+
# Simple keyword filter on representative focus areas
68
+
filtered_reps = []
69
+
for rep in representatives:
70
+
# Check if any committee or focus area matches
71
+
rep_text = ' '.join([
72
+
rep.full_name,
73
+
' '.join([c.name for c in rep.committees.all()]),
74
+
]).lower()
75
+
76
+
if any(keyword.lower() in rep_text for keyword in topic_keywords):
77
+
filtered_reps.append(rep)
78
+
79
+
representatives = filtered_reps if filtered_reps else representatives
80
+
81
+
# Display results
82
+
for rep in representatives[:limit]:
83
+
constituency = rep.primary_constituency
84
+
constituency_label = constituency.name if constituency else rep.parliament.name
85
+
self.stdout.write(f'{rep.full_name} ({rep.party}) - {constituency_label}')
86
+
87
+
# Show committees
88
+
committees = list(rep.committees.all()[:3])
89
+
if committees:
90
+
committee_names = ', '.join([c.name for c in committees])
91
+
self.stdout.write(f' Committees: {committee_names}')
92
+
93
+
# Use topic-based search if only topics provided
94
+
elif topics_str:
95
+
self.stdout.write('Topic-based representative search not yet implemented')
96
+
self.stdout.write('Please provide at least a postal code for location-based search')
97
+
98
+
else:
99
+
self.stderr.write(self.style.ERROR(
100
+
'Error: Please provide either an address (--postal-code required) or --topics'
101
+
))
102
+
103
+
except Exception as e:
104
+
self.stderr.write(self.style.ERROR(f'Error: {str(e)}'))
105
+
return
+62
website/letters/management/commands/query_topics.py
+62
website/letters/management/commands/query_topics.py
···
···
1
+
# ABOUTME: Query management command to find matching topics for letter text.
2
+
# ABOUTME: Interactive tool for testing topic keyword matching and scoring.
3
+
4
+
from django.core.management.base import BaseCommand
5
+
from letters.services import TopicSuggestionService
6
+
7
+
8
+
class Command(BaseCommand):
9
+
help = 'Find matching topics for a letter title or text'
10
+
11
+
def add_arguments(self, parser):
12
+
parser.add_argument(
13
+
'--text',
14
+
type=str,
15
+
required=True,
16
+
help='Letter title or text to analyze'
17
+
)
18
+
parser.add_argument(
19
+
'--limit',
20
+
type=int,
21
+
default=5,
22
+
help='Maximum number of topics to return (default: 5)'
23
+
)
24
+
25
+
def handle(self, *args, **options):
26
+
text = options['text']
27
+
limit = options['limit']
28
+
29
+
try:
30
+
# Use the suggest_representatives_for_concern method to get topic suggestions
31
+
result = TopicSuggestionService.suggest_representatives_for_concern(
32
+
text,
33
+
limit=limit
34
+
)
35
+
36
+
matched_topics = result.get('matched_topics', [])
37
+
38
+
if not matched_topics:
39
+
self.stdout.write('No matching topics found')
40
+
return
41
+
42
+
# Display matched topics
43
+
for topic in matched_topics[:limit]:
44
+
# TopicArea objects have name, primary_level, and description
45
+
level = getattr(topic, 'primary_level', 'UNKNOWN')
46
+
name = getattr(topic, 'name', str(topic))
47
+
description = getattr(topic, 'description', '')
48
+
49
+
self.stdout.write(f"{name} ({level})")
50
+
if description:
51
+
self.stdout.write(f" {description}")
52
+
53
+
# Also show suggested level and explanation
54
+
if result.get('suggested_level'):
55
+
self.stdout.write('')
56
+
self.stdout.write(f"Suggested Level: {result['suggested_level']}")
57
+
if result.get('explanation'):
58
+
self.stdout.write(f"Explanation: {result['explanation']}")
59
+
60
+
except Exception as e:
61
+
self.stderr.write(self.style.ERROR(f'Error: {str(e)}'))
62
+
return
+66
website/letters/management/commands/query_wahlkreis.py
+66
website/letters/management/commands/query_wahlkreis.py
···
···
1
+
# ABOUTME: Query management command to find constituency by address or postal code.
2
+
# ABOUTME: Interactive tool for testing address-based constituency matching.
3
+
4
+
from django.core.management.base import BaseCommand
5
+
from letters.services import AddressGeocoder, WahlkreisLocator, ConstituencyLocator
6
+
7
+
8
+
class Command(BaseCommand):
9
+
help = 'Find constituency (Wahlkreis) by address or postal code'
10
+
11
+
def add_arguments(self, parser):
12
+
parser.add_argument(
13
+
'--street',
14
+
type=str,
15
+
help='Street name and number'
16
+
)
17
+
parser.add_argument(
18
+
'--postal-code',
19
+
type=str,
20
+
help='Postal code (PLZ)',
21
+
required=True
22
+
)
23
+
parser.add_argument(
24
+
'--city',
25
+
type=str,
26
+
help='City name'
27
+
)
28
+
29
+
def handle(self, *args, **options):
30
+
street = options.get('street')
31
+
postal_code = options['postal_code']
32
+
city = options.get('city')
33
+
34
+
try:
35
+
# Try full address geocoding if all parts provided
36
+
if street and city:
37
+
geocoder = AddressGeocoder()
38
+
lat, lon, success, error = geocoder.geocode(street, postal_code, city)
39
+
40
+
if not success:
41
+
self.stdout.write(self.style.ERROR(f'Error: Could not geocode address: {error}'))
42
+
return
43
+
44
+
locator = WahlkreisLocator()
45
+
result = locator.locate(lat, lon)
46
+
47
+
if not result:
48
+
self.stdout.write('No constituency found for these coordinates')
49
+
return
50
+
51
+
wkr_nr, wkr_name, land_name = result
52
+
self.stdout.write(f'WK {wkr_nr:03d} - {wkr_name} ({land_name})')
53
+
54
+
# Fallback to PLZ prefix lookup
55
+
else:
56
+
plz_prefix = postal_code[:2]
57
+
state_name = ConstituencyLocator.STATE_BY_PLZ_PREFIX.get(plz_prefix)
58
+
59
+
if state_name:
60
+
self.stdout.write(f'State: {state_name} (from postal code prefix)')
61
+
else:
62
+
self.stdout.write('Error: Could not determine state from postal code')
63
+
64
+
except Exception as e:
65
+
self.stderr.write(self.style.ERROR(f'Error: {str(e)}'))
66
+
return
-81
website/letters/management/commands/test_constituency_suggestion.py
-81
website/letters/management/commands/test_constituency_suggestion.py
···
1
-
"""
2
-
Management command to test the ConstituencySuggestionService with example queries.
3
-
"""
4
-
5
-
from django.core.management.base import BaseCommand
6
-
from letters.services import ConstituencySuggestionService
7
-
8
-
9
-
class Command(BaseCommand):
10
-
help = 'Test the constituency suggestion service with example queries'
11
-
12
-
def add_arguments(self, parser):
13
-
parser.add_argument(
14
-
'--query',
15
-
type=str,
16
-
help='Custom query to test (if not provided, runs example queries)'
17
-
)
18
-
19
-
def handle(self, *args, **options):
20
-
custom_query = options.get('query')
21
-
22
-
if custom_query:
23
-
# Test custom query
24
-
self.test_query(custom_query)
25
-
else:
26
-
# Run all example queries
27
-
self.stdout.write(self.style.SUCCESS('Testing ConstituencySuggestionService\n'))
28
-
self.stdout.write('=' * 80)
29
-
30
-
examples = ConstituencySuggestionService.get_example_queries()
31
-
32
-
for i, example in enumerate(examples, 1):
33
-
self.stdout.write(f"\n{i}. Query: \"{example['query']}\"")
34
-
self.stdout.write(f" Expected: {example['expected_level']} - {example['topic']}")
35
-
self.stdout.write('-' * 80)
36
-
37
-
self.test_query(example['query'])
38
-
39
-
self.stdout.write('=' * 80)
40
-
41
-
def test_query(self, query):
42
-
"""Test a single query and display results"""
43
-
result = ConstituencySuggestionService.suggest_from_concern(query)
44
-
45
-
# Display matched topics
46
-
self.stdout.write(self.style.WARNING('\nMatched Topics:'))
47
-
if result['matched_topics']:
48
-
for topic in result['matched_topics']:
49
-
self.stdout.write(
50
-
f" • {topic.name} ({topic.get_primary_level_display()}) - {topic.competency_type}"
51
-
)
52
-
else:
53
-
self.stdout.write(' None')
54
-
55
-
# Display suggested level
56
-
self.stdout.write(self.style.WARNING('\nSuggested Level:'))
57
-
self.stdout.write(f" {result['suggested_level']}")
58
-
59
-
# Display explanation
60
-
self.stdout.write(self.style.WARNING('\nExplanation:'))
61
-
self.stdout.write(f" {result['explanation']}")
62
-
63
-
# Display constituencies
64
-
self.stdout.write(self.style.WARNING('\nConstituencies:'))
65
-
if result['constituencies']:
66
-
for const in result['constituencies'][:5]: # Limit to 5
67
-
self.stdout.write(f" • {const.name} ({const.get_level_display()})")
68
-
else:
69
-
self.stdout.write(' None')
70
-
71
-
# Display representatives
72
-
self.stdout.write(self.style.WARNING('\nRepresentatives:'))
73
-
if result['representatives']:
74
-
for rep in result['representatives'][:5]: # Limit to 5
75
-
constituency = rep.primary_constituency
76
-
constituency_label = constituency.name if constituency else rep.parliament.name
77
-
self.stdout.write(
78
-
f" • {rep.full_name} ({rep.party}) - {constituency_label}"
79
-
)
80
-
else:
81
-
self.stdout.write(' None (no representatives in database yet)')
···
-85
website/letters/management/commands/test_topic_mapping.py
-85
website/letters/management/commands/test_topic_mapping.py
···
1
-
"""
2
-
Management command to test topic-to-constituency mapping.
3
-
4
-
Provides examples of how the topic suggestion service works.
5
-
"""
6
-
7
-
from django.core.management.base import BaseCommand
8
-
from letters.services import TopicSuggestionService
9
-
10
-
11
-
class Command(BaseCommand):
12
-
help = 'Test topic-to-constituency mapping with example concerns'
13
-
14
-
def add_arguments(self, parser):
15
-
parser.add_argument(
16
-
'--concern',
17
-
type=str,
18
-
help='Custom concern text to test',
19
-
)
20
-
21
-
def handle(self, *args, **options):
22
-
custom_concern = options.get('concern')
23
-
24
-
if custom_concern:
25
-
# Test custom concern
26
-
self.test_concern(custom_concern)
27
-
else:
28
-
# Test predefined examples
29
-
self.stdout.write(self.style.MIGRATE_HEADING('\nTesting Topic-to-Constituency Mapping\n'))
30
-
31
-
test_cases = [
32
-
"I want to see better train connections between cities",
33
-
"We need more affordable housing and rent control",
34
-
"Our school curriculum needs reform",
35
-
"Climate protection and CO2 emissions must be addressed",
36
-
"Better bus services in my town",
37
-
"Deutsche Bahn is always late",
38
-
"University tuition fees should be abolished",
39
-
"We need stronger EU trade agreements",
40
-
"Police funding in our state is too low",
41
-
"Renewable energy expansion is too slow",
42
-
]
43
-
44
-
for concern in test_cases:
45
-
self.test_concern(concern)
46
-
self.stdout.write('') # Blank line
47
-
48
-
def test_concern(self, concern_text: str):
49
-
"""Test a single concern and display results."""
50
-
self.stdout.write(self.style.SUCCESS(f'Concern: "{concern_text}"'))
51
-
52
-
# Get topic suggestions only (lightweight)
53
-
topics = TopicSuggestionService.get_topic_suggestions(concern_text)
54
-
55
-
if topics:
56
-
self.stdout.write(self.style.WARNING(' Matched Topics:'))
57
-
for topic in topics[:3]: # Show top 3
58
-
self.stdout.write(f' • {topic["name"]} ({topic["level"]}) - Score: {topic["match_score"]}')
59
-
self.stdout.write(f' {topic["description"]}')
60
-
else:
61
-
self.stdout.write(self.style.WARNING(' No specific topics matched'))
62
-
63
-
# Get full suggestions with representatives
64
-
result = TopicSuggestionService.suggest_representatives_for_concern(
65
-
concern_text,
66
-
limit=3
67
-
)
68
-
69
-
self.stdout.write(self.style.WARNING(f' Suggested Level: {result["suggested_level"]}'))
70
-
self.stdout.write(self.style.WARNING(f' Explanation: {result["explanation"]}'))
71
-
72
-
if result['suggested_constituencies']:
73
-
self.stdout.write(self.style.WARNING(' Suggested Constituencies:'))
74
-
for const in result['suggested_constituencies'][:3]:
75
-
self.stdout.write(f' • {const.name} ({const.level})')
76
-
77
-
if result['suggested_representatives']:
78
-
self.stdout.write(self.style.WARNING(' Suggested Representatives:'))
79
-
for rep in result['suggested_representatives']:
80
-
party = f' ({rep.party})' if rep.party else ''
81
-
constituency = rep.primary_constituency
82
-
constituency_label = constituency.name if constituency else rep.parliament.name
83
-
self.stdout.write(f' • {rep.full_name}{party} - {constituency_label}')
84
-
else:
85
-
self.stdout.write(self.style.WARNING(' (No representatives found - run sync_representatives first)'))
···
+35
website/letters/migrations/0012_geocodecache.py
+35
website/letters/migrations/0012_geocodecache.py
···
···
1
+
# Generated by Django 5.2.6 on 2025-10-11 18:11
2
+
3
+
from django.db import migrations, models
4
+
5
+
6
+
class Migration(migrations.Migration):
7
+
8
+
dependencies = [
9
+
('letters', '0011_alter_letter_author'),
10
+
]
11
+
12
+
operations = [
13
+
migrations.CreateModel(
14
+
name='GeocodeCache',
15
+
fields=[
16
+
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
17
+
('address_hash', models.CharField(db_index=True, help_text='SHA256 hash of normalized address for fast lookup', max_length=64, unique=True)),
18
+
('street', models.CharField(blank=True, max_length=255)),
19
+
('postal_code', models.CharField(blank=True, max_length=10)),
20
+
('city', models.CharField(blank=True, max_length=100)),
21
+
('country', models.CharField(default='DE', max_length=2)),
22
+
('latitude', models.FloatField(blank=True, null=True)),
23
+
('longitude', models.FloatField(blank=True, null=True)),
24
+
('success', models.BooleanField(default=True, help_text='False if geocoding failed, to avoid repeated failed lookups')),
25
+
('error_message', models.TextField(blank=True)),
26
+
('created_at', models.DateTimeField(auto_now_add=True)),
27
+
('updated_at', models.DateTimeField(auto_now=True)),
28
+
],
29
+
options={
30
+
'verbose_name': 'Geocode Cache Entry',
31
+
'verbose_name_plural': 'Geocode Cache Entries',
32
+
'ordering': ['-created_at'],
33
+
},
34
+
),
35
+
]
+37
website/letters/models.py
+37
website/letters/models.py
···
768
769
def __str__(self):
770
return f"Report on '{self.letter.title}' - {self.get_reason_display()}"
771
+
772
+
773
+
class GeocodeCache(models.Model):
774
+
"""Cache geocoding results to minimize API calls."""
775
+
776
+
address_hash = models.CharField(
777
+
max_length=64,
778
+
unique=True,
779
+
db_index=True,
780
+
help_text="SHA256 hash of normalized address for fast lookup"
781
+
)
782
+
street = models.CharField(max_length=255, blank=True)
783
+
postal_code = models.CharField(max_length=10, blank=True)
784
+
city = models.CharField(max_length=100, blank=True)
785
+
country = models.CharField(max_length=2, default='DE')
786
+
787
+
latitude = models.FloatField(null=True, blank=True)
788
+
longitude = models.FloatField(null=True, blank=True)
789
+
790
+
success = models.BooleanField(
791
+
default=True,
792
+
help_text="False if geocoding failed, to avoid repeated failed lookups"
793
+
)
794
+
error_message = models.TextField(blank=True)
795
+
796
+
created_at = models.DateTimeField(auto_now_add=True)
797
+
updated_at = models.DateTimeField(auto_now=True)
798
+
799
+
class Meta:
800
+
verbose_name = "Geocode Cache Entry"
801
+
verbose_name_plural = "Geocode Cache Entries"
802
+
ordering = ['-created_at']
803
+
804
+
def __str__(self):
805
+
if self.latitude and self.longitude:
806
+
return f"{self.city} ({self.latitude}, {self.longitude})"
807
+
return f"{self.city} (failed)"
+537
-18
website/letters/services.py
+537
-18
website/letters/services.py
···
11
12
from __future__ import annotations
13
14
import json
15
import logging
16
import re
17
import mimetypes
18
from datetime import datetime, date
19
from dataclasses import dataclass
···
34
Committee,
35
CommitteeMembership,
36
Constituency,
37
Parliament,
38
ParliamentTerm,
39
Representative,
···
125
126
127
# ---------------------------------------------------------------------------
128
# Constituency / address helper
129
# ---------------------------------------------------------------------------
130
···
141
postal_code: Optional[str]
142
state: Optional[str]
143
constituencies: List[Constituency]
144
145
@property
146
def has_constituencies(self) -> bool:
···
176
177
178
class ConstituencyLocator:
179
-
"""Heuristic mapping from postal codes to broad constituencies."""
180
181
-
# Rough PLZ -> state mapping (first two digits).
182
STATE_BY_PLZ_PREFIX: Dict[str, str] = {
183
**{prefix: 'Berlin' for prefix in ['10', '11']},
184
**{prefix: 'Bayern' for prefix in ['80', '81', '82', '83', '84', '85', '86', '87', '88', '89', '90', '91']} ,
···
188
**{prefix: 'Niedersachsen' for prefix in ['26', '27', '28', '29', '30', '31', '32', '33', '37', '38', '49']},
189
}
190
191
@classmethod
192
-
def locate(cls, postal_code: str) -> LocatedConstituencies:
193
postal_code = (postal_code or '').strip()
194
if len(postal_code) < 2:
195
return LocatedConstituencies(None, None, None)
···
1170
from .models import IdentityVerification
1171
1172
postal_code = (verification_data.get('postal_code') or '').strip()
1173
-
located = ConstituencyLocator.locate(postal_code) if postal_code else LocatedConstituencies(None, None, None)
1174
constituency = located.local or located.state or located.federal
1175
1176
expires_at_value = verification_data.get('expires_at')
···
1354
1355
@classmethod
1356
def _resolve_location(cls, user_location: Dict[str, str]) -> LocationContext:
1357
postal_code = (user_location.get('postal_code') or '').strip()
1358
constituencies: List[Constituency] = []
1359
1360
provided_constituencies = user_location.get('constituencies')
1361
if provided_constituencies:
1362
iterable = provided_constituencies if isinstance(provided_constituencies, (list, tuple, set)) else [provided_constituencies]
···
1374
if constituency and all(c.id != constituency.id for c in constituencies):
1375
constituencies.append(constituency)
1376
1377
-
if not constituencies and postal_code:
1378
-
located = ConstituencyLocator.locate(postal_code)
1379
-
constituencies.extend(
1380
-
constituency
1381
-
for constituency in (located.local, located.state, located.federal)
1382
-
if constituency
1383
-
)
1384
-
else:
1385
-
located = LocatedConstituencies(None, None, None)
1386
1387
explicit_state = normalize_german_state(user_location.get('state')) if user_location.get('state') else None
1388
inferred_state = None
1389
for constituency in constituencies:
1390
metadata_state = (constituency.metadata or {}).get('state') if constituency.metadata else None
1391
if metadata_state:
···
1393
if inferred_state:
1394
break
1395
1396
-
if not inferred_state and postal_code and not constituencies and located.state:
1397
-
metadata_state = (located.state.metadata or {}).get('state') if located.state and located.state.metadata else None
1398
-
if metadata_state:
1399
-
inferred_state = normalize_german_state(metadata_state)
1400
-
1401
state = explicit_state or inferred_state
1402
1403
return LocationContext(
1404
postal_code=postal_code or None,
1405
state=state,
1406
constituencies=constituencies,
1407
)
1408
1409
@classmethod
···
11
12
from __future__ import annotations
13
14
+
import hashlib
15
import json
16
import logging
17
import re
18
+
import time
19
import mimetypes
20
from datetime import datetime, date
21
from dataclasses import dataclass
···
36
Committee,
37
CommitteeMembership,
38
Constituency,
39
+
GeocodeCache,
40
Parliament,
41
ParliamentTerm,
42
Representative,
···
128
129
130
# ---------------------------------------------------------------------------
131
+
# Address Geocoding with OSM Nominatim
132
+
# ---------------------------------------------------------------------------
133
+
134
+
135
+
class AddressGeocoder:
136
+
"""
137
+
Geocode German addresses using OpenStreetMap Nominatim API.
138
+
139
+
Features:
140
+
- Caches results using GeocodeCache model
141
+
- Rate limits to 1 request/second for public API compliance
142
+
- Handles errors gracefully
143
+
- Caches both successful and failed lookups to avoid repeated failures
144
+
"""
145
+
146
+
NOMINATIM_ENDPOINT = 'https://nominatim.openstreetmap.org/search'
147
+
USER_AGENT = 'WriteThem.eu/0.1 (civic engagement platform)'
148
+
RATE_LIMIT_SECONDS = 1.0
149
+
150
+
def __init__(self):
151
+
self._last_request_time = 0
152
+
153
+
def geocode(
154
+
self,
155
+
street: str,
156
+
postal_code: str,
157
+
city: str,
158
+
country: str = 'DE'
159
+
) -> Tuple[Optional[float], Optional[float], bool, Optional[str]]:
160
+
"""
161
+
Geocode a German address to latitude/longitude coordinates.
162
+
163
+
Args:
164
+
street: Street name and number
165
+
postal_code: Postal code (PLZ)
166
+
city: City name
167
+
country: Country code (default: 'DE')
168
+
169
+
Returns:
170
+
Tuple of (latitude, longitude, success, error_message)
171
+
- On success: (lat, lon, True, None)
172
+
- On failure: (None, None, False, error_message)
173
+
"""
174
+
# Normalize inputs
175
+
street = (street or '').strip()
176
+
postal_code = (postal_code or '').strip()
177
+
city = (city or '').strip()
178
+
country = (country or 'DE').upper()
179
+
180
+
# Generate cache key
181
+
address_hash = self._generate_cache_key(street, postal_code, city, country)
182
+
183
+
# Check cache first
184
+
cached = self._get_from_cache(address_hash)
185
+
if cached is not None:
186
+
return cached
187
+
188
+
# Make API request with rate limiting
189
+
try:
190
+
self._apply_rate_limit()
191
+
result = self._query_nominatim(street, postal_code, city, country)
192
+
193
+
if result:
194
+
lat, lon = result
195
+
self._store_in_cache(
196
+
address_hash, street, postal_code, city, country,
197
+
lat, lon, success=True, error_message=None
198
+
)
199
+
return lat, lon, True, None
200
+
else:
201
+
error_msg = 'Address not found'
202
+
self._store_in_cache(
203
+
address_hash, street, postal_code, city, country,
204
+
None, None, success=False, error_message=error_msg
205
+
)
206
+
return None, None, False, error_msg
207
+
208
+
except Exception as e:
209
+
error_msg = f'Geocoding API error: {str(e)}'
210
+
logger.warning('Geocoding failed for %s, %s %s: %s', street, postal_code, city, error_msg)
211
+
212
+
# Cache the failure to avoid repeated attempts
213
+
self._store_in_cache(
214
+
address_hash, street, postal_code, city, country,
215
+
None, None, success=False, error_message=error_msg
216
+
)
217
+
return None, None, False, error_msg
218
+
219
+
def _generate_cache_key(
220
+
self,
221
+
street: str,
222
+
postal_code: str,
223
+
city: str,
224
+
country: str
225
+
) -> str:
226
+
"""Generate SHA256 hash of normalized address for cache lookup."""
227
+
# Normalize address components for consistent hashing
228
+
normalized = f"{street}|{postal_code}|{city}|{country}"
229
+
return hashlib.sha256(normalized.encode('utf-8')).hexdigest()
230
+
231
+
def _get_from_cache(
232
+
self,
233
+
address_hash: str
234
+
) -> Optional[Tuple[Optional[float], Optional[float], bool, Optional[str]]]:
235
+
"""Check cache for existing geocoding result."""
236
+
try:
237
+
cache_entry = GeocodeCache.objects.get(address_hash=address_hash)
238
+
if cache_entry.success:
239
+
return cache_entry.latitude, cache_entry.longitude, True, None
240
+
else:
241
+
return None, None, False, cache_entry.error_message
242
+
except GeocodeCache.DoesNotExist:
243
+
return None
244
+
245
+
def _store_in_cache(
246
+
self,
247
+
address_hash: str,
248
+
street: str,
249
+
postal_code: str,
250
+
city: str,
251
+
country: str,
252
+
latitude: Optional[float],
253
+
longitude: Optional[float],
254
+
success: bool,
255
+
error_message: Optional[str]
256
+
) -> None:
257
+
"""Store geocoding result in cache."""
258
+
GeocodeCache.objects.update_or_create(
259
+
address_hash=address_hash,
260
+
defaults={
261
+
'street': street,
262
+
'postal_code': postal_code,
263
+
'city': city,
264
+
'country': country,
265
+
'latitude': latitude,
266
+
'longitude': longitude,
267
+
'success': success,
268
+
'error_message': error_message or '',
269
+
}
270
+
)
271
+
272
+
def _apply_rate_limit(self) -> None:
273
+
"""Ensure we don't exceed 1 request per second."""
274
+
current_time = time.time()
275
+
time_since_last = current_time - self._last_request_time
276
+
277
+
if time_since_last < self.RATE_LIMIT_SECONDS:
278
+
sleep_time = self.RATE_LIMIT_SECONDS - time_since_last
279
+
time.sleep(sleep_time)
280
+
281
+
self._last_request_time = time.time()
282
+
283
+
def _query_nominatim(
284
+
self,
285
+
street: str,
286
+
postal_code: str,
287
+
city: str,
288
+
country: str
289
+
) -> Optional[Tuple[float, float]]:
290
+
"""
291
+
Query Nominatim API for address coordinates.
292
+
293
+
Returns:
294
+
(latitude, longitude) on success, None if not found
295
+
296
+
Raises:
297
+
requests.RequestException on API errors
298
+
"""
299
+
# Build query string
300
+
query_parts = []
301
+
if street:
302
+
query_parts.append(street)
303
+
if postal_code:
304
+
query_parts.append(postal_code)
305
+
if city:
306
+
query_parts.append(city)
307
+
308
+
query = ', '.join(query_parts)
309
+
310
+
params = {
311
+
'q': query,
312
+
'format': 'json',
313
+
'addressdetails': 1,
314
+
'limit': 1,
315
+
'countrycodes': country.lower(),
316
+
}
317
+
318
+
headers = {
319
+
'User-Agent': self.USER_AGENT
320
+
}
321
+
322
+
response = requests.get(
323
+
self.NOMINATIM_ENDPOINT,
324
+
params=params,
325
+
headers=headers,
326
+
timeout=10
327
+
)
328
+
response.raise_for_status()
329
+
330
+
results = response.json()
331
+
332
+
if results and len(results) > 0:
333
+
result = results[0]
334
+
lat = float(result['lat'])
335
+
lon = float(result['lon'])
336
+
return lat, lon
337
+
338
+
return None
339
+
340
+
341
+
# ---------------------------------------------------------------------------
342
+
# Wahlkreis (Constituency) Locator using Point-in-Polygon matching
343
+
# ---------------------------------------------------------------------------
344
+
345
+
346
+
class WahlkreisLocator:
347
+
"""Locate which Wahlkreis (constituency) a coordinate falls within using Shapely."""
348
+
349
+
# Class-level cache for parsed constituencies
350
+
_cached_constituencies = None
351
+
_cached_path = None
352
+
353
+
def __init__(self, geojson_path=None):
354
+
"""
355
+
Load and parse GeoJSON constituencies.
356
+
357
+
Args:
358
+
geojson_path: Path to the GeoJSON file. If None, uses settings.CONSTITUENCY_BOUNDARIES_PATH
359
+
"""
360
+
from shapely.geometry import shape
361
+
362
+
if geojson_path is None:
363
+
geojson_path = settings.CONSTITUENCY_BOUNDARIES_PATH
364
+
365
+
# Use cached constituencies if available and path matches
366
+
if (WahlkreisLocator._cached_constituencies is not None and
367
+
WahlkreisLocator._cached_path == geojson_path):
368
+
self.constituencies = WahlkreisLocator._cached_constituencies
369
+
return
370
+
371
+
# Load and parse GeoJSON file
372
+
self.constituencies = []
373
+
with open(geojson_path, 'r', encoding='utf-8') as f:
374
+
data = json.load(f)
375
+
376
+
# Parse each feature and store geometry with properties
377
+
for feature in data.get('features', []):
378
+
properties = feature.get('properties', {})
379
+
wkr_nr = properties.get('WKR_NR')
380
+
wkr_name = properties.get('WKR_NAME', '')
381
+
land_name = properties.get('LAND_NAME', '')
382
+
383
+
# Parse geometry using Shapely
384
+
geometry = shape(feature['geometry'])
385
+
386
+
# Store as tuple: (wkr_nr, wkr_name, land_name, geometry)
387
+
self.constituencies.append((wkr_nr, wkr_name, land_name, geometry))
388
+
389
+
# Cache the parsed constituencies
390
+
WahlkreisLocator._cached_constituencies = self.constituencies
391
+
WahlkreisLocator._cached_path = geojson_path
392
+
393
+
def locate(self, latitude, longitude):
394
+
"""
395
+
Find constituency containing the given coordinates.
396
+
397
+
Args:
398
+
latitude: Latitude coordinate
399
+
longitude: Longitude coordinate
400
+
401
+
Returns:
402
+
tuple: (wkr_nr, wkr_name, land_name) or None if not found
403
+
"""
404
+
from shapely.geometry import Point
405
+
406
+
# Create point from coordinates
407
+
point = Point(longitude, latitude)
408
+
409
+
# Iterate through constituencies and check containment
410
+
for wkr_nr, wkr_name, land_name, geometry in self.constituencies:
411
+
if geometry.contains(point):
412
+
return (wkr_nr, wkr_name, land_name)
413
+
414
+
# No match found
415
+
return None
416
+
417
+
418
+
# ---------------------------------------------------------------------------
419
# Constituency / address helper
420
# ---------------------------------------------------------------------------
421
···
432
postal_code: Optional[str]
433
state: Optional[str]
434
constituencies: List[Constituency]
435
+
street: Optional[str] = None
436
+
city: Optional[str] = None
437
+
country: str = 'DE'
438
439
@property
440
def has_constituencies(self) -> bool:
···
470
471
472
class ConstituencyLocator:
473
+
"""
474
+
Locate representatives by address or postal code.
475
+
476
+
Features:
477
+
- Address-based lookup: Uses AddressGeocoder + WahlkreisLocator for accurate constituency matching
478
+
- PLZ-based fallback: Falls back to PLZ-prefix matching when geocoding fails
479
+
- Backward compatible: Still accepts PLZ-only queries
480
+
"""
481
482
+
# Rough PLZ -> state mapping (first two digits) for fallback
483
STATE_BY_PLZ_PREFIX: Dict[str, str] = {
484
**{prefix: 'Berlin' for prefix in ['10', '11']},
485
**{prefix: 'Bayern' for prefix in ['80', '81', '82', '83', '84', '85', '86', '87', '88', '89', '90', '91']} ,
···
489
**{prefix: 'Niedersachsen' for prefix in ['26', '27', '28', '29', '30', '31', '32', '33', '37', '38', '49']},
490
}
491
492
+
def __init__(self):
493
+
"""Initialize geocoder and wahlkreis locator services."""
494
+
self._geocoder = None
495
+
self._wahlkreis_locator = None
496
+
497
+
@property
498
+
def geocoder(self):
499
+
"""Lazy-load AddressGeocoder."""
500
+
if self._geocoder is None:
501
+
self._geocoder = AddressGeocoder()
502
+
return self._geocoder
503
+
504
+
@property
505
+
def wahlkreis_locator(self):
506
+
"""Lazy-load WahlkreisLocator."""
507
+
if self._wahlkreis_locator is None:
508
+
self._wahlkreis_locator = WahlkreisLocator()
509
+
return self._wahlkreis_locator
510
+
511
+
def locate(
512
+
self,
513
+
street: Optional[str] = None,
514
+
postal_code: Optional[str] = None,
515
+
city: Optional[str] = None,
516
+
country: str = 'DE'
517
+
) -> List[Representative]:
518
+
"""
519
+
Locate representatives by address or postal code.
520
+
521
+
Args:
522
+
street: Street name and number (optional)
523
+
postal_code: Postal code / PLZ (optional)
524
+
city: City name (optional)
525
+
country: Country code (default: 'DE')
526
+
527
+
Returns:
528
+
List of Representative objects for the located constituency
529
+
530
+
Strategy:
531
+
1. If full address provided (street + postal_code + city):
532
+
- Geocode address to coordinates
533
+
- Use WahlkreisLocator to find constituency
534
+
- Return Representatives for that constituency
535
+
2. Fallback to PLZ-prefix matching if:
536
+
- No street provided
537
+
- Geocoding fails
538
+
- WahlkreisLocator returns no result
539
+
"""
540
+
street = (street or '').strip()
541
+
postal_code = (postal_code or '').strip()
542
+
city = (city or '').strip()
543
+
544
+
# Try full address-based lookup if we have all components
545
+
if street and postal_code and city:
546
+
try:
547
+
lat, lon, success, error = self.geocoder.geocode(street, postal_code, city, country)
548
+
549
+
if success and lat is not None and lon is not None:
550
+
# Find constituency using coordinates
551
+
result = self.wahlkreis_locator.locate(lat, lon)
552
+
553
+
if result:
554
+
wkr_nr, wkr_name, land_name = result
555
+
logger.info(
556
+
"Address geocoded to constituency: %s (WK %s, %s)",
557
+
wkr_name, wkr_nr, land_name
558
+
)
559
+
560
+
# Find Representatives for this Wahlkreis
561
+
representatives = self._find_representatives_by_wahlkreis(
562
+
wkr_nr, wkr_name, land_name
563
+
)
564
+
565
+
if representatives:
566
+
return representatives
567
+
568
+
# If no representatives found for direct constituency,
569
+
# fall through to PLZ-based lookup
570
+
logger.warning(
571
+
"No representatives found for WK %s, falling back to PLZ",
572
+
wkr_nr
573
+
)
574
+
else:
575
+
logger.debug(
576
+
"Geocoding failed for %s, %s %s: %s",
577
+
street, postal_code, city, error
578
+
)
579
+
except Exception as e:
580
+
logger.warning(
581
+
"Error during address-based lookup for %s, %s %s: %s",
582
+
street, postal_code, city, e
583
+
)
584
+
585
+
# Fallback to PLZ-based lookup
586
+
if postal_code:
587
+
return self._locate_by_plz(postal_code)
588
+
589
+
# No parameters provided
590
+
return []
591
+
592
+
def _find_representatives_by_wahlkreis(
593
+
self,
594
+
wkr_nr: int,
595
+
wkr_name: str,
596
+
land_name: str
597
+
) -> List[Representative]:
598
+
"""
599
+
Find representatives for a given Wahlkreis.
600
+
601
+
Strategy:
602
+
1. Look for constituencies with matching WKR_NR in metadata
603
+
2. Look for constituencies with matching name
604
+
3. Return active representatives from matched constituencies
605
+
"""
606
+
# Try to find constituency by WKR_NR in metadata
607
+
constituencies = Constituency.objects.filter(
608
+
metadata__WKR_NR=wkr_nr,
609
+
scope='FEDERAL_DISTRICT'
610
+
).prefetch_related('representatives')
611
+
612
+
if not constituencies.exists():
613
+
# Try by name matching
614
+
constituencies = Constituency.objects.filter(
615
+
name__icontains=str(wkr_nr),
616
+
scope='FEDERAL_DISTRICT'
617
+
).prefetch_related('representatives')
618
+
619
+
if not constituencies.exists():
620
+
# Try finding by state and scope
621
+
normalized_state = normalize_german_state(land_name)
622
+
if normalized_state:
623
+
constituencies = Constituency.objects.filter(
624
+
metadata__state=normalized_state,
625
+
scope__in=['FEDERAL_DISTRICT', 'FEDERAL_STATE_LIST']
626
+
).prefetch_related('representatives')
627
+
628
+
# Collect all representatives from matched constituencies
629
+
representatives = []
630
+
for constituency in constituencies:
631
+
reps = list(constituency.representatives.filter(is_active=True))
632
+
representatives.extend(reps)
633
+
634
+
# Remove duplicates while preserving order
635
+
seen = set()
636
+
unique_reps = []
637
+
for rep in representatives:
638
+
if rep.id not in seen:
639
+
seen.add(rep.id)
640
+
unique_reps.append(rep)
641
+
642
+
return unique_reps
643
+
644
+
def _locate_by_plz(self, postal_code: str) -> List[Representative]:
645
+
"""
646
+
Fallback: Locate representatives using PLZ-prefix matching.
647
+
648
+
Returns list of Representatives instead of LocatedConstituencies.
649
+
"""
650
+
if len(postal_code) < 2:
651
+
return []
652
+
653
+
prefix = postal_code[:2]
654
+
state_name = self.STATE_BY_PLZ_PREFIX.get(prefix)
655
+
normalized_state = normalize_german_state(state_name) if state_name else None
656
+
657
+
federal = self._match_federal(normalized_state)
658
+
state = self._match_state(normalized_state)
659
+
660
+
# Convert constituencies to representatives
661
+
representatives = []
662
+
for constituency in [federal, state]:
663
+
if constituency:
664
+
reps = list(constituency.representatives.filter(is_active=True))
665
+
representatives.extend(reps)
666
+
667
+
# Remove duplicates
668
+
seen = set()
669
+
unique_reps = []
670
+
for rep in representatives:
671
+
if rep.id not in seen:
672
+
seen.add(rep.id)
673
+
unique_reps.append(rep)
674
+
675
+
return unique_reps
676
+
677
@classmethod
678
+
def locate_legacy(cls, postal_code: str) -> LocatedConstituencies:
679
+
"""
680
+
Legacy method: Returns LocatedConstituencies for backward compatibility.
681
+
682
+
This method maintains the old API for existing code that expects
683
+
LocatedConstituencies instead of List[Representative].
684
+
"""
685
postal_code = (postal_code or '').strip()
686
if len(postal_code) < 2:
687
return LocatedConstituencies(None, None, None)
···
1662
from .models import IdentityVerification
1663
1664
postal_code = (verification_data.get('postal_code') or '').strip()
1665
+
located = ConstituencyLocator.locate_legacy(postal_code) if postal_code else LocatedConstituencies(None, None, None)
1666
constituency = located.local or located.state or located.federal
1667
1668
expires_at_value = verification_data.get('expires_at')
···
1846
1847
@classmethod
1848
def _resolve_location(cls, user_location: Dict[str, str]) -> LocationContext:
1849
+
# Extract address components
1850
postal_code = (user_location.get('postal_code') or '').strip()
1851
+
street = (user_location.get('street') or '').strip()
1852
+
city = (user_location.get('city') or '').strip()
1853
+
country = (user_location.get('country') or 'DE').upper()
1854
+
1855
constituencies: List[Constituency] = []
1856
1857
+
# First, check if constituencies are provided directly
1858
provided_constituencies = user_location.get('constituencies')
1859
if provided_constituencies:
1860
iterable = provided_constituencies if isinstance(provided_constituencies, (list, tuple, set)) else [provided_constituencies]
···
1872
if constituency and all(c.id != constituency.id for c in constituencies):
1873
constituencies.append(constituency)
1874
1875
+
# If no constituencies provided, try address-based or PLZ-based lookup
1876
+
if not constituencies:
1877
+
locator = ConstituencyLocator()
1878
+
1879
+
# Try full address lookup if available
1880
+
if street and postal_code and city:
1881
+
# Use new address-based API
1882
+
representatives = locator.locate(
1883
+
street=street,
1884
+
postal_code=postal_code,
1885
+
city=city,
1886
+
country=country
1887
+
)
1888
+
1889
+
# Extract unique constituencies from representatives
1890
+
constituency_ids_seen = set()
1891
+
for rep in representatives:
1892
+
for constituency in rep.constituencies.all():
1893
+
if constituency.id not in constituency_ids_seen:
1894
+
constituencies.append(constituency)
1895
+
constituency_ids_seen.add(constituency.id)
1896
+
1897
+
# Fallback to PLZ-only if no full address or if address lookup failed
1898
+
if not constituencies and postal_code:
1899
+
located = ConstituencyLocator.locate_legacy(postal_code)
1900
+
constituencies.extend(
1901
+
constituency
1902
+
for constituency in (located.local, located.state, located.federal)
1903
+
if constituency
1904
+
)
1905
1906
+
# Determine state from various sources
1907
explicit_state = normalize_german_state(user_location.get('state')) if user_location.get('state') else None
1908
inferred_state = None
1909
+
1910
for constituency in constituencies:
1911
metadata_state = (constituency.metadata or {}).get('state') if constituency.metadata else None
1912
if metadata_state:
···
1914
if inferred_state:
1915
break
1916
1917
state = explicit_state or inferred_state
1918
1919
return LocationContext(
1920
postal_code=postal_code or None,
1921
state=state,
1922
constituencies=constituencies,
1923
+
street=street or None,
1924
+
city=city or None,
1925
+
country=country,
1926
)
1927
1928
@classmethod
+33
website/letters/templates/letters/profile.html
+33
website/letters/templates/letters/profile.html
···
52
</div>
53
54
<div class="mt-4">
55
+
<h3>{% trans "Ihre Adresse" %}</h3>
56
+
<p class="text-muted">
57
+
{% trans "Geben Sie Ihre vollständige Adresse ein, um präzise Wahlkreis- und Abgeordnetenempfehlungen zu erhalten." %}
58
+
</p>
59
+
{% if verification and verification.street_address %}
60
+
<div class="alert alert-info" role="status">
61
+
<strong>{% trans "Gespeicherte Adresse:" %}</strong><br>
62
+
{{ verification.street_address }}<br>
63
+
{{ verification.postal_code }} {{ verification.city }}
64
+
</div>
65
+
{% endif %}
66
+
<form method="post" class="mt-3">
67
+
{% csrf_token %}
68
+
{% if address_form.non_field_errors %}
69
+
<div class="alert alert-danger">{{ address_form.non_field_errors }}</div>
70
+
{% endif %}
71
+
{% for field in address_form %}
72
+
<div class="form-group mb-3">
73
+
<label for="{{ field.id_for_label }}" class="form-label">{{ field.label }}</label>
74
+
{{ field }}
75
+
{% if field.errors %}
76
+
<div class="text-danger small">{{ field.errors|join:', ' }}</div>
77
+
{% endif %}
78
+
{% if field.help_text %}
79
+
<small class="form-text text-muted">{{ field.help_text }}</small>
80
+
{% endif %}
81
+
</div>
82
+
{% endfor %}
83
+
<button type="submit" name="address_form_submit" class="btn btn-primary">{% trans "Adresse speichern" %}</button>
84
+
</form>
85
+
</div>
86
+
87
+
<div class="mt-4">
88
<h3>{% trans "Self-declare your constituency" %}</h3>
89
<p class="text-muted">
90
{% trans "Select the constituencies you live in so we can prioritise the right representatives." %}
+2
website/letters/tests/__init__.py
+2
website/letters/tests/__init__.py
+219
website/letters/tests/test_address_matching.py
+219
website/letters/tests/test_address_matching.py
···
···
1
+
# ABOUTME: Test address-based constituency matching with geocoding and point-in-polygon lookup.
2
+
# ABOUTME: Covers AddressGeocoder, WahlkreisLocator, and ConstituencyLocator services.
3
+
4
+
from django.test import TestCase
5
+
from unittest.mock import patch, MagicMock
6
+
from letters.services import AddressGeocoder, WahlkreisLocator, ConstituencyLocator
7
+
from letters.models import GeocodeCache, Representative
8
+
9
+
10
+
# Test addresses covering all German states
11
+
TEST_ADDRESSES = [
12
+
{
13
+
'name': 'Bundestag (Berlin)',
14
+
'street': 'Platz der Republik 1',
15
+
'postal_code': '11011',
16
+
'city': 'Berlin',
17
+
'expected_state': 'Berlin'
18
+
},
19
+
{
20
+
'name': 'Hamburg Rathaus',
21
+
'street': 'Rathausmarkt 1',
22
+
'postal_code': '20095',
23
+
'city': 'Hamburg',
24
+
'expected_state': 'Hamburg'
25
+
},
26
+
{
27
+
'name': 'Marienplatz München (Bavaria)',
28
+
'street': 'Marienplatz 1',
29
+
'postal_code': '80331',
30
+
'city': 'München',
31
+
'expected_state': 'Bayern'
32
+
},
33
+
{
34
+
'name': 'Kölner Dom (North Rhine-Westphalia)',
35
+
'street': 'Domkloster 4',
36
+
'postal_code': '50667',
37
+
'city': 'Köln',
38
+
'expected_state': 'Nordrhein-Westfalen'
39
+
},
40
+
{
41
+
'name': 'Brandenburger Tor (Berlin)',
42
+
'street': 'Pariser Platz',
43
+
'postal_code': '10117',
44
+
'city': 'Berlin',
45
+
'expected_state': 'Berlin'
46
+
},
47
+
]
48
+
49
+
50
+
class AddressGeocodingTests(TestCase):
51
+
"""Test address geocoding with OSM Nominatim."""
52
+
53
+
def setUp(self):
54
+
self.geocoder = AddressGeocoder()
55
+
56
+
def test_geocode_success_with_mocked_api(self):
57
+
"""Test successful geocoding with mocked Nominatim response."""
58
+
with patch('requests.get') as mock_get:
59
+
# Mock successful Nominatim response
60
+
mock_response = MagicMock()
61
+
mock_response.status_code = 200
62
+
mock_response.json.return_value = [{
63
+
'lat': '52.5186',
64
+
'lon': '13.3761'
65
+
}]
66
+
mock_get.return_value = mock_response
67
+
68
+
lat, lon, success, error = self.geocoder.geocode(
69
+
'Platz der Republik 1',
70
+
'11011',
71
+
'Berlin'
72
+
)
73
+
74
+
self.assertTrue(success)
75
+
self.assertIsNone(error)
76
+
self.assertAlmostEqual(lat, 52.5186, places=4)
77
+
self.assertAlmostEqual(lon, 13.3761, places=4)
78
+
79
+
def test_geocode_caches_results(self):
80
+
"""Test that geocoding results are cached in database."""
81
+
with patch('requests.get') as mock_get:
82
+
mock_response = MagicMock()
83
+
mock_response.status_code = 200
84
+
mock_response.json.return_value = [{
85
+
'lat': '52.5186',
86
+
'lon': '13.3761'
87
+
}]
88
+
mock_get.return_value = mock_response
89
+
90
+
# First call should cache
91
+
self.geocoder.geocode('Platz der Republik 1', '11011', 'Berlin')
92
+
93
+
# Check cache entry exists
94
+
cache_key = self.geocoder._generate_cache_key(
95
+
'Platz der Republik 1', '11011', 'Berlin', 'DE'
96
+
)
97
+
cache_entry = GeocodeCache.objects.filter(address_hash=cache_key).first()
98
+
self.assertIsNotNone(cache_entry)
99
+
self.assertTrue(cache_entry.success)
100
+
101
+
def test_geocode_returns_cached_results(self):
102
+
"""Test that cached geocoding results are reused."""
103
+
# Create cache entry
104
+
cache_key = self.geocoder._generate_cache_key(
105
+
'Test Street', '12345', 'Test City', 'DE'
106
+
)
107
+
GeocodeCache.objects.create(
108
+
address_hash=cache_key,
109
+
success=True,
110
+
latitude=52.0,
111
+
longitude=13.0
112
+
)
113
+
114
+
# Should return cached result without API call
115
+
with patch('requests.get') as mock_get:
116
+
lat, lon, success, error = self.geocoder.geocode(
117
+
'Test Street', '12345', 'Test City'
118
+
)
119
+
120
+
# Verify no API call was made
121
+
mock_get.assert_not_called()
122
+
123
+
# Verify cached results returned
124
+
self.assertTrue(success)
125
+
self.assertEqual(lat, 52.0)
126
+
self.assertEqual(lon, 13.0)
127
+
128
+
def test_geocode_handles_api_error(self):
129
+
"""Test graceful handling of Nominatim API errors."""
130
+
with patch('requests.get') as mock_get:
131
+
mock_get.side_effect = Exception("API Error")
132
+
133
+
# Capture expected warning log
134
+
with self.assertLogs('letters.services', level='WARNING') as log_context:
135
+
lat, lon, success, error = self.geocoder.geocode(
136
+
'Invalid Street', '99999', 'Nowhere'
137
+
)
138
+
139
+
self.assertFalse(success)
140
+
self.assertIsNone(lat)
141
+
self.assertIsNone(lon)
142
+
self.assertIn('API Error', error)
143
+
# Verify expected warning was logged
144
+
self.assertEqual(len(log_context.output), 1)
145
+
self.assertIn('Geocoding failed', log_context.output[0])
146
+
147
+
148
+
class WahlkreisLocationTests(TestCase):
149
+
"""Test point-in-polygon constituency matching."""
150
+
151
+
def test_locate_bundestag_coordinates(self):
152
+
"""Test that Bundestag coordinates find correct Berlin constituency."""
153
+
locator = WahlkreisLocator()
154
+
result = locator.locate(52.5186, 13.3761)
155
+
156
+
self.assertIsNotNone(result)
157
+
wkr_nr, wkr_name, land_name = result
158
+
self.assertIsInstance(wkr_nr, int)
159
+
self.assertIn('Berlin', land_name)
160
+
161
+
def test_locate_hamburg_coordinates(self):
162
+
"""Test that Hamburg coordinates find correct constituency."""
163
+
locator = WahlkreisLocator()
164
+
result = locator.locate(53.5511, 9.9937)
165
+
166
+
self.assertIsNotNone(result)
167
+
wkr_nr, wkr_name, land_name = result
168
+
self.assertIsInstance(wkr_nr, int)
169
+
self.assertIn('Hamburg', land_name)
170
+
171
+
def test_coordinates_outside_germany(self):
172
+
"""Test that coordinates outside Germany return None."""
173
+
locator = WahlkreisLocator()
174
+
175
+
# Paris coordinates
176
+
result = locator.locate(48.8566, 2.3522)
177
+
self.assertIsNone(result)
178
+
179
+
# London coordinates
180
+
result = locator.locate(51.5074, -0.1278)
181
+
self.assertIsNone(result)
182
+
183
+
184
+
class FullAddressMatchingTests(TestCase):
185
+
"""Integration tests for full address → constituency → representatives pipeline."""
186
+
187
+
@patch('letters.services.AddressGeocoder.geocode')
188
+
def test_address_to_constituency_pipeline(self, mock_geocode):
189
+
"""Test full pipeline from address to constituency with mocked geocoding."""
190
+
# Mock geocoding to return Bundestag coordinates
191
+
mock_geocode.return_value = (52.5186, 13.3761, True, None)
192
+
193
+
locator = ConstituencyLocator()
194
+
representatives = locator.locate(
195
+
street='Platz der Republik 1',
196
+
postal_code='11011',
197
+
city='Berlin'
198
+
)
199
+
200
+
# Should return representatives (even if list is empty due to no DB data)
201
+
self.assertIsInstance(representatives, list)
202
+
mock_geocode.assert_called_once()
203
+
204
+
def test_plz_fallback_when_geocoding_fails(self):
205
+
"""Test PLZ prefix fallback when geocoding fails."""
206
+
with patch('letters.services.AddressGeocoder.geocode') as mock_geocode:
207
+
# Mock geocoding failure
208
+
mock_geocode.return_value = (None, None, False, "Geocoding failed")
209
+
210
+
locator = ConstituencyLocator()
211
+
representatives = locator.locate(
212
+
postal_code='10115' # Berlin postal code
213
+
)
214
+
215
+
# Should still return list (using PLZ fallback)
216
+
self.assertIsInstance(representatives, list)
217
+
218
+
219
+
# End of file
+58
website/letters/tests/test_constituency_suggestions.py
+58
website/letters/tests/test_constituency_suggestions.py
···
···
1
+
# ABOUTME: Test ConstituencySuggestionService combining topics and geography.
2
+
# ABOUTME: Integration tests for letter title/address to representative suggestions.
3
+
4
+
from django.test import TestCase
5
+
from unittest.mock import patch
6
+
from letters.services import ConstituencySuggestionService
7
+
8
+
9
+
class ConstituencySuggestionTests(TestCase):
10
+
"""Test constituency suggestion combining topic and address matching."""
11
+
12
+
@patch('letters.services.AddressGeocoder.geocode')
13
+
def test_suggest_with_title_and_address(self, mock_geocode):
14
+
"""Test suggestions work with both title and address."""
15
+
# Mock geocoding
16
+
mock_geocode.return_value = (52.5186, 13.3761, True, None)
17
+
18
+
result = ConstituencySuggestionService.suggest_from_concern(
19
+
concern_text="We need better train connections",
20
+
user_location={
21
+
"street": "Platz der Republik 1",
22
+
"postal_code": "11011",
23
+
"city": "Berlin"
24
+
}
25
+
)
26
+
27
+
self.assertIn('matched_topics', result)
28
+
self.assertIn('suggested_level', result)
29
+
self.assertIn('explanation', result)
30
+
self.assertIn('representatives', result)
31
+
self.assertIn('constituencies', result)
32
+
33
+
def test_suggest_with_only_title(self):
34
+
"""Test suggestions work with only title (no address)."""
35
+
result = ConstituencySuggestionService.suggest_from_concern(
36
+
concern_text="Climate protection is important"
37
+
)
38
+
39
+
self.assertIn('matched_topics', result)
40
+
self.assertIn('suggested_level', result)
41
+
# Without address, should still suggest level and topics
42
+
self.assertIsNotNone(result['suggested_level'])
43
+
44
+
def test_suggest_with_only_postal_code(self):
45
+
"""Test suggestions work with only postal code."""
46
+
result = ConstituencySuggestionService.suggest_from_concern(
47
+
concern_text="Local infrastructure problems",
48
+
user_location={
49
+
"postal_code": "10115"
50
+
}
51
+
)
52
+
53
+
self.assertIn('constituencies', result)
54
+
# Should use PLZ fallback
55
+
self.assertIsInstance(result['constituencies'], list)
56
+
57
+
58
+
# End of file
+101
website/letters/tests/test_topic_mapping.py
+101
website/letters/tests/test_topic_mapping.py
···
···
1
+
# ABOUTME: Test topic suggestion and matching based on letter content.
2
+
# ABOUTME: Covers TopicSuggestionService keyword matching and level suggestion logic.
3
+
4
+
from django.test import TestCase
5
+
from letters.services import TopicSuggestionService
6
+
from letters.models import TopicArea
7
+
8
+
9
+
class TopicMatchingTests(TestCase):
10
+
"""Test topic keyword matching and scoring."""
11
+
12
+
def setUp(self):
13
+
"""Check if topic data is available."""
14
+
self.has_topics = TopicArea.objects.exists()
15
+
16
+
def test_transport_keywords_match_verkehr_topic(self):
17
+
"""Test that transport-related keywords match Verkehr topic."""
18
+
if not self.has_topics:
19
+
self.skipTest("TopicArea data not loaded")
20
+
21
+
concern = "I want to see better train connections between cities"
22
+
result = TopicSuggestionService.suggest_representatives_for_concern(concern)
23
+
24
+
# Should find at least one topic
25
+
matched_topics = result.get('matched_topics', [])
26
+
self.assertGreater(len(matched_topics), 0)
27
+
28
+
def test_housing_keywords_match_wohnen_topic(self):
29
+
"""Test that housing keywords match Wohnen topic."""
30
+
if not self.has_topics:
31
+
self.skipTest("TopicArea data not loaded")
32
+
33
+
concern = "We need more affordable housing and rent control"
34
+
result = TopicSuggestionService.suggest_representatives_for_concern(concern)
35
+
36
+
matched_topics = result.get('matched_topics', [])
37
+
self.assertGreater(len(matched_topics), 0)
38
+
39
+
def test_education_keywords_match_bildung_topic(self):
40
+
"""Test that education keywords match Bildung topic."""
41
+
if not self.has_topics:
42
+
self.skipTest("TopicArea data not loaded")
43
+
44
+
concern = "Our school curriculum needs reform"
45
+
result = TopicSuggestionService.suggest_representatives_for_concern(concern)
46
+
47
+
matched_topics = result.get('matched_topics', [])
48
+
self.assertGreater(len(matched_topics), 0)
49
+
50
+
def test_climate_keywords_match_umwelt_topic(self):
51
+
"""Test that climate keywords match environment topic."""
52
+
if not self.has_topics:
53
+
self.skipTest("TopicArea data not loaded")
54
+
55
+
concern = "Climate protection and CO2 emissions must be addressed"
56
+
result = TopicSuggestionService.suggest_representatives_for_concern(concern)
57
+
58
+
matched_topics = result.get('matched_topics', [])
59
+
self.assertGreater(len(matched_topics), 0)
60
+
61
+
def test_no_match_returns_empty_list(self):
62
+
"""Test that completely unrelated text returns empty list."""
63
+
concern = "xyzabc nonsense gibberish"
64
+
result = TopicSuggestionService.suggest_representatives_for_concern(concern)
65
+
66
+
matched_topics = result.get('matched_topics', [])
67
+
# Should return empty list for gibberish
68
+
self.assertEqual(len(matched_topics), 0)
69
+
70
+
71
+
class LevelSuggestionTests(TestCase):
72
+
"""Test government level suggestion logic."""
73
+
74
+
def test_federal_transport_suggests_federal_level(self):
75
+
"""Test that long-distance transport suggests federal level."""
76
+
result = TopicSuggestionService.suggest_representatives_for_concern(
77
+
"Deutsche Bahn is always late",
78
+
limit=5
79
+
)
80
+
81
+
self.assertIn('suggested_level', result)
82
+
self.assertIn('explanation', result)
83
+
# Federal issues should suggest FEDERAL level
84
+
suggested_level = result['suggested_level']
85
+
self.assertIsNotNone(suggested_level)
86
+
self.assertIn('FEDERAL', suggested_level)
87
+
88
+
def test_local_bus_suggests_state_or_local(self):
89
+
"""Test that local transport suggests state/local level."""
90
+
result = TopicSuggestionService.suggest_representatives_for_concern(
91
+
"Better bus services in my town",
92
+
limit=5
93
+
)
94
+
95
+
self.assertIn('suggested_level', result)
96
+
self.assertIn('explanation', result)
97
+
# Should have an explanation
98
+
self.assertIsNotNone(result['explanation'])
99
+
100
+
101
+
# End of file
+697
website/letters/tests.py
+697
website/letters/tests.py
···
688
self.assertNotIn(self.state_rep_list.last_name, content)
689
self.assertIn(self.federal_expert_rep.last_name, content)
690
691
692
class CompetencyPageTests(TestCase):
693
"""Ensure the competency overview renders topics for visitors."""
···
731
self.assertIn('<li>Zwei</li>', rendered)
732
733
734
class RepresentativeMetadataExtractionTests(TestCase):
735
736
def setUp(self):
···
795
service._ensure_photo_reference(rep)
796
rep.refresh_from_db()
797
self.assertEqual(rep.photo_path, 'representatives/999.jpg')
···
688
self.assertNotIn(self.state_rep_list.last_name, content)
689
self.assertIn(self.federal_expert_rep.last_name, content)
690
691
+
def test_suggest_with_full_address(self):
692
+
"""Test that suggestions work with full address (street, postal_code, city)."""
693
+
from unittest.mock import patch, Mock
694
+
695
+
# Mock geocoding to return coordinates for Bundestag building
696
+
mock_response = Mock()
697
+
mock_response.status_code = 200
698
+
mock_response.json.return_value = [
699
+
{
700
+
'lat': '52.5186',
701
+
'lon': '13.3761',
702
+
'display_name': 'Berlin, Germany'
703
+
}
704
+
]
705
+
706
+
with patch('requests.get', return_value=mock_response):
707
+
result = ConstituencySuggestionService.suggest_from_concern(
708
+
'Mehr Verkehr und ÖPNV in Berlin Mitte',
709
+
user_location={
710
+
'street': 'Platz der Republik 1',
711
+
'postal_code': '11011',
712
+
'city': 'Berlin',
713
+
'country': 'DE'
714
+
}
715
+
)
716
+
717
+
# Should find representatives (direct representatives from Berlin)
718
+
self.assertGreater(len(result['representatives']), 0)
719
+
self.assertIn(self.transport_topic, result['matched_topics'])
720
+
721
+
# Should have direct representatives from Berlin
722
+
direct_reps = result.get('direct_representatives', [])
723
+
# At least one Berlin rep should be suggested
724
+
berlin_reps = [
725
+
rep for rep in direct_reps
726
+
if any(
727
+
(c.metadata or {}).get('state') == 'Berlin'
728
+
for c in rep.constituencies.all()
729
+
)
730
+
]
731
+
self.assertGreater(len(berlin_reps), 0, "Should suggest at least one Berlin representative")
732
+
733
+
def test_suggest_with_plz_only_backward_compatibility(self):
734
+
"""Test that PLZ-only suggestions still work (backward compatibility)."""
735
+
result = ConstituencySuggestionService.suggest_from_concern(
736
+
'Klimaschutz ist wichtig',
737
+
user_location={
738
+
'postal_code': '10115', # Berlin PLZ
739
+
}
740
+
)
741
+
742
+
# Should work without crashing
743
+
self.assertIsInstance(result, dict)
744
+
self.assertIn('representatives', result)
745
+
self.assertIn('matched_topics', result)
746
+
747
+
# Result should be valid even if empty
748
+
self.assertIsInstance(result['representatives'], list)
749
+
750
751
class CompetencyPageTests(TestCase):
752
"""Ensure the competency overview renders topics for visitors."""
···
790
self.assertIn('<li>Zwei</li>', rendered)
791
792
793
+
class GeocodeCacheTests(TestCase):
794
+
"""Test geocoding cache model."""
795
+
796
+
def test_cache_stores_and_retrieves_coordinates(self):
797
+
from .models import GeocodeCache
798
+
799
+
cache_entry = GeocodeCache.objects.create(
800
+
address_hash='test_hash_123',
801
+
street='Unter den Linden 77',
802
+
postal_code='10117',
803
+
city='Berlin',
804
+
latitude=52.5170365,
805
+
longitude=13.3888599,
806
+
)
807
+
808
+
retrieved = GeocodeCache.objects.get(address_hash='test_hash_123')
809
+
self.assertEqual(retrieved.latitude, 52.5170365)
810
+
self.assertEqual(retrieved.longitude, 13.3888599)
811
+
self.assertEqual(retrieved.street, 'Unter den Linden 77')
812
+
813
+
814
+
class AddressGeocoderTests(TestCase):
815
+
"""Test the AddressGeocoder service with OSM Nominatim API."""
816
+
817
+
def test_successful_geocoding_with_mocked_api(self):
818
+
"""Test successful geocoding with mocked Nominatim API response."""
819
+
from unittest.mock import patch, Mock
820
+
from .services import AddressGeocoder
821
+
822
+
mock_response = Mock()
823
+
mock_response.status_code = 200
824
+
mock_response.json.return_value = [
825
+
{
826
+
'lat': '52.5200066',
827
+
'lon': '13.404954',
828
+
'display_name': 'Berlin, Germany'
829
+
}
830
+
]
831
+
832
+
with patch('requests.get', return_value=mock_response) as mock_get:
833
+
geocoder = AddressGeocoder()
834
+
lat, lon, success, error = geocoder.geocode(
835
+
street='Unter den Linden 77',
836
+
postal_code='10117',
837
+
city='Berlin'
838
+
)
839
+
840
+
self.assertTrue(success)
841
+
self.assertIsNone(error)
842
+
self.assertAlmostEqual(lat, 52.5200066, places=6)
843
+
self.assertAlmostEqual(lon, 13.404954, places=6)
844
+
845
+
# Verify the request was made correctly
846
+
mock_get.assert_called_once()
847
+
call_args = mock_get.call_args
848
+
self.assertIn('nominatim.openstreetmap.org', call_args[0][0])
849
+
self.assertEqual(call_args[1]['headers']['User-Agent'], 'WriteThem.eu/0.1 (civic engagement platform)')
850
+
851
+
def test_cache_hit_no_api_call(self):
852
+
"""Test that cache hits don't make API calls."""
853
+
from unittest.mock import patch
854
+
from .models import GeocodeCache
855
+
from .services import AddressGeocoder
856
+
import hashlib
857
+
858
+
# Create a cache entry
859
+
address_string = 'Unter den Linden 77|10117|Berlin|DE'
860
+
address_hash = hashlib.sha256(address_string.encode('utf-8')).hexdigest()
861
+
862
+
GeocodeCache.objects.create(
863
+
address_hash=address_hash,
864
+
street='Unter den Linden 77',
865
+
postal_code='10117',
866
+
city='Berlin',
867
+
country='DE',
868
+
latitude=52.5200066,
869
+
longitude=13.404954,
870
+
success=True
871
+
)
872
+
873
+
with patch('requests.get') as mock_get:
874
+
geocoder = AddressGeocoder()
875
+
lat, lon, success, error = geocoder.geocode(
876
+
street='Unter den Linden 77',
877
+
postal_code='10117',
878
+
city='Berlin'
879
+
)
880
+
881
+
self.assertTrue(success)
882
+
self.assertIsNone(error)
883
+
self.assertAlmostEqual(lat, 52.5200066, places=6)
884
+
self.assertAlmostEqual(lon, 13.404954, places=6)
885
+
886
+
# Verify NO API call was made
887
+
mock_get.assert_not_called()
888
+
889
+
def test_failed_geocoding_cached(self):
890
+
"""Test that failed geocoding attempts are cached."""
891
+
from unittest.mock import patch, Mock
892
+
from .models import GeocodeCache
893
+
from .services import AddressGeocoder
894
+
895
+
mock_response = Mock()
896
+
mock_response.status_code = 200
897
+
mock_response.json.return_value = [] # Empty result = not found
898
+
899
+
with patch('requests.get', return_value=mock_response):
900
+
geocoder = AddressGeocoder()
901
+
lat, lon, success, error = geocoder.geocode(
902
+
street='Nonexistent Street 999',
903
+
postal_code='99999',
904
+
city='Nowhere'
905
+
)
906
+
907
+
self.assertFalse(success)
908
+
self.assertIsNotNone(error)
909
+
self.assertIsNone(lat)
910
+
self.assertIsNone(lon)
911
+
912
+
# Verify the failure was cached
913
+
import hashlib
914
+
address_string = 'Nonexistent Street 999|99999|Nowhere|DE'
915
+
address_hash = hashlib.sha256(address_string.encode('utf-8')).hexdigest()
916
+
917
+
cache_entry = GeocodeCache.objects.get(address_hash=address_hash)
918
+
self.assertFalse(cache_entry.success)
919
+
self.assertEqual(cache_entry.error_message, error)
920
+
921
+
def test_api_error_handling(self):
922
+
"""Test that API errors are handled gracefully."""
923
+
from unittest.mock import patch
924
+
from .services import AddressGeocoder
925
+
import requests
926
+
927
+
with patch('requests.get', side_effect=requests.RequestException('API Error')):
928
+
geocoder = AddressGeocoder()
929
+
lat, lon, success, error = geocoder.geocode(
930
+
street='Test Street',
931
+
postal_code='12345',
932
+
city='Test City'
933
+
)
934
+
935
+
self.assertFalse(success)
936
+
self.assertIsNotNone(error)
937
+
self.assertIsNone(lat)
938
+
self.assertIsNone(lon)
939
+
self.assertIn('API Error', error)
940
+
941
+
942
class RepresentativeMetadataExtractionTests(TestCase):
943
944
def setUp(self):
···
1003
service._ensure_photo_reference(rep)
1004
rep.refresh_from_db()
1005
self.assertEqual(rep.photo_path, 'representatives/999.jpg')
1006
+
1007
+
1008
+
class GeoJSONDataTests(TestCase):
1009
+
"""Test that official Bundestag constituency GeoJSON data is available and valid."""
1010
+
1011
+
def test_geojson_file_exists(self):
1012
+
"""Verify the wahlkreise.geojson file exists in the data directory."""
1013
+
from pathlib import Path
1014
+
from django.conf import settings
1015
+
1016
+
geojson_path = Path(settings.BASE_DIR) / 'letters' / 'data' / 'wahlkreise.geojson'
1017
+
self.assertTrue(geojson_path.exists(), f"GeoJSON file not found at {geojson_path}")
1018
+
1019
+
def test_geojson_is_valid_and_loadable(self):
1020
+
"""Verify the GeoJSON file is valid JSON and has expected structure."""
1021
+
import json
1022
+
from pathlib import Path
1023
+
from django.conf import settings
1024
+
1025
+
geojson_path = Path(settings.BASE_DIR) / 'letters' / 'data' / 'wahlkreise.geojson'
1026
+
1027
+
with open(geojson_path, 'r', encoding='utf-8') as f:
1028
+
data = json.load(f)
1029
+
1030
+
self.assertEqual(data['type'], 'FeatureCollection')
1031
+
self.assertIn('features', data)
1032
+
self.assertIsInstance(data['features'], list)
1033
+
1034
+
def test_geojson_contains_all_constituencies(self):
1035
+
"""Verify the GeoJSON contains all 299 Bundestag constituencies."""
1036
+
import json
1037
+
from pathlib import Path
1038
+
from django.conf import settings
1039
+
1040
+
geojson_path = Path(settings.BASE_DIR) / 'letters' / 'data' / 'wahlkreise.geojson'
1041
+
1042
+
with open(geojson_path, 'r', encoding='utf-8') as f:
1043
+
data = json.load(f)
1044
+
1045
+
self.assertEqual(len(data['features']), 299)
1046
+
1047
+
def test_geojson_features_have_required_properties(self):
1048
+
"""Verify each feature has required properties: WKR_NR, WKR_NAME, LAND_NR, LAND_NAME."""
1049
+
import json
1050
+
from pathlib import Path
1051
+
from django.conf import settings
1052
+
1053
+
geojson_path = Path(settings.BASE_DIR) / 'letters' / 'data' / 'wahlkreise.geojson'
1054
+
1055
+
with open(geojson_path, 'r', encoding='utf-8') as f:
1056
+
data = json.load(f)
1057
+
1058
+
# Check first feature
1059
+
if len(data['features']) > 0:
1060
+
feature = data['features'][0]
1061
+
self.assertEqual(feature['type'], 'Feature')
1062
+
self.assertIn('properties', feature)
1063
+
self.assertIn('geometry', feature)
1064
+
1065
+
properties = feature['properties']
1066
+
self.assertIn('WKR_NR', properties)
1067
+
self.assertIn('WKR_NAME', properties)
1068
+
self.assertIn('LAND_NR', properties)
1069
+
self.assertIn('LAND_NAME', properties)
1070
+
1071
+
# Verify geometry type
1072
+
self.assertEqual(feature['geometry']['type'], 'Polygon')
1073
+
1074
+
def test_geojson_constituency_numbers_complete(self):
1075
+
"""Verify constituency numbers range from 1 to 299 with no gaps."""
1076
+
import json
1077
+
from pathlib import Path
1078
+
from django.conf import settings
1079
+
1080
+
geojson_path = Path(settings.BASE_DIR) / 'letters' / 'data' / 'wahlkreise.geojson'
1081
+
1082
+
with open(geojson_path, 'r', encoding='utf-8') as f:
1083
+
data = json.load(f)
1084
+
1085
+
wkr_numbers = [f['properties']['WKR_NR'] for f in data['features']]
1086
+
expected_numbers = set(range(1, 300))
1087
+
actual_numbers = set(wkr_numbers)
1088
+
1089
+
self.assertEqual(actual_numbers, expected_numbers, "Constituency numbers should be 1-299 with no gaps")
1090
+
1091
+
1092
+
class WahlkreisLocatorTests(TestCase):
1093
+
"""Test the WahlkreisLocator service for point-in-polygon constituency matching."""
1094
+
1095
+
def test_locate_bundestag_building(self):
1096
+
"""Test that Bundestag coordinates (52.5186, 13.3761) find a Berlin constituency."""
1097
+
from .services import WahlkreisLocator
1098
+
1099
+
locator = WahlkreisLocator()
1100
+
result = locator.locate(52.5186, 13.3761)
1101
+
1102
+
self.assertIsNotNone(result, "Bundestag coordinates should find a constituency")
1103
+
wkr_nr, wkr_name, land_name = result
1104
+
self.assertIsInstance(wkr_nr, int)
1105
+
self.assertGreater(wkr_nr, 0)
1106
+
self.assertLessEqual(wkr_nr, 299)
1107
+
self.assertIsInstance(wkr_name, str)
1108
+
self.assertIsInstance(land_name, str)
1109
+
self.assertIn('Berlin', land_name, "Bundestag should be in a Berlin constituency")
1110
+
1111
+
def test_locate_hamburg_rathaus(self):
1112
+
"""Test that Hamburg Rathaus (53.5511, 9.9937) finds a Hamburg constituency."""
1113
+
from .services import WahlkreisLocator
1114
+
1115
+
locator = WahlkreisLocator()
1116
+
result = locator.locate(53.5511, 9.9937)
1117
+
1118
+
self.assertIsNotNone(result, "Hamburg Rathaus coordinates should find a constituency")
1119
+
wkr_nr, wkr_name, land_name = result
1120
+
self.assertIsInstance(wkr_nr, int)
1121
+
self.assertGreater(wkr_nr, 0)
1122
+
self.assertLessEqual(wkr_nr, 299)
1123
+
self.assertIn('Hamburg', land_name, "Hamburg Rathaus should be in a Hamburg constituency")
1124
+
1125
+
def test_locate_multiple_known_locations(self):
1126
+
"""Test multiple German cities to ensure accurate constituency matching."""
1127
+
from .services import WahlkreisLocator
1128
+
1129
+
locator = WahlkreisLocator()
1130
+
1131
+
# Munich: Marienplatz (48.1374, 11.5755)
1132
+
munich_result = locator.locate(48.1374, 11.5755)
1133
+
self.assertIsNotNone(munich_result, "Munich coordinates should find a constituency")
1134
+
self.assertIn('Bayern', munich_result[2], "Munich should be in Bavaria")
1135
+
1136
+
# Cologne: Dom (50.9413, 6.9583)
1137
+
cologne_result = locator.locate(50.9413, 6.9583)
1138
+
self.assertIsNotNone(cologne_result, "Cologne coordinates should find a constituency")
1139
+
self.assertIn('Nordrhein-Westfalen', cologne_result[2], "Cologne should be in NRW")
1140
+
1141
+
# Frankfurt: Römer (50.1106, 8.6821)
1142
+
frankfurt_result = locator.locate(50.1106, 8.6821)
1143
+
self.assertIsNotNone(frankfurt_result, "Frankfurt coordinates should find a constituency")
1144
+
self.assertIn('Hessen', frankfurt_result[2], "Frankfurt should be in Hessen")
1145
+
1146
+
# Dresden: Frauenkirche (51.0515, 13.7416)
1147
+
dresden_result = locator.locate(51.0515, 13.7416)
1148
+
self.assertIsNotNone(dresden_result, "Dresden coordinates should find a constituency")
1149
+
self.assertIn('Sachsen', dresden_result[2], "Dresden should be in Saxony")
1150
+
1151
+
# Stuttgart: Schlossplatz (48.7775, 9.1797)
1152
+
stuttgart_result = locator.locate(48.7775, 9.1797)
1153
+
self.assertIsNotNone(stuttgart_result, "Stuttgart coordinates should find a constituency")
1154
+
self.assertIn('Baden-Württemberg', stuttgart_result[2], "Stuttgart should be in Baden-Württemberg")
1155
+
1156
+
def test_coordinates_outside_germany(self):
1157
+
"""Test that coordinates outside Germany return None."""
1158
+
from .services import WahlkreisLocator
1159
+
1160
+
locator = WahlkreisLocator()
1161
+
1162
+
# Paris, France (48.8566, 2.3522)
1163
+
paris_result = locator.locate(48.8566, 2.3522)
1164
+
self.assertIsNone(paris_result, "Paris coordinates should not find a German constituency")
1165
+
1166
+
# London, UK (51.5074, -0.1278)
1167
+
london_result = locator.locate(51.5074, -0.1278)
1168
+
self.assertIsNone(london_result, "London coordinates should not find a German constituency")
1169
+
1170
+
# New York, USA (40.7128, -74.0060)
1171
+
nyc_result = locator.locate(40.7128, -74.0060)
1172
+
self.assertIsNone(nyc_result, "NYC coordinates should not find a German constituency")
1173
+
1174
+
def test_geojson_loads_successfully(self):
1175
+
"""Test that the service can load the 44MB GeoJSON file without errors."""
1176
+
from .services import WahlkreisLocator
1177
+
import time
1178
+
1179
+
start_time = time.time()
1180
+
locator = WahlkreisLocator()
1181
+
load_time = time.time() - start_time
1182
+
1183
+
# Verify the service loaded constituencies
1184
+
self.assertIsNotNone(locator.constituencies, "Constituencies should be loaded")
1185
+
self.assertGreater(len(locator.constituencies), 0, "Should have loaded constituencies")
1186
+
self.assertEqual(len(locator.constituencies), 299, "Should have loaded all 299 constituencies")
1187
+
1188
+
# Verify loading is reasonably fast (< 2 seconds)
1189
+
self.assertLess(load_time, 2.0, f"GeoJSON loading took {load_time:.2f}s, should be under 2 seconds")
1190
+
1191
+
1192
+
class ConstituencyLocatorIntegrationTests(ParliamentFixtureMixin, TestCase):
1193
+
"""Test the integrated ConstituencyLocator with address-based and PLZ fallback."""
1194
+
1195
+
def test_locate_by_full_address_returns_constituencies(self):
1196
+
"""Test address-based constituency lookup returns Representatives."""
1197
+
from unittest.mock import patch, Mock
1198
+
from .services import ConstituencyLocator
1199
+
1200
+
# Mock the geocoding response for Platz der Republik 1, Berlin
1201
+
# This is the Bundestag building location
1202
+
mock_geocode_response = Mock()
1203
+
mock_geocode_response.status_code = 200
1204
+
mock_geocode_response.json.return_value = [
1205
+
{
1206
+
'lat': '52.5186',
1207
+
'lon': '13.3761',
1208
+
'display_name': 'Platz der Republik, Berlin, Germany'
1209
+
}
1210
+
]
1211
+
1212
+
with patch('requests.get', return_value=mock_geocode_response):
1213
+
locator = ConstituencyLocator()
1214
+
result = locator.locate(
1215
+
street='Platz der Republik 1',
1216
+
postal_code='11011',
1217
+
city='Berlin'
1218
+
)
1219
+
1220
+
# Result should be a list of Representative objects
1221
+
self.assertIsInstance(result, list)
1222
+
# Should find at least one representative
1223
+
self.assertGreater(len(result), 0)
1224
+
1225
+
# All results should be Representative instances
1226
+
from .models import Representative
1227
+
for rep in result:
1228
+
self.assertIsInstance(rep, Representative)
1229
+
# Representatives should be from Berlin
1230
+
rep_states = {
1231
+
(c.metadata or {}).get('state')
1232
+
for c in rep.constituencies.all()
1233
+
if c.metadata
1234
+
}
1235
+
self.assertIn('Berlin', rep_states, f"Representative {rep} should be from Berlin")
1236
+
1237
+
def test_locate_by_full_address_with_cache(self):
1238
+
"""Test that cached geocoding results are used for constituency lookup."""
1239
+
from .models import GeocodeCache
1240
+
from .services import ConstituencyLocator
1241
+
import hashlib
1242
+
1243
+
# Pre-populate cache with Berlin coordinates
1244
+
address_string = 'Unter den Linden 77|10117|Berlin|DE'
1245
+
address_hash = hashlib.sha256(address_string.encode('utf-8')).hexdigest()
1246
+
1247
+
GeocodeCache.objects.create(
1248
+
address_hash=address_hash,
1249
+
street='Unter den Linden 77',
1250
+
postal_code='10117',
1251
+
city='Berlin',
1252
+
country='DE',
1253
+
latitude=52.5186,
1254
+
longitude=13.3761,
1255
+
success=True
1256
+
)
1257
+
1258
+
# No mocking needed - should use cache
1259
+
locator = ConstituencyLocator()
1260
+
result = locator.locate(
1261
+
street='Unter den Linden 77',
1262
+
postal_code='10117',
1263
+
city='Berlin'
1264
+
)
1265
+
1266
+
# Should find representatives
1267
+
self.assertIsInstance(result, list)
1268
+
self.assertGreater(len(result), 0)
1269
+
1270
+
def test_locate_by_plz_fallback_when_geocoding_fails(self):
1271
+
"""Test PLZ-based fallback when address geocoding fails."""
1272
+
from unittest.mock import patch, Mock
1273
+
from .services import ConstituencyLocator
1274
+
1275
+
# Mock geocoding to return failure
1276
+
mock_response = Mock()
1277
+
mock_response.status_code = 200
1278
+
mock_response.json.return_value = [] # Empty = not found
1279
+
1280
+
with patch('requests.get', return_value=mock_response):
1281
+
locator = ConstituencyLocator()
1282
+
result = locator.locate(
1283
+
street='Nonexistent Street 999',
1284
+
postal_code='10115', # Berlin PLZ
1285
+
city='Berlin'
1286
+
)
1287
+
1288
+
# Should still find representatives using PLZ fallback
1289
+
self.assertIsInstance(result, list)
1290
+
# PLZ fallback might return empty list or representatives depending on data
1291
+
# The important thing is it doesn't crash
1292
+
1293
+
def test_locate_by_plz_only_maintains_backward_compatibility(self):
1294
+
"""Test that PLZ-only lookup still works (backward compatibility)."""
1295
+
from .services import ConstituencyLocator
1296
+
1297
+
locator = ConstituencyLocator()
1298
+
result = locator.locate(postal_code='10115') # Berlin PLZ
1299
+
1300
+
# Should work without crashing
1301
+
self.assertIsInstance(result, list)
1302
+
# Result depends on existing data, but should at least not error
1303
+
1304
+
def test_locate_without_parameters_returns_empty(self):
1305
+
"""Test that calling locate without parameters returns empty list."""
1306
+
from .services import ConstituencyLocator
1307
+
1308
+
locator = ConstituencyLocator()
1309
+
result = locator.locate()
1310
+
1311
+
self.assertIsInstance(result, list)
1312
+
self.assertEqual(len(result), 0)
1313
+
1314
+
1315
+
class IdentityVerificationFormTests(TestCase):
1316
+
"""Test the IdentityVerificationForm for full address collection."""
1317
+
1318
+
def setUp(self):
1319
+
self.user = User.objects.create_user(
1320
+
username='testuser',
1321
+
password='password123',
1322
+
email='testuser@example.com',
1323
+
)
1324
+
1325
+
def test_form_requires_all_address_fields_together(self):
1326
+
"""Test that form validation requires all address fields if any is provided."""
1327
+
from .forms import IdentityVerificationForm
1328
+
1329
+
# Only street provided - should fail
1330
+
form = IdentityVerificationForm(
1331
+
data={
1332
+
'street_address': 'Unter den Linden 77',
1333
+
'postal_code': '',
1334
+
'city': '',
1335
+
},
1336
+
user=self.user
1337
+
)
1338
+
self.assertFalse(form.is_valid())
1339
+
self.assertIn('Bitte geben Sie eine vollständige Adresse ein', str(form.errors))
1340
+
1341
+
# Only PLZ provided - should fail
1342
+
form = IdentityVerificationForm(
1343
+
data={
1344
+
'street_address': '',
1345
+
'postal_code': '10117',
1346
+
'city': '',
1347
+
},
1348
+
user=self.user
1349
+
)
1350
+
self.assertFalse(form.is_valid())
1351
+
self.assertIn('Bitte geben Sie eine vollständige Adresse ein', str(form.errors))
1352
+
1353
+
# Only city provided - should fail
1354
+
form = IdentityVerificationForm(
1355
+
data={
1356
+
'street_address': '',
1357
+
'postal_code': '',
1358
+
'city': 'Berlin',
1359
+
},
1360
+
user=self.user
1361
+
)
1362
+
self.assertFalse(form.is_valid())
1363
+
self.assertIn('Bitte geben Sie eine vollständige Adresse ein', str(form.errors))
1364
+
1365
+
def test_form_accepts_all_address_fields(self):
1366
+
"""Test that form is valid when all address fields are provided."""
1367
+
from .forms import IdentityVerificationForm
1368
+
1369
+
form = IdentityVerificationForm(
1370
+
data={
1371
+
'street_address': 'Unter den Linden 77',
1372
+
'postal_code': '10117',
1373
+
'city': 'Berlin',
1374
+
},
1375
+
user=self.user
1376
+
)
1377
+
self.assertTrue(form.is_valid())
1378
+
1379
+
def test_form_accepts_empty_address(self):
1380
+
"""Test that form is valid when all address fields are empty."""
1381
+
from .forms import IdentityVerificationForm
1382
+
1383
+
form = IdentityVerificationForm(
1384
+
data={
1385
+
'street_address': '',
1386
+
'postal_code': '',
1387
+
'city': '',
1388
+
},
1389
+
user=self.user
1390
+
)
1391
+
self.assertTrue(form.is_valid())
1392
+
1393
+
def test_form_prefills_existing_address(self):
1394
+
"""Test that form prefills existing address from verification."""
1395
+
from .forms import IdentityVerificationForm
1396
+
1397
+
# Create verification with address
1398
+
verification = IdentityVerification.objects.create(
1399
+
user=self.user,
1400
+
status='SELF_DECLARED',
1401
+
street_address='Unter den Linden 77',
1402
+
postal_code='10117',
1403
+
city='Berlin',
1404
+
)
1405
+
1406
+
form = IdentityVerificationForm(user=self.user)
1407
+
1408
+
self.assertEqual(form.fields['street_address'].initial, 'Unter den Linden 77')
1409
+
self.assertEqual(form.fields['postal_code'].initial, '10117')
1410
+
self.assertEqual(form.fields['city'].initial, 'Berlin')
1411
+
1412
+
1413
+
class ProfileViewAddressTests(TestCase):
1414
+
"""Test profile view address form submission."""
1415
+
1416
+
def setUp(self):
1417
+
self.user = User.objects.create_user(
1418
+
username='testuser',
1419
+
password='password123',
1420
+
email='testuser@example.com',
1421
+
)
1422
+
self.client.login(username='testuser', password='password123')
1423
+
1424
+
def test_profile_view_saves_address(self):
1425
+
"""Test that profile view saves address correctly."""
1426
+
response = self.client.post(
1427
+
reverse('profile'),
1428
+
{
1429
+
'address_form_submit': '1',
1430
+
'street_address': 'Unter den Linden 77',
1431
+
'postal_code': '10117',
1432
+
'city': 'Berlin',
1433
+
},
1434
+
follow=True
1435
+
)
1436
+
1437
+
self.assertEqual(response.status_code, 200)
1438
+
self.assertRedirects(response, reverse('profile'))
1439
+
1440
+
# Verify address was saved
1441
+
verification = IdentityVerification.objects.get(user=self.user)
1442
+
self.assertEqual(verification.street_address, 'Unter den Linden 77')
1443
+
self.assertEqual(verification.postal_code, '10117')
1444
+
self.assertEqual(verification.city, 'Berlin')
1445
+
self.assertEqual(verification.status, 'SELF_DECLARED')
1446
+
self.assertEqual(verification.verification_type, 'SELF_DECLARED')
1447
+
1448
+
def test_profile_view_updates_existing_address(self):
1449
+
"""Test that profile view updates existing address."""
1450
+
# Create initial verification
1451
+
verification = IdentityVerification.objects.create(
1452
+
user=self.user,
1453
+
status='SELF_DECLARED',
1454
+
street_address='Old Street 1',
1455
+
postal_code='12345',
1456
+
city='OldCity',
1457
+
)
1458
+
1459
+
response = self.client.post(
1460
+
reverse('profile'),
1461
+
{
1462
+
'address_form_submit': '1',
1463
+
'street_address': 'Unter den Linden 77',
1464
+
'postal_code': '10117',
1465
+
'city': 'Berlin',
1466
+
},
1467
+
follow=True
1468
+
)
1469
+
1470
+
self.assertEqual(response.status_code, 200)
1471
+
1472
+
# Verify address was updated
1473
+
verification.refresh_from_db()
1474
+
self.assertEqual(verification.street_address, 'Unter den Linden 77')
1475
+
self.assertEqual(verification.postal_code, '10117')
1476
+
self.assertEqual(verification.city, 'Berlin')
1477
+
1478
+
def test_profile_view_displays_saved_address(self):
1479
+
"""Test that profile view displays saved address."""
1480
+
# Create verification with address
1481
+
verification = IdentityVerification.objects.create(
1482
+
user=self.user,
1483
+
status='SELF_DECLARED',
1484
+
street_address='Unter den Linden 77',
1485
+
postal_code='10117',
1486
+
city='Berlin',
1487
+
)
1488
+
1489
+
response = self.client.get(reverse('profile'))
1490
+
1491
+
self.assertEqual(response.status_code, 200)
1492
+
self.assertContains(response, 'Unter den Linden 77')
1493
+
self.assertContains(response, '10117')
1494
+
self.assertContains(response, 'Berlin')
+52
-13
website/letters/views.py
+52
-13
website/letters/views.py
···
29
ReportForm,
30
LetterSearchForm,
31
UserRegisterForm,
32
-
SelfDeclaredConstituencyForm
33
)
34
from .services import IdentityVerificationService, ConstituencySuggestionService
35
···
314
verification = None
315
316
if request.method == 'POST':
317
-
constituency_form = SelfDeclaredConstituencyForm(request.POST, user=user)
318
-
if constituency_form.is_valid():
319
-
IdentityVerificationService.self_declare(
320
-
user=user,
321
-
federal_constituency=constituency_form.cleaned_data['federal_constituency'],
322
-
state_constituency=constituency_form.cleaned_data['state_constituency'],
323
-
)
324
-
messages.success(
325
-
request,
326
-
_('Your constituency information has been updated.')
327
-
)
328
-
return redirect('profile')
329
else:
330
constituency_form = SelfDeclaredConstituencyForm(user=user)
331
332
context = {
333
'user_letters': user_letters,
334
'user_signatures': user_signatures,
335
'verification': verification,
336
'constituency_form': constituency_form,
337
}
338
339
return render(request, 'letters/profile.html', context)
···
29
ReportForm,
30
LetterSearchForm,
31
UserRegisterForm,
32
+
SelfDeclaredConstituencyForm,
33
+
IdentityVerificationForm
34
)
35
from .services import IdentityVerificationService, ConstituencySuggestionService
36
···
315
verification = None
316
317
if request.method == 'POST':
318
+
# Check which form was submitted
319
+
if 'address_form_submit' in request.POST:
320
+
address_form = IdentityVerificationForm(request.POST, user=user)
321
+
constituency_form = SelfDeclaredConstituencyForm(user=user)
322
+
323
+
if address_form.is_valid():
324
+
street_address = address_form.cleaned_data.get('street_address')
325
+
postal_code = address_form.cleaned_data.get('postal_code')
326
+
city = address_form.cleaned_data.get('city')
327
+
328
+
# Only update if all fields are provided
329
+
if street_address and postal_code and city:
330
+
# Get or create verification record
331
+
verification, created = IdentityVerification.objects.get_or_create(
332
+
user=user,
333
+
defaults={
334
+
'status': 'SELF_DECLARED',
335
+
'verification_type': 'SELF_DECLARED',
336
+
}
337
+
)
338
+
339
+
# Update address fields
340
+
verification.street_address = street_address
341
+
verification.postal_code = postal_code
342
+
verification.city = city
343
+
verification.save()
344
+
345
+
messages.success(
346
+
request,
347
+
_('Ihre Adresse wurde gespeichert.')
348
+
)
349
+
return redirect('profile')
350
+
else:
351
+
# Constituency form submission
352
+
constituency_form = SelfDeclaredConstituencyForm(request.POST, user=user)
353
+
address_form = IdentityVerificationForm(user=user)
354
+
355
+
if constituency_form.is_valid():
356
+
IdentityVerificationService.self_declare(
357
+
user=user,
358
+
federal_constituency=constituency_form.cleaned_data['federal_constituency'],
359
+
state_constituency=constituency_form.cleaned_data['state_constituency'],
360
+
)
361
+
messages.success(
362
+
request,
363
+
_('Your constituency information has been updated.')
364
+
)
365
+
return redirect('profile')
366
else:
367
constituency_form = SelfDeclaredConstituencyForm(user=user)
368
+
address_form = IdentityVerificationForm(user=user)
369
370
context = {
371
'user_letters': user_letters,
372
'user_signatures': user_signatures,
373
'verification': verification,
374
'constituency_form': constituency_form,
375
+
'address_form': address_form,
376
}
377
378
return render(request, 'letters/profile.html', context)
+1
-1
website/writethem/settings.py
+1
-1
website/writethem/settings.py