# Testing Guide

Comprehensive testing documentation for OWILIX developers and AI agents.

> **For AI Agents**: This is your testing reference. Use the Quick Verification section for fast checks, and CLI Smoke Tests for subprocess-based testing.

## Quick Verification (Recommended for AI)

Before committing changes, run these quick checks:

```bash
# turbo
# 1. Check imports work
uv run python -c "from owilix.core.manager import OWIlixManager; print('✅ Imports OK')"

# turbo
# 2. Run fast CLI smoke tests (~8s)
uv run pytest tests/owilix/cli/test_smoke.py -v -m "not integration"

# turbo
# 3. Run AI verification suite (structured output)
uv run python tests/owilix/cli/test_ai_verifiable.py
```

## Latest Local Runs

Latest observed local runs in this repository:

- `uv run pytest -q`
  Result on 2026-04-11: `471 passed, 2 skipped, 42 deselected, 2 warnings`
- `uv run pytest tests/owilix/cli/test_remote_search_cli.py tests/owilix/core/tasks/test_search.py -q`
  Result on 2026-04-10: `24 passed`
- `uv run pytest tests/owilix/cli/test_smoke.py -q -m integration -k remote_search`
  Result on 2026-04-10: passed in the current environment
- `uv run pytest tests/owilix/core/test_sql_logging.py tests/owilix/core/db/test_executors.py -q`
  Result on 2026-04-11: `22 passed, 2 deselected`
- `uv run pytest tests/owilix/cli/test_smoke.py -q -k 'admin_logs_location or admin_logs_sql' -o addopts=''`
  Result on 2026-04-11: `2 passed`

The remaining warning noise is the documented Rich/Jupyter cosmetic warning from `tests/owilix/core/tasks/test_search.py`.

## SQL Debug Logging

To capture executed SQL statements during CLI runs, enable the global SQL log:

```bash
uv run owi --sql-log query less -R lexis:latest --limit 10
```

The statements are written to:

```bash
~/.owi/.logs/sql.log
```

This log is opt-in and intended for debugging query construction and execution.

To inspect the logs from the CLI:

```bash
uv run owi admin logs location
uv run owi admin logs sql --lines 20
uv run owi admin logs events --lines 20
```

## Test Organization

```
tests/
├── owilix/
│   ├── cli/                    # CLI smoke tests (fast, subprocess-based)
│   │   ├── test_smoke.py
│   │   └── test_ai_verifiable.py  # AI-interpretable JSON output
│   └── core/
│       ├── db/                 # DuckDB executor & pool tests
│       │   ├── test_executors.py      # SQL query, pool, retry logic
│       │   ├── test_duckdb.py
│       │   └── test_copy_executor.py
│       ├── repository/         # Repository tests
│       │   ├── test_aggregate.py      # Multi-datacenter aggregation
│       │   ├── test_integration.py    # CLI integration (slow, network)
│       │   └── test_repository_legacy_ported.py
│       ├── tasks/              # Task function tests
│       │   ├── test_search.py         # Full-text search (SQL builder, remote)
│       │   └── test_query_unit.py     # query_less / query_sites / query_analyze
│       ├── fsspec/             # fsspec tests
│       │   ├── test_core_fsspec_unit.py
│       │   └── benchmark_async.py
│       └── test_metadata.py
└── data/                       # Test fixtures
```

By default, integration tests are deselected via `addopts` in `pyproject.toml`. The full unit suite runs in ~25 seconds.

## Test Types & Timing

| Test Suite | Command | Duration | Network |
|------------|---------|----------|---------|
| Full unit suite | `pytest tests/` | ~25s | No |
| CLI Smoke | `pytest tests/owilix/cli/test_smoke.py` | ~8s | No |
| DB executor unit | `pytest tests/owilix/core/db/` | <1s | No |
| Repository Integration | `pytest tests/owilix/core/repository/test_integration.py -m integration` | 5-10min | Yes |

## Expected Release Test Matrix

For a normal client release, these are the tests that should be expected to run:

```bash
# 1. Full default gate
uv run pytest -q

# 2. Search/UI focused gate
uv run pytest tests/owilix/cli/test_remote_search_cli.py tests/owilix/core/tasks/test_search.py -q

# 3. Fast CLI smoke gate
uv run pytest tests/owilix/cli/test_smoke.py -q -m "not integration"

# 4. Selected network smoke checks when credentials/network are available
uv run pytest tests/owilix/cli/test_smoke.py -q -m integration -k "remote_doctor or remote_ls_latest or remote_search"

# 5. Broader integration pass before publishing when time permits
uv run pytest tests/ -m integration -v -s
```

Notes:

- `query less --search` integration is valid but heavier than `remote search` alone because it resolves collections and scans parquet after the DuckLake lookup.
- The docs build is also part of release hygiene when the docs dependency group is installed:
  `uv sync --group docs && uv run sphinx-build -b html docs/source docs/build/html`

## Pytest Markers

Registered in `pyproject.toml`:

```bash
# Skip network-dependent tests
uv run pytest tests/ -m "not integration"

# Run only integration tests
uv run pytest tests/ -m integration -v -s

# Skip slow tests
uv run pytest tests/ -m "not slow"
```

## CLI Smoke Tests

Fast, subprocess-based tests that run actual `owi` commands:

```python
# tests/owilix/cli/test_smoke.py

def run_owi(*args, timeout=30):
    """Run owi command via subprocess."""
    result = subprocess.run(
        ["uv", "run", "owi"] + list(args),
        capture_output=True,
        text=True,
        timeout=timeout
    )
    return result.returncode, result.stdout, result.stderr

# Example test
def test_config_version(self):
    code, out, err = run_owi("config", "version")
    assert code == 0
    assert "Config version:" in out
```

## AI-Verifiable Tests

Tests with structured JSON output for AI parsing:

```bash
# Run verification suite
uv run python tests/owilix/cli/test_ai_verifiable.py

# Example output:
# ✅ owi --help (1.33s)
# ✅ owi config version (1.29s)
# --- SUMMARY ---
# {"total": 3, "passed": 3, "failed": 0, "all_passed": true}
```

The output includes `--- AI RESULT ---` blocks with structured JSON:

```json
{
  "command": "owi config version",
  "exit_code": 0,
  "success": true,
  "elapsed_seconds": 1.29,
  "stdout_preview": "Config version: 3.0...",
  "contains": {
    "error": false,
    "traceback": false,
    "success_icon": true
  }
}
```

## Integration Tests (Network Required)

These tests invoke full CLI commands with network access:

```bash
# Run with output visible (-s flag important for progress)
uv run pytest tests/owilix/core/repository/test_integration.py -v -s

# Expected duration: 5-10 minutes
# Tests include: remote ls, remote pull, local ls
```

Progress indicators show timing during slow operations:

```
[0.0s] Starting remote ls test...
[5.2s] CLI completed with exit code 0
[5.2s] ✅ Remote ls test passed
```

## Known Issues

### Import Errors

- `test_graph_commands.py` requires `owilix.core.ui` (not yet implemented)
- Skip with: `--ignore=tests/owilix/core/test_graph_commands.py`

### Mock Failures in fsspec

2 tests fail due to mock subscripting issues:
- `test_ls_returns_paths`
- `test_ls_with_detail`

These are test issues, not code issues.

## Adding New Tests

### Unit Tests (Fast)

```python
def test_my_feature():
    """Unit test - fast, no network."""
    result = my_function()
    assert result == expected
```

### Integration Tests (Network)

```python
import pytest

@pytest.mark.integration
def test_remote_operation(capsys):
    """Integration test - requires network."""
    print("[0.0s] Starting test...")  # Progress with -s flag
    # ... test code ...
    print("[5.0s] ✅ Test passed")
```

## Recommended Workflow

1. **Before each commit**: Run smoke tests
   ```bash
   uv run pytest tests/owilix/cli/test_smoke.py -v -m "not integration"
   ```

2. **Before merging**: Run full test suite
   ```bash
   uv run pytest tests/ -v --ignore=tests/owilix/core/test_graph_commands.py
   ```

   For release work, also run the selected network smoke commands listed in the release matrix above.

3. **For AI agents**: Use the verification script
   ```bash
   uv run python tests/owilix/cli/test_ai_verifiable.py
   ```

## Test Documentation

Each test package has comprehensive docstrings in `__init__.py` files:
- `tests/owilix/core/__init__.py` - Overview of core tests
- `tests/owilix/core/db/__init__.py` - DuckDB executor tests and benchmarks
- `tests/owilix/core/fsspec/__init__.py` - Filesystem tests
- `tests/owilix/core/repository/__init__.py` - Repository tests

For detailed benchmarks and results, see `docs/source/testing_and_benchmarks.md`.
For phased integration execution and optimization guidance, see `docs/integration_test_plan.md`.
For current warning triage and tolerated cosmetic noise, see `docs/test-status.md`.

## Database Executor Tests

```bash
# turbo
# Run executor unit tests
uv run pytest tests/owilix/core/db/test_executors.py -v

# turbo  
# Run comprehensive benchmarks (sync vs async)
uv run python tests/owilix/core/db/benchmark_comprehensive.py
```