owi-cli
Command line interface for accessing the Open Web Index shards and slices.
Star 0
Fork 0
Watch
owi-cli docs source core.md
4.4 kB · 185 lines
Markdown
at main

OWILIX Core Manager Package #

The owilix.core.manager package provides centralized configuration, session management, logging, and UI utilities.

Package Structure #

owilix/core/manager/
├── __init__.py     # Package exports
├── env.py          # OWILIXEnv singleton
├── config.py       # Pydantic configuration models
├── session.py      # OWILIXSession wrapper
├── manager.py      # OWILIXManager class
├── ui.py           # Console and progress utilities
└── logging.py      # DBLog transaction logging

Architecture #

The core architecture revolves around the OWIlixManager, which acts as the central hub for:

  1. Configuration: Managing OWIlixConfig and environment variables.
  2. Session: Holding the OWILIXSession (wrapping py4lexis session) for authentication.
  3. Repositories: Creating and aggregating AbstractRepository instances (Local, Lexis, etc.).

Component Interaction #

graph TD
    CLI[OWILIX CLI] --> Manager[OWIlixManager]
    Manager --> Config[OWIlixConfig]
    Manager --> Session[OWILIXSession]
    Manager --> Remote[AggregatedRepository]
    Manager --> Local[LocalRepository]
    
    Session --> Lexis[LexisSession (py4lexis)]
    Remote --> LexisRepo[LexisRepository]
    LexisRepo --> FS[Http2IrodsFileSystem]
    FS --> Lexis

Reference Documentation #

Quick Start #

from owilix.core.manager import OWILIXManager, OWILIXEnv

# Environment configuration (singleton)
env = OWILIXEnv.values
print(f"Project: {env.project_name}")
print(f"Path: {env.owi_path}")

# Manager (orchestrates config, session, repos)
manager = OWILIXManager()
session = manager.session  # OWILIXSession wrapper

Components #

OWILIXEnv #

Singleton for environment variables.

Variable Default Description
OWILIX_LEXIS_PROJECT_NAME openwebsearch Project name
OWILIX_LEXIS_PROJECT_ID Auto-generated Project ID
OWS_OWI_PATH ~/.owi Base path
OWILIX_IRODS_CONNECTION_TIMEOUT 120 Timeout (seconds) 
.. autoclass:: owilix.core.manager.OWILIXEnv
   :members:

OWILIXSettings (Pydantic) #

Type-safe configuration with validation.

from owilix.core.manager import OWILIXSettings, RepositoryConfig

settings = OWILIXSettings()
repo = RepositoryConfig(repository="lexis+http", options={"zone": "IT4ILexisV2"})

.. autoclass:: owilix.core.manager.OWILIXSettings
   :members:

OWILIXSession #

Unified wrapper for LexisSession and iRODS access.

session = manager.session

# Access LEXIS API
token = session.lexis.get_access_token()

# Access iRODS (lazy-loaded)
irods = session.irods

# Get HTTP client with fresh token
client = session.get_irods_client()

.. autoclass:: owilix.core.manager.OWILIXSession
   :members:

OWILIXManager #

Central orchestrator for OWILIX operations.

.. autoclass:: owilix.core.manager.OWILIXManager
   :members:
   :show-inheritance:

Logging #

DBLog #

Thread-safe transaction logging for file processing with resume support.

from owilix.core.manager.logging import DBLog

# Initialize for a job
dblog = DBLog("my_job", "/path/to/logs")

# Register files
dblog.register_files([("/path/file1.parquet", "/dataset/path", "ds1")])

# Track progress
dblog.mark_files_success(["/path/file1.parquet"])

# Get statistics
stats = dblog.get_statistics()

.. autoclass:: owilix.core.manager.logging.DBLog
   :members:

UI Utilities #

OWILIXConsole #

Rich console with logging integration.

from owilix.core.manager import OWILIXConsole

console = OWILIXConsole()
console.info("Processing started")
console.error("Something went wrong")

EnhancedProgressDisplay #

Progress bars with error tracking.

from owilix.core.manager import EnhancedProgressDisplay, ErrorCollector

errors = ErrorCollector()
progress = EnhancedProgressDisplay(console, errors)
progress.start(total_files=100)
progress.update(advance=1, records=1000)
progress.stop()

See Also #