Command line interface for accessing the Open Web Index shards and slices.
owi-cli docs source migration.md
3.2 kB · 107 lines
Markdown
at main

Migration Guide for OWILIX #

Version 3.0 → 4.0.0 (Release Update) #

Key Changes #

  • Configuration: Updated server update URL to version-specific endpoint (default-cfg-4-0).
  • Repositories: Standardized on unified lexis repository type (dropping deprecated names).
  • Core: Transitioned to py4lexis 5.x HTTP client support.
  • CLI: Added config add-repo, config select, and config migrate commands.

Migration Steps #

  1. Check version status:

    pwilix config version
    
  2. Run automated migration:

    owilix config migrate
    

    This will backup your existing config and update server URLs and repository definitions.

  3. Verify configuration:

    owilix config list
    

Version 0.24 → 0.25 (Old Beta Changes) #

Local Dataset Storage Structure #

Starting with version 0.25, local datasets are stored using a structured folder hierarchy:

~/.owi/
├── public/
│   ├── main/                    # Default collection
│   │   ├── {dataset-id}/
│   │   └── {dataset-id}.json
│   └── {collection-name}/      # Custom collections (e.g., gpu, userslice)
│       ├── {dataset-id}/
│       └── {dataset-id}.json
├── project/
│   └── ...
└── user/
    └── ...

Caution

Data Migration Required: Datasets created in version 0.24 and earlier may not follow this structure. You must manually migrate or delete existing datasets.

Migration Steps #

  1. Backup your data (optional but recommended):

    cp -r ~/.owi ~/.owi.backup.0.24
    
  2. Delete existing local datasets (simplest approach):

    rm -rf ~/.owi/public
    rm -rf ~/.owi/project
    rm -rf ~/.owi/user
    
  3. Re-pull datasets from remote repositories after upgrading:

    owi remote pull <specifier>
    

Why This Change? #

The new access/collectionName/dataset-id structure enables:

  • Clear separation of datasets by access level and collection
  • Faster filtering when listing datasets by collection
  • Consistent path construction between local and remote repositories
  • Support for custom dataset IDs (no longer UUID-only)

Affected Commands #

Command Impact
owi local ls Now filters by collectionName correctly
owi local insert Creates datasets in proper {access}/{collection}/ path
owi local export Exports from correct collection-based paths
owi local free Removes datasets from correct paths

Version 0.25 → 0.26: Metadata Schema (Implemented) #

Pydantic-based metadata schema has replaced the dictionary-based approach in v0.26:

  • Location: owilix/core/models/schema.py - DataCite 4.6 compatible Pydantic models
  • Dataset class: owilix/core/models/dataset.py (moved from owilix/core/metadata.py)
  • Strict validation of DataCite-compatible fields
  • Type safety for all metadata operations
  • Better error messages for invalid metadata
from owilix.core.models.schema import DatasetMetadataModel
from owilix.core.models.dataset import Dataset, DatasetMetadata