# OWILIX - Open Web Index CLI

[![Version](https://img.shields.io/badge/version-5.10.9-blue.svg)](https://openwebsearcheu-public.pages.it4i.eu/owi-cli/)
[![Python](https://img.shields.io/badge/python-3.12-green.svg)](https://www.python.org/)
[![License](https://img.shields.io/badge/license-Apache_2.0-orange.svg)](http://www.apache.org/licenses/LICENSE-2.0)

A command-line tool for managing [OpenWebIndex](http://openwebindex.eu) datasets across local and remote repositories.

## Quick Start

### One-Line Install (Recommended)

```bash
curl -fsSL https://openwebindex.net/owilix/install.sh | sh
```

This script:
- Installs [uv](https://docs.astral.sh/uv/) if not present
- Creates a virtual environment at `~/.venv/owilix`
- Installs the latest owilix
- Adds to your PATH
- Checks for config migrations

### Update

```bash
OWILIX_UPGRADE=1 curl -fsSL https://openwebindex.net/owilix/install.sh | sh
```

If your terminal paste inserts `\|`, remove the backslash so the command contains a normal pipe (`|`).

If the installer fails with a `python-http-irods-client` URL/dependency error, use the manual `uv` install shown below.

### Manual Install

<details>
<summary>Click to expand manual installation steps</summary>

```bash
# Install uv if not already installed
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create a virtual environment and install owilix
uv venv --python 3.12 ~/.venv/owilix
source ~/.venv/owilix/bin/activate
uv pip install \
  "python-http-irods-client @ git+https://opencode.it4i.eu/lexis-platform/data/python-http-irods-client.git@1.1.4" \
  owilix \
  --index-url https://opencode.it4i.eu/api/v4/projects/92/packages/pypi/simple \
  --extra-index-url https://opencode.it4i.eu/api/v4/projects/107/packages/pypi/simple

# Add to PATH (optional)
echo 'export PATH="$HOME/.venv/owilix/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc
```
</details>

### Verify Installation

```bash
owilix --version
owilix remote doctor
owilix remote ls all
```

After updates, check if config needs migration: `owilix config version`

## Help and Reporting

- join us in the [mattermost community channel](https://openwebsearch.eu/community/) 
- Or report issues at [the Open Code Gitlab](https://opencode.it4i.eu/openwebsearcheu-public/owi-cli/-/issues)

## Documentation

Full documentation: [openwebsearcheu-public.pages.it4i.eu/owi-cli](https://openwebsearcheu-public.pages.it4i.eu/owi-cli/)

- [Installation](https://openwebsearcheu-public.pages.it4i.eu/owi-cli/install)

- [Configuration](https://openwebsearcheu-public.pages.it4i.eu/owi-cli/config)
- [Migration](https://openwebsearcheu-public.pages.it4i.eu/owi-cli/migration)
- [Commands & Usage](https://openwebsearcheu-public.pages.it4i.eu/owi-cli/commands)
- [Query Commands](https://openwebsearcheu-public.pages.it4i.eu/owi-cli/details/query)
- [Development](https://openwebsearcheu-public.pages.it4i.eu/owi-cli/dev)

## Release Notes

For the latest changes and version history, see:
- [CHANGELOG.md](./CHANGELOG.md)

### Migration from v0.x

```bash
# Check current version
owilix config version

# Migrate config (creates backup)
owilix config migrate

# Or delete and start fresh
rm ~/.owi/owilix.cfg
owilix remote doctor
```

## Key Commands

```bash
# Remote operations
owilix remote ls all              # List all datasets
owilix remote ls lrz:latest       # List latest from specific DC
owilix remote pull all:latest     # Download latest datasets

# Local operations
owilix local ls all               # List local datasets
owilix local info <id>            # Show dataset details

# Configuration
owilix config version             # Show version info
owilix config list                # Show full config
owilix config add-repo my-s3 -t s3a --endpoint https://s3.example.com

# Query (DuckDB)
owilix query less --remote main:latest  # Quick data preview
owilix query sites --remote main:latest # Site statistics

# Remote full-text search
owilix remote search "open web search" --language eng --limit 10
owilix remote search "graz" --language deu --fetch --select "url,title,quality_score"

# Optional local HTTP API preview
uv pip install "owilix[http]"
owilix http serve                 # Binds to 127.0.0.1:8080 by default
```

The HTTP API is a preview local service. Do not expose it on a public or shared network without an external access-control layer.

## Docker

```bash
# Pull and run
docker run --rm -v ~/.owi:/home/owi/.owi \
  opencode.it4i.eu:5050/openwebsearcheu-public/owi-cli:latest --version

# Set alias
alias owilix='docker run --rm -it -v ~/.owi:/home/owi/.owi opencode.it4i.eu:5050/openwebsearcheu-public/owi-cli:latest'
```

## Screencast

Demo: [screencast video](https://syncandshare.lrz.de/getlink/fi9Dat4pLDLtE9bZsiGqFd/owi-demo.mov)

## Data License

By using this tool you agree to the [Open Web Index License (OWIL)](https://ows.eu/owil/current) and our [Terms of Use](https://dashboard.ows.eu/terms-of-use).

## Software License

OWILIX is licensed under the [Apache 2.0 License](http://www.apache.org/licenses/LICENSE-2.0).