phorge / src / docs / user / userguide / diffusion_uris.diviner

@title Diffusion User Guide: URIs
@group userguide

Guide to configuring repository URIs for fetching, cloning and mirroring.

Overview
========

Phorge can create, host, observe, mirror, proxy, and import repositories.
For example, you can:

**Host Repositories**: Phorge can host repositories locally. Phorge
maintains the writable master version of the repository, and you can push and
pull the repository. This is the most straightforward kind of repository
configuration, and similar to repositories on other services like GitHub or
Bitbucket.

**Observe Repositories**: Phorge can create a copy of an repository which
is hosted elsewhere (like GitHub or Bitbucket) and track updates to the remote
repository. This will create a read-only copy of the repository in Phorge.

**Mirror Repositories**: Phorge can publish any repository to mirrors,
overwriting them with an exact copy of the repository that stays up to date as
the source changes. This works with both local repositories that Phorge is
hosting and remote repositories that Phorge is observing.

**Proxy Repositories**: If you are observing a repository, you can allow users
to read Phorge's copy of the repository. Phorge supports granular
read permissions, so this can let you open a private repository up a little
bit in a flexible way.

**Import Repositories**: If you have a repository elsewhere that you want to
host on Phorge, you can observe the remote repository first, then turn
the tracking off once the repository fully synchronizes. This allows you to
copy an existing repository and begin hosting it in Phorge.

You can also import repositories by creating an empty hosted repository and
then pushing everything to the repository directly.

You configure the behavior of a Phorge repository by adding and
configuring URIs and marking them to be fetched from, mirrored to, clonable,
and so on. By configuring all the URIs that a repository should interact with
and expose to users, you configure the read, write, and mirroring behavior
of the repository.

The remainder of this document walks through this configuration in greater
detail.


Host a Repository
=================

You can create new repositories that Phorge will host, like you would
create repositories on services like GitHub or Bitbucket. Phorge will
serve a read-write copy of the repository and you can clone it from Phorge
and push changes to Phorge.

If you haven't already, you may need to configure Phorge for hosting
before you can create your first hosted repository. For a detailed guide,
see @{article:Diffusion User Guide: Repository Hosting}.

This is the default mode for new repositories. To host a repository:

  - Create a new repository.
  - Activate it.

Phorge will create an empty repository and allow you to fetch from it and
push to it.


Observe a Repository
====================

If you have an existing repository hosted on another service (like GitHub,
Bitbucket, or a private server) that you want to work with in Phorge,
you can configure Phorge to observe it.

When observing a repository, Phorge will keep track of changes in the
remote repository and allow you to browse and interact with the repository from
the web UI in Diffusion and other applications, but you can continue hosting it
elsewhere.

To observe a repository:

  - Create a new repository, but don't activate it yet.
  - Add the remote URI you want to observe as a repository URI.
  - Set the **I/O Type** for the URI to **Observe**.
  - If necessary, configure a credential.
  - Activate the repository.

Phorge will perform an initial import of the repository, creating a local
read-only copy. Once this process completes, it will continue keeping track of
changes in the remote, fetching them, and reflecting them in the UI.


Mirror a Repository
===================

NOTE: Mirroring is not supported in Subversion.

You can create a read-only mirror of an existing repository. Phorge will
continuously publish the state of the source repository to the mirror, creating
an exact copy.

For example, if you have a repository hosted in Phorge that you want to
mirror to GitHub, you can configure Phorge to automatically maintain the
mirror. This is how the upstream repositories are set up.

The mirror copy must be read-only for users because any writes made to the
mirror will be undone when Phorge updates it. The mirroring process copies
the entire repository state exactly, so the remote state will be completely
replaced with an exact copy of the source repository. This may remove or
destroy information. Normally, you should only mirror to an empty repository.

You can mirror any repository, even if Phorge is only observing it and not
hosting it directly.

To begin mirroring a repository:

  - Create a hosted or observed repository by following the relevant
    instructions above.
  - Add the remote URI you want to mirror to as a repository URI.
  - Set the **I/O Type** for the URI to **Mirror**.
  - If necessary, configure a credential.

To stop mirroring:

  - Disable the mirror URI; or
  - Change the **I/O Type** for the URI to **None**.


Import a Repository
===================

If you have an existing repository that you want to move so it is hosted on
Phorge, there are three ways to do it:

**Observe First**: //(Git, Mercurial)// Observe the existing repository first,
according to the instructions above. Once Phorge's copy of the repository
is fully synchronized, change the **I/O Type** for the **Observe** URI to
**None** to stop fetching changes from the remote.

By default, this will automatically make Phorge's copy of the repository
writable, and you can begin pushing to it. If you've adjusted URI
configuration away from the defaults, you may need to set at least one URI
to **Read/Write** mode so you can push to it.

**Push Everything**: //(Git, Mercurial, Subversion)// Create a new empty hosted
repository according to the instructions above. Once the empty repository
initializes, push your entire existing repository to it.

In Subversion, you can do this with the `svnsync` tool.

**Copy on Disk**: //(Git, Mercurial, Subversion)// Create a new empty hosted
repository according to the instructions above, but do not activate it yet.

Using the **Storage** tab, find the location of the repository's working copy
on disk, and place a working copy of the repository you wish to import there.

For Git and Mercurial, use a bare working copy for best results.

This is the only way to import a Subversion repository because only the master
copy of the repository has history.

Once you've put a working copy in the right place on disk, activate the
repository.


Builtin Clone URIs
==================

By default, Phorge automatically exposes and activates HTTP, HTTPS and
SSH clone URIs by examining configuration.

**HTTP**: The `http://` clone URI will be available if these conditions are
satisfied:

  - `diffusion.allow-http-auth` must be enabled or the repository view policy
    must be "Public".
  - The repository must be a Git or Mercurial repository.
  - `security.require-https` must be disabled.

**HTTPS**: The `https://` clone URI will be available if these conditions are
satisfied:

  - `diffusion.allow-http-auth` must be enabled or the repository view policy
    must be "Public".
  - The repository must be a Git or Mercurial repository.
  - The `phabricator.base-uri` protocol must be `https://`.

**SSH**: The `ssh://` or `svn+ssh://` clone URI will be available if these
conditions are satisfied:

  - `phd.user` must be configured.


Customizing Displayed Clone URIs
================================

If you have an unusual configuration and want the UI to offers users specific
clone URIs other than the URIs that Phorge serves or interacts with, you
can add those URIs with the **I/O Type** set to **None** and then set their
**Display Type** to **Always**.

Likewise, you can set the **Display Type** of any URIs you do //not// want
to be visible to **Never**.

This allows you to precisely configure which clone URIs are shown to users for
a repository.


Reference: I/O Types
====================

This section details the available **I/O Type** options for URIs.

Each repository has some **builtin** URIs. These are URIs hosted by Phorge
itself. The modes available for each URI depend primarily on whether it is a
builtin URI or not.

**Default**: This setting has Phorge guess the correct option for the
URI.

For **builtin** URIs, the default behavior is //Read/Write// if the repository
is hosted, and //Read-Only// if the repository is observed.

For custom URIs, the default type is //None// because we can not automatically
guess if you want to ignore, observe, or mirror a URI and //None// is the
safest default.

**Observe**: Phorge will observe this repository and regularly fetch any
changes made to it to a local read-only copy.

You can not observe builtin URIs because reading a repository from itself
does not make sense.

You can not add a URI in Observe mode if an existing builtin URI is in
//Read/Write// mode, because this would mean the repository had two different
authorities: the observed remote copy and the hosted local copy. Take the
other URI out of //Read/Write// mode first.

WARNING: If you observe a remote repository, the entire state of the working
copy that Phorge maintains will be deleted and replaced with the state of
the remote. If some changes are present only in Phorge's working copy,
they will be unrecoverably destroyed.

**Mirror**: Phorge will push any changes made to this repository to the
remote URI, keeping a read-only mirror hosted at that URI up to date.

This works for both observed and hosted repositories.

This option is not available for builtin URIs because it does not make sense
to mirror a repository to itself.

It is possible to mirror a repository to another repository that is also
hosted by Phorge by adding that other repository's URI, although this is
silly and probably very rarely of any use.

WARNING: If you mirror to a remote repository, the entire state of that remote
will be replaced with the state of the working copy Phorge maintains. If
some changes are present only in the remote, they will be unrecoverably
destroyed.

**None**: Phorge will not fetch changes from or push changes to this URI.
For builtin URIs, it will not let users fetch changes from or push changes to
this URI.

You can use this mode to turn off an Observe URI after an import, stop a Mirror
URI from updating, or to add URIs that you're only using to customize which
clone URIs are displayed to the user but don't want Phorge to interact
with directly.

**Read Only**: Phorge will serve the repository from this URI in read-only
mode. Users will be able to fetch from it but will not be able to push to it.

Because Phorge must be able to serve the repository from URIs configured
in this mode, this option is only available for builtin URIs.

**Read/Write**: Phorge will serve the repository from this URI in
read/write mode. Users will be able to fetch from it and push to it.

URIs can not be set into this mode if another URI is set to //Observe// mode,
because that would mean the repository had two different authorities: the
observed remote copy and the hosted local copy. Take the other URI out of
//Observe// mode first.

Because Phorge must be able to serve the repository from URIs configured
in this mode, this option is only available for builtin URIs.


Reference: Display Types
========================

This section details the available **Display Type** options for URIs.

**Default**: Phorge will guess the correct option for the URI. It
guesses based on the configured **I/O Type** and whether the URI is
**builtin** or not.

For //Observe//, //Mirror// and //None// URIs, the default is //Never//.

For builtin URIs in //Read Only// or //Read/Write// mode, the most
human-readable URI defaults to //Always// and the others default to //Never//.

**Always**: This URI will be shown to users as a clone/checkout URI. You can
add URIs in this mode to customize exactly what users are shown.

**Never**: This URI will not be shown to users. You can hide less-preferred
URIs to guide users to the URIs they should be using to interact with the
repository.


Next Steps
==========

Continue by:

  - configuring Phorge to host repositories with
    @{article:Diffusion User Guide: Repository Hosting}.