\input texinfo
@documentencoding UTF-8
@settitle GoCheese

@copying
Copyright @copyright{} 2019-2020 @email{stargrave@@stargrave.org, Sergey Matveev}
@end copying

@node Top
@top

GoCheese is Python private package repository and caching proxy.

It serves two purposes:

@itemize
@item proxying and caching of missing packages from upstream
    @url{https://pypi.org/, PyPI}, conforming to
    @url{https://www.python.org/dev/peps/pep-0503/, PEP-0503}
    (Simple Repository API)
@item hosting of private locally uploaded packages, conforming to
    @url{https://warehouse.pypa.io/api-reference/legacy/, Warehouse Legacy API}
@end itemize

Why could you like it and how it can be better to fit your needs?

@itemize
@item No database required. Only filesystem storage with few simple
    files per package. Package deletion, renaming, making it uploadable
    (private) is done with simple @command{mkdir}, @command{touch}, etc
    commands
@item Just single statically compiled Go binary
@item No configuration file, but several simple command line arguments
@item Consistency (because of atomic synced operations) and integrity
    (because of SHA256 checksums stored nearby)
@end itemize

Initially it was created as a fork of
@url{https://github.com/c4s4/cheeseshop, cheeseshop},
but nearly all the code was rewritten. It has huge differences:

@itemize
@item Proxying and caching of missing packages, including GPG signatures
@item @url{https://pythonwheels.com/, Wheel} uploading support
@item Integrity check of proxied packages: MD5, SHA256, SHA512, BLAKE2b-256
@item SHA256 checksums for stored packages
@item Verifying of SHA256 checksum for uploaded packages
@item Storing of uploaded GPG signatures
@item Secure Argon2i (or SHA256) stored passwords hashing
@item No YAML configuration, just command-line arguments
@item No package overwriting ability (as PyPI does too)
@item Graceful HTTP-server shutdown
@item Atomic packages store on filesystem
@end itemize

Also it contains @file{contrib/pyshop2packages.sh} migration script for
converting @url{https://pypi.org/project/pyshop/, Pyshop} database into
GoCheese one, including private packages.

GoCheese is free software, licenced under
@url{https://www.gnu.org/licenses/gpl-3.0.html, GNU GPLv3}:
see the file COPYING for copying conditions.

Please send questions, bug reports and patches to @url{gocheese@@cypherpunks.ru}.

@insertcopying

@menu
* Install::
* Usage::
* Password authentication: Passwords.
* TLS support: TLS.
* Storage format: Storage.
@end menu

@include install.texi

@node Usage
@unnumbered Usage

To use it for download purposes, just configure your @file{pip.conf}:

@example
[install]
index-url = http://gocheese.host:8080/simple/
@end example

@option{-refresh} URL (@code{/simple/} by default) automatically
refreshes metainformation (available versions and their checksums)
from the upstream, when queried for package directory listing.
@option{-norefresh} prevents upstream queries.

@option{-gpgupdate} is useful mainly for migrated for Pyshop migrated
repositories. It forces GPG signature files downloading for all existing
package files.

You can upload packages to it with @url{https://pypi.org/project/twine/, twine}:

@example
twine upload
    --repository-url http://gocheese.host:8080/simple/ \
    --username spam \
    --password foo dist/tarball.tar.gz
@end example

Or you can store it permanently in @file{.pypirc}:

@example
[pypi]
repository: https://gocheese.host/simple/
username: spam
password: foo
@end example

If @command{twine} sends SHA256 checksum in the request, then uploaded
file is checked against it.

Pay attention that you have to manually create corresponding private
package directory! You are not allowed to upload anything explicitly
flagged as internal package.

@node Passwords
@unnumbered Password authentication

Password authentication is required for packages uploading.
You have to store your authentication data in @option{-passwd} file in
following format:

@example
username:hashed-password
@end example

Empty lines and having @verb{|#|} at the beginning are skipped.

Supported hashing algorithms are:

@table @asis

@item @url{https://www.argon2i.com/, Argon2i} (recommended one!)
    To get Argon2i hashed-password you can use any of following tools:
    @itemize
    @item go get @url{https://github.com/balakhonova/argon2i,
        github.com/balakhonova/argon2i} (Go)
    @item @url{https://github.com/p-h-c/phc-winner-argon2} (C)
    @end itemize
    Example user @code{foo} with password @code{bar} can have the
    following password file entry:

@verbatim
foo:$argon2i$v=19$m=32768,t=3,p=4$OGU5MTM3YjVlYzQwZjhkZA$rVn53v6Ckpf7WH0676ZQLr9Hbm6VH3YnL6I9ONJcIIU
@end verbatim

@item SHA256
    You can use your operating system tools:

@example
# BSD-based systems:
$ echo -n "password" | sha256

# GNU/Linux-based systems
$ echo -n "password" | sha256sum
@end example

    Example user @code{foo} with password @code{bar} will have the
    following password file entry:

@verbatim
foo:$sha256$fcde2b2edba56bf408601fb721fe9b5c338d10ee429ea04fae5511b68fbf8fb9
@end verbatim

@end table

You can refresh passwords by sending @code{SIGHUP} signal to the working daemon:

@example
$ pkill -HUP gocheese
$ kill -HUP `pidof gocheese`
@end example

Before refreshing it's recommended to check @option{-passwd} file with
@option{-passwd-check} option to prevent daemon failure.

@node TLS
@unnumbered TLS support

You can enable TLS support by specifying PEM-encoded X.509 certificate
and private key files. Go's TLS implementation supports TLS 1.3, HTTP/2
negotiation, Keep-Alives, modern ciphersuites and ECC.

For example generate some self-signed certificate using GnuTLS toolset:

@example
$ certtool --generate-privkey --ecc --outfile prv.pem
$ cert_template=`mktemp`
$ echo cn=gocheese.host > $cert_template
$ certtool \
    --generate-self-signed \
    --load-privkey=prv.pem \
    --template $cert_template \
    --outfile=cert.pem
$ rm $cert_template
$ gocheese -tls-cert cert.pem -tls-key prv.pem [...]
@end example

@node Storage
@unnumbered Storage format

Root directory has the following hierarchy:

@verbatim
root
  +-- public-package
  |     +- public-package-0.1.tar.gz.md5
  |     +- public-package-0.1.tar.gz.blake2_256
  |     +- public-package-0.1.1.tar.gz.blake2_256
  |     +- public-package-0.2.tar.gz
  |     +- public-package-0.2.tar.gz.asc
  |     +- public-package-0.2.tar.gz.sha256
  +-- private-package
  |     +- .internal
  |     +- private-package-0.1.tar.gz
  |     +- private-package-0.1.tar.gz.asc
  |     +- private-package-0.1.tar.gz.sha256
  |...
@end verbatim

Each directory is a normalized package name. When you try to list non
existent directory contents (you are downloading package you have not
seen before), then GoCheese will download information about package's
versions with checksums and write them in corresponding
@file{.sha256}, @file{.blake2_256}, @file{.sha512}, @file{.md5} files.
However no package package tarball is downloaded.

When you request for particular package version, then its tarball is
downloaded and verified against the stored checksum. But SHA256 is
forced to be stored and used later.

For example @file{public-package} has @code{0.1} version, downloaded a
long time ago with MD5 checksum. @code{0.1.1} version is downloaded more
recently with BLAKE2b-256 checksum, also storing that checksum for
@code{0.1}. @code{0.2} version is downloaded tarball, having forced
SHA256 recalculated checksum. Also upstream has corresponding
@file{.asc} signature file.

@file{private-package} is private package, because it contains
@file{.internal} file. It can be uploaded and queries to it are not
proxied to upstream PyPI. You have to create it manually. If you upload
GPG signature, then it will be also stored.

@bye