@documentencoding UTF-8
@settitle GoCheese
+@copying
+Copyright @copyright{} 2019-2021 @email{stargrave@@stargrave.org, Sergey Matveev}
+@end copying
+
@node Top
@top
It serves two purposes:
@itemize
-@item hosting of private locally uploaded packages (conforming to
- @url{https://www.python.org/dev/peps/pep-0503/, PEP-0503} (Simple
- Repository API))
@item proxying and caching of missing packages from upstream
- @url{https://pypi.org/, PyPI}
+ @url{https://pypi.org/, PyPI}, conforming to
+ @url{https://www.python.org/dev/peps/pep-0503/, PEP-0503}
+ (Simple Repository API)
+@item hosting of private locally uploaded packages, conforming to
+ @url{https://warehouse.pypa.io/api-reference/legacy/, Warehouse Legacy API}
+@end itemize
+
+Why could you like it and how it can be better to fit your needs?
+
+@itemize
+@item No database required. Only filesystem storage with few simple
+ files per package. Package deletion, renaming, making it uploadable
+ (private) is done with simple @command{mkdir}, @command{touch}, etc
+ commands
+@item Just single statically compiled Go binary
+@item No configuration file, but several simple command line arguments
+@item Consistency (because of atomic synced operations) and integrity
+ (because of SHA256 checksums stored nearby)
@end itemize
Initially it was created as a fork of
but nearly all the code was rewritten. It has huge differences:
@itemize
-@item proxying and caching of missing packages
-@item atomic packages store on filesystem
-@item SHA256-checksummed packages (both uploaded and proxied one)
-@item graceful HTTP-server shutdown
-@item no YAML configuration, just command-line arguments
-@item no package overwriting ability (as PyPI does)
+@item Proxying and caching of missing packages, including GPG signatures
+@item @url{https://pythonwheels.com/, Wheel} uploading support
+@item Integrity check of proxied packages: MD5, SHA256, SHA512, BLAKE2b-256
+@item SHA256 checksums for stored packages
+@item Verifying of SHA256 checksum for uploaded packages
+@item Ability to authenticate upstream PyPI with its X.509 certificate's hash
+@item Storing of uploaded GPG signatures
+@item Secure Argon2i (or SHA256) stored passwords hashing
+@item No YAML configuration, just command-line arguments
+@item No package overwriting ability (as PyPI does too)
+@item Graceful HTTP-server shutdown
+@item Atomic packages store on filesystem
@end itemize
-GoCheese is free software, licenced under
-@url{https://www.gnu.org/licenses/gpl-3.0.html, GNU GPLv3}:
-see the file COPYING for copying conditions.
+Also it contains @file{contrib/pyshop2packages.sh} migration script for
+converting @url{https://pypi.org/project/pyshop/, Pyshop} database into
+GoCheese one, including private packages.
+
+GoCheese is
+@url{https://www.gnu.org/philosophy/pragmatic.html, copylefted}
+@url{https://www.gnu.org/philosophy/free-sw.html, free software}
+licenced under @url{https://www.gnu.org/licenses/gpl-3.0.html, GNU GPLv3}.
+
+Please send questions, bug reports and patches to @url{gocheese@@cypherpunks.ru}.
+
+@insertcopying
@menu
+* Install::
* Usage::
* Password authentication: Passwords.
* TLS support: TLS.
* Storage format: Storage.
@end menu
+@include install.texi
+
@node Usage
@unnumbered Usage
To use it for download purposes, just configure your @file{pip.conf}:
-@verbatim
+@example
[install]
index-url = http://gocheese.host:8080/simple/
-@end verbatim
+@end example
-@option{-refresh} URL behaves the same way as @option{-simple} one, but
-is always refreshes package versions from PyPI when listing it. You can
-use it to forcefully update known package versions.
+@option{-refresh} URL (@code{/simple/} by default) automatically
+refreshes metainformation (available versions and their checksums)
+from the upstream, when queried for package directory listing.
+@option{-norefresh} prevents upstream queries.
-You can upload packages to it with
-@url{https://pypi.org/project/twine/, twine}:
+@option{-gpgupdate} is useful mainly for migrated for Pyshop migrated
+repositories. It forces GPG signature files downloading for all existing
+package files.
-@verbatim
+You can upload packages to it with @url{https://pypi.org/project/twine/, twine}:
+
+@example
twine upload
--repository-url http://gocheese.host:8080/simple/ \
--username spam \
--password foo dist/tarball.tar.gz
-@end verbatim
+@end example
+
+Or you can store it permanently in @file{.pypirc}:
+
+@example
+[pypi]
+repository: https://gocheese.host/simple/
+username: spam
+password: foo
+@end example
+
+If @command{twine} sends SHA256 checksum in the request, then uploaded
+file is checked against it.
+
+Pay attention that you have to manually create corresponding private
+package directory! You are not allowed to upload anything explicitly
+flagged as internal package.
+
+It is advisable to run GoCheese under some kind of
+@url{http://cr.yp.to/daemontools.html, daemontools}.
@node Passwords
@unnumbered Password authentication
You have to store your authentication data in @option{-passwd} file in
following format:
-@verbatim
+@example
username:hashed-password
-@end verbatim
+@end example
Empty lines and having @verb{|#|} at the beginning are skipped.
@item @url{https://www.argon2i.com/, Argon2i} (recommended one!)
To get Argon2i hashed-password you can use any of following tools:
@itemize
- @item @url{https://github.com/balakhonova/argon2i,
- go get github.com/balakhonova/argon2i} (Go)
+ @item go get @url{https://github.com/balakhonova/argon2i,
+ github.com/balakhonova/argon2i} (Go)
@item @url{https://github.com/p-h-c/phc-winner-argon2} (C)
@end itemize
Example user @code{foo} with password @code{bar} can have the
@item SHA256
You can use your operating system tools:
-@verbatim
+@example
# BSD-based systems:
$ echo -n "password" | sha256
# GNU/Linux-based systems
$ echo -n "password" | sha256sum
-@end verbatim
+@end example
+
Example user @code{foo} with password @code{bar} will have the
following password file entry:
You can refresh passwords by sending @code{SIGHUP} signal to the working daemon:
-@verbatim
+@example
$ pkill -HUP gocheese
$ kill -HUP `pidof gocheese`
-@end verbatim
+$ svc -h /var/service/gocheese
+@end example
Before refreshing it's recommended to check @option{-passwd} file with
@option{-passwd-check} option to prevent daemon failure.
For example generate some self-signed certificate using GnuTLS toolset:
-@verbatim
+@example
$ certtool --generate-privkey --ecc --outfile prv.pem
$ cert_template=`mktemp`
$ echo cn=gocheese.host > $cert_template
--outfile=cert.pem
$ rm $cert_template
$ gocheese -tls-cert cert.pem -tls-key prv.pem [...]
-@end verbatim
+@end example
@node Storage
@unnumbered Storage format
@verbatim
root
+-- public-package
- | +- public-package-0.1.tar.gz.sha256
+ | +- public-package-0.1.tar.gz.md5
+ | +- public-package-0.1.tar.gz.blake2_256
+ | +- public-package-0.1.1.tar.gz.blake2_256
| +- public-package-0.2.tar.gz
+ | +- public-package-0.2.tar.gz.asc
| +- public-package-0.2.tar.gz.sha256
+-- private-package
- | +- .private
+ | +- .internal
| +- private-package-0.1.tar.gz
+ | +- private-package-0.1.tar.gz.asc
| +- private-package-0.1.tar.gz.sha256
|...
@end verbatim
-Each directory is a package name. When you try to list non existent
-directory contents (you are downloading package you have not seen
-before), then GoCheese will download information about package's
-versions with checksums and write them in corresponding @file{.sha256}
-files. However no package package tarball is downloaded.
+Each directory is a normalized package name. When you try to list non
+existent directory contents (you are downloading package you have not
+seen before), then GoCheese will download information about package's
+versions with checksums and write them in corresponding
+@file{.sha256}, @file{.blake2_256}, @file{.sha512}, @file{.md5} files.
+However no package package tarball is downloaded.
When you request for particular package version, then its tarball is
-downloaded and verified against the checksum. For example in the root
-directory above we have downloaded only @file{public-package-0.2}.
-
-Private packages contain @file{.private} file, indicating that it must
-not be asked in PyPI if required version is missing. You have to create
-it manually.
+downloaded and verified against the stored checksum. But SHA256 is
+forced to be stored and used later.
+
+For example @file{public-package} has @code{0.1} version, downloaded a
+long time ago with MD5 checksum. @code{0.1.1} version is downloaded more
+recently with BLAKE2b-256 checksum, also storing that checksum for
+@code{0.1}. @code{0.2} version is downloaded tarball, having forced
+SHA256 recalculated checksum. Also upstream has corresponding
+@file{.asc} signature file.
+
+@file{private-package} is private package, because it contains
+@file{.internal} file. It can be uploaded and queries to it are not
+proxied to upstream PyPI. You have to create it manually. If you upload
+GPG signature, then it will be also stored.
@bye