X-Git-Url: http://www.git.cypherpunks.ru/?a=blobdiff_plain;f=gocheese.texi;h=ee620dc9330c0be0d8ba34c93fca51fd0ae349c5;hb=3020e814c1e4e23baf281f4ff7a1121dd54f8fe3;hp=6cf12bca82477ddfc2cdcc7e064ab553759edaa4;hpb=870decf5b24cae444450c4105ad807b44b6676ca;p=gocheese.git diff --git a/gocheese.texi b/gocheese.texi index 6cf12bc..ee620dc 100644 --- a/gocheese.texi +++ b/gocheese.texi @@ -2,6 +2,10 @@ @documentencoding UTF-8 @settitle GoCheese +@copying +Copyright @copyright{} 2019-2020 @email{stargrave@@stargrave.org, Sergey Matveev} +@end copying + @node Top @top @@ -18,23 +22,38 @@ It serves two purposes: @url{https://warehouse.pypa.io/api-reference/legacy/, Warehouse Legacy API} @end itemize +Why could you like it and how it can be better to fit your needs? + +@itemize +@item No database required. Only filesystem storage with few simple + files per package. Package deletion, renaming, making it uploadable + (private) is done with simple @command{mkdir}, @command{touch}, etc + commands +@item Just single statically compiled Go binary +@item No configuration file, but several simple command line arguments +@item Consistency (because of atomic synced operations) and integrity + (because of SHA256 checksums stored nearby) +@end itemize + Initially it was created as a fork of @url{https://github.com/c4s4/cheeseshop, cheeseshop}, but nearly all the code was rewritten. It has huge differences: @itemize -@item proxying and caching of missing packages, including GPG signatures +@item Proxying and caching of missing packages, including GPG signatures @item @url{https://pythonwheels.com/, Wheel} uploading support -@item atomic packages store on filesystem -@item SHA256-checksummed packages: storing checksums, giving them back, - verifying stored files integrity, verifying checksum of uploaded - packaged -@item graceful HTTP-server shutdown -@item no YAML configuration, just command-line arguments -@item no package overwriting ability (as PyPI does too) +@item Integrity check of proxied packages: MD5, SHA256, SHA512, BLAKE2b-256 +@item SHA256 checksums for stored packages +@item Verifying of SHA256 checksum for uploaded packages +@item Storing of uploaded GPG signatures +@item Secure Argon2i (or SHA256) stored passwords hashing +@item No YAML configuration, just command-line arguments +@item No package overwriting ability (as PyPI does too) +@item Graceful HTTP-server shutdown +@item Atomic packages store on filesystem @end itemize -Also it contains @file{pyshop2packages.sh} migration script for +Also it contains @file{contrib/pyshop2packages.sh} migration script for converting @url{https://pypi.org/project/pyshop/, Pyshop} database into GoCheese one, including private packages. @@ -42,22 +61,29 @@ GoCheese is free software, licenced under @url{https://www.gnu.org/licenses/gpl-3.0.html, GNU GPLv3}: see the file COPYING for copying conditions. +Please send questions, bug reports and patches to @url{gocheese@@cypherpunks.ru}. + +@insertcopying + @menu +* Install:: * Usage:: * Password authentication: Passwords. * TLS support: TLS. * Storage format: Storage. @end menu +@include install.texi + @node Usage @unnumbered Usage To use it for download purposes, just configure your @file{pip.conf}: -@verbatim +@example [install] index-url = http://gocheese.host:8080/simple/ -@end verbatim +@end example @option{-refresh} URL (@code{/simple/} by default) automatically refreshes metainformation (available versions and their checksums) @@ -70,25 +96,29 @@ package files. You can upload packages to it with @url{https://pypi.org/project/twine/, twine}: -@verbatim +@example twine upload --repository-url http://gocheese.host:8080/simple/ \ --username spam \ --password foo dist/tarball.tar.gz -@end verbatim +@end example Or you can store it permanently in @file{.pypirc}: -@verbatim +@example [pypi] repository: https://gocheese.host/simple/ username: spam password: foo -@end verbatim +@end example If @command{twine} sends SHA256 checksum in the request, then uploaded file is checked against it. +Pay attention that you have to manually create corresponding private +package directory! You are not allowed to upload anything explicitly +flagged as internal package. + @node Passwords @unnumbered Password authentication @@ -96,9 +126,9 @@ Password authentication is required for packages uploading. You have to store your authentication data in @option{-passwd} file in following format: -@verbatim +@example username:hashed-password -@end verbatim +@end example Empty lines and having @verb{|#|} at the beginning are skipped. @@ -109,8 +139,8 @@ Supported hashing algorithms are: @item @url{https://www.argon2i.com/, Argon2i} (recommended one!) To get Argon2i hashed-password you can use any of following tools: @itemize - @item @url{https://github.com/balakhonova/argon2i, - go get github.com/balakhonova/argon2i} (Go) + @item go get @url{https://github.com/balakhonova/argon2i, + github.com/balakhonova/argon2i} (Go) @item @url{https://github.com/p-h-c/phc-winner-argon2} (C) @end itemize Example user @code{foo} with password @code{bar} can have the @@ -123,13 +153,14 @@ foo:$argon2i$v=19$m=32768,t=3,p=4$OGU5MTM3YjVlYzQwZjhkZA$rVn53v6Ckpf7WH0676ZQLr9 @item SHA256 You can use your operating system tools: -@verbatim +@example # BSD-based systems: $ echo -n "password" | sha256 # GNU/Linux-based systems $ echo -n "password" | sha256sum -@end verbatim +@end example + Example user @code{foo} with password @code{bar} will have the following password file entry: @@ -141,10 +172,10 @@ foo:$sha256$fcde2b2edba56bf408601fb721fe9b5c338d10ee429ea04fae5511b68fbf8fb9 You can refresh passwords by sending @code{SIGHUP} signal to the working daemon: -@verbatim +@example $ pkill -HUP gocheese $ kill -HUP `pidof gocheese` -@end verbatim +@end example Before refreshing it's recommended to check @option{-passwd} file with @option{-passwd-check} option to prevent daemon failure. @@ -158,7 +189,7 @@ negotiation, Keep-Alives, modern ciphersuites and ECC. For example generate some self-signed certificate using GnuTLS toolset: -@verbatim +@example $ certtool --generate-privkey --ecc --outfile prv.pem $ cert_template=`mktemp` $ echo cn=gocheese.host > $cert_template @@ -169,7 +200,7 @@ $ certtool \ --outfile=cert.pem $ rm $cert_template $ gocheese -tls-cert cert.pem -tls-key prv.pem [...] -@end verbatim +@end example @node Storage @unnumbered Storage format @@ -179,31 +210,41 @@ Root directory has the following hierarchy: @verbatim root +-- public-package - | +- public-package-0.1.tar.gz.sha256 + | +- public-package-0.1.tar.gz.md5 + | +- public-package-0.1.tar.gz.blake2_256 + | +- public-package-0.1.1.tar.gz.blake2_256 | +- public-package-0.2.tar.gz | +- public-package-0.2.tar.gz.asc | +- public-package-0.2.tar.gz.sha256 +-- private-package | +- .internal | +- private-package-0.1.tar.gz + | +- private-package-0.1.tar.gz.asc | +- private-package-0.1.tar.gz.sha256 |... @end verbatim -Each directory is a package name. When you try to list non existent -directory contents (you are downloading package you have not seen -before), then GoCheese will download information about package's -versions with checksums and write them in corresponding @file{.sha256} -files. However no package package tarball is downloaded. +Each directory is a normalized package name. When you try to list non +existent directory contents (you are downloading package you have not +seen before), then GoCheese will download information about package's +versions with checksums and write them in corresponding +@file{.sha256}, @file{.blake2_256}, @file{.sha512}, @file{.md5} files. +However no package package tarball is downloaded. When you request for particular package version, then its tarball is -downloaded and verified against the checksum. For example in the root -directory above we have downloaded only @file{public-package-0.2}. -If upstream has corresponding @file{.asc} file, then it also will be -downloaded. - -Private packages contain @file{.internal} file, indicating that it must -not be asked in PyPI if required version is missing. You have to create -it manually. +downloaded and verified against the stored checksum. But SHA256 is +forced to be stored and used later. + +For example @file{public-package} has @code{0.1} version, downloaded a +long time ago with MD5 checksum. @code{0.1.1} version is downloaded more +recently with BLAKE2b-256 checksum, also storing that checksum for +@code{0.1}. @code{0.2} version is downloaded tarball, having forced +SHA256 recalculated checksum. Also upstream has corresponding +@file{.asc} signature file. + +@file{private-package} is private package, because it contains +@file{.internal} file. It can be uploaded and queries to it are not +proxied to upstream PyPI. You have to create it manually. If you upload +GPG signature, then it will be also stored. @bye