| @@ -11,7 +11,8 @@ Small and lightweight git server for self hosting. Repositories are served over |
| 11 | 11 | - Forks and pull requests from either a branch or a fork. Discussion comments and inline review comments on any line of the diff are Markdown, review threads can be resolved and go to an outdated section when the line leaves the diff, and a request can be closed or reopened or brought up to date from its source at any point |
| 12 | 12 | - Three merge strategies, a merge commit with an editable message, a squash authored by the request author, or a rebase that replays each commit onto the target. All three are built from `merge-tree` and `commit-tree` without a working tree and fail cleanly on conflict |
| 13 | 13 | - Collaborators per repository who can push, read it when private and merge pull requests, while settings and deletion stay with the owner |
| 14 | | -- A per user disk quota set with `gitpocket quota 2G` and enforced on push over both transports |
| 14 | +- Organisations, which are accounts that cannot log in and whose repositories every member owns. Anyone can create one from the dashboard, and members edit its picture, bio and membership from its settings page, so a project can outlive the person who started it |
| 15 | +- A per owner disk quota that the push itself enforces. Each push over either transport runs `receive-pack` with `receive.maxInputSize` set to whatever the owner has left, so git refuses an oversized pack before writing it |
| 15 | 16 | - Signature verification for both SSH and PGP. Register the key in settings, sign commits with it, and they show `Verified ~you`. PGP keys are imported into a server keyring and matched by primary fingerprint, SSH keys through an allowed signers file matched by fingerprint. Keys are public at `~you.keys` and `~you.pgp` |
| 16 | 17 | - Issues per repository with Markdown bodies and comments, open and closed states, and an open count in the tab strip |
| 17 | 18 | - A site wide search at `/search` over repositories and people, ranked so an exact name beats a prefix beats a substring. Qualifiers narrow it the way they do elsewhere, `language:ocaml` (case insensitive, quoted when the name has a space), `user:name` and `tag:sel4`, and the language facets write the qualifier into the query box so the filter survives the next search. Each repository's languages come from the byte size of its files at HEAD by extension, skipping minified and vendored paths, and are cached against the HEAD hash so the language filter and the breakdown bar on the summary page cost one `git ls-tree` per push |
| @@ -21,16 +22,21 @@ Small and lightweight git server for self hosting. Repositories are served over |
| 21 | 22 | - Every pushed revision of a pull request is kept under `refs/pull/<n>/v<k>`, and the request page links a `git range-diff` between any two of them, so a reviewer sees what a force push actually changed in each commit instead of rereading the whole series |
| 22 | 23 | - History search on the log page through git's pickaxe, either commits that change how often a string occurs (`-S`) or commits whose diff has a line matching a regex (`-G`), which answers when a line arrived or vanished without cloning |
| 23 | 24 | - Atom feeds of commits per branch, of issues and of pull requests per repository, and of every public commit by a user at `~you/feed`, which stand in for notifications when mail is not configured. Each page names its feeds in a `<link rel="alternate">`, so a reader finds them from the ordinary page URL |
| 24 | | -- Rename, transfer to another user and archive from the repository settings. An archived repository refuses pushes, pull requests and comments but stays readable |
| 25 | +- Rename, transfer to another user and archive from the repository settings. The old path keeps working as a permanent redirect for the web, HTTPS clones and SSH, provided the visitor can read the repository at its new home, and an archived repository refuses pushes, pull requests and comments but stays readable |
| 25 | 26 | - An optional email address on each account, verified by a link, used for password resets and for notifications about pull requests, reviews and issues on your repositories. Both can be turned off and nothing works without the address, so the whole thing is opt in |
| 26 | 27 | - TOTP two factor authentication with a QR code for the authenticator app and eight single use recovery codes, which is what a password alone cannot give you on something that accepts pushes |
| 27 | 28 | - SSH keys and access tokens live in settings. HTTPS auth is plain basic auth with a token or your password |
| 28 | 29 | - Relative links and images in a README resolve against the repository, so a `docs/diagram.png` reference renders rather than breaking |
| 29 | | -- Registration is invite only unless you say otherwise, and there is a failure counter per address covering login and register and password changes and HTTPS auth |
| 30 | +- Registration is open, invite only or closed, and an administrator switches between the three on the admin page or with `gitpocket registration`. Signing up needs only a name and password, and each address can create a handful of accounts every few minutes |
| 31 | +- `/admin` exists for accounts you mark with `gitpocket admin <name>` and 404s for everyone else. From there you flip registration, mint invite links, set the quota, order the front page and ban people |
| 30 | 32 | |
| 31 | 33 | ## Design |
| 32 | 34 | |
| 33 | | -Every read of git data is a call out to the plumbing with a thirty second timeout, which in practice means `ls-tree -l` for trees and `log` with a NUL separated format for history and `diff-tree -p --root -M` for commit pages and `cat-file` for blobs. Nothing is cached anywhere and nothing is parsed out of packfiles in process, so a large repository costs exactly what git says it costs and the server itself holds no state between one request and the next. |
| 35 | +Every read of git data is a call out to the plumbing with a thirty second timeout, which in practice means `ls-tree -l` for trees and `log` with a NUL separated format for history and `diff-tree -p --root -M` for commit pages and `cat-file` for blobs. Nothing is parsed out of packfiles in process, so a large repository costs exactly what git says it costs. A buffered call that tries to return more than 16 MiB gets killed. Only twice the core count may run together, or four on a single core box, and raw files and archives stream two at a time. On git.vm.fail the server idles at about 35 MB resident, so most of a 512 MB VPS is left for git. |
| 36 | + |
| 37 | +A little is remembered between requests, though deleting all of it loses nothing. Language bars and the history behind the activity graph are stored against the mtime of the refs, so after a push one `stat` notices and before one nothing is read at all. That history holds sixteen bytes a commit (a hash prefix, the day and which author address it was), and the list of people that search walks is refreshed at most twice a minute. |
| 38 | + |
| 39 | +Each client address gets a token bucket of five requests a second with a burst of 120. Search, blame, downloads, pickaxe and login draw from a second, much slower bucket because they are the pages that start real work in git. Pages are sent with a content security policy that only runs scripts served by the site, forbids framing and keeps the referrer inside the origin. Raw files go out as plain text unless they are images, audio or video, and those get a sandbox of their own, so a committed HTML or SVG file cannot run anything in a visitor's session. Go's cross origin protection rejects form posts from other sites before any handler sees them. |
| 34 | 40 | |
| 35 | 41 | When a pull request is opened or updated the source branch gets fetched into `refs/pull/N/head` of the target repository, and from that point on the diff and the commit list and the question of whether it merges cleanly all come from `merge-base` and `diff` and `merge-tree --write-tree` inside that single repo. The merge itself is `commit-tree` with both parents followed by `update-ref` carrying the old value it expects to find, so if someone pushes to the branch while you are clicking the button the merge simply fails rather than silently overwriting them. Conflicts are reported back to you and left for you to sort out locally. |
| 36 | 42 | |
| @@ -44,16 +50,14 @@ useradd -r -m -d /srv/gitpocket -s /bin/sh git |
| 44 | 50 | install -m 755 gitpocket /usr/local/bin/gitpocket |
| 45 | 51 | ``` |
| 46 | 52 | |
| 53 | +Copy the systemd units and nginx site from the deploy directory and change the host name in them. The service unit sets `GOMEMLIMIT` to 192 MiB and gives systemd a hard 512 MB ceiling covering the server and every git it starts, and the backup timer fires once a night. |
| 54 | + |
| 47 | 55 | ``` |
| 48 | | -[Service] |
| 49 | | -User=git |
| 50 | | -ExecStart=/usr/local/bin/gitpocket -root /srv/gitpocket serve -listen 127.0.0.1:3200 -host git.example.org |
| 51 | | -Restart=always |
| 52 | | -ProtectSystem=strict |
| 53 | | -ReadWritePaths=/srv/gitpocket |
| 56 | +install -m 644 deploy/gitpocket.service deploy/gitpocket-backup.service deploy/gitpocket-backup.timer /etc/systemd/system/ |
| 57 | +systemctl enable --now gitpocket gitpocket-backup.timer |
| 54 | 58 | ``` |
| 55 | 59 | |
| 56 | | -Put a reverse proxy with TLS in front of it. The proxy needs to forward `Host` and `X-Forwarded-Proto`, and it should set `X-Forwarded-For` to the client address rather than appending to it because the rate limiter keys on the first value it finds there. Turn request buffering off and raise the body limit as well, otherwise larger pushes over HTTPS will be cut short. |
| 60 | +Put a reverse proxy with TLS in front of it. The proxy needs to forward `Host` and `X-Forwarded-Proto` and should set `X-Forwarded-For` to the client address. The server only believes that header when the connection comes from loopback, and then takes the last address in it, so a client cannot choose its own rate limit bucket. Turn request buffering off and raise the body limit as well, otherwise larger pushes over HTTPS will be cut short. The nginx site in the deploy directory does all of this and adds a coarse request limit of its own. |
| 57 | 61 | |
| 58 | 62 | ``` |
| 59 | 63 | Match User git |
| @@ -68,19 +72,26 @@ Match User git |
| 68 | 72 | The `keys` command runs as root and opens the database read only and answers sshd with a forced command of `gitpocket shell <user>`. That in turn runs as the login user, checks what `SSH_ORIGINAL_COMMAND` is actually asking for against the database, and then execs straight into `git upload-pack` or `git receive-pack` so nothing is left running afterwards. |
| 69 | 73 | |
| 70 | 74 | ``` |
| 71 | | -sudo -u git gitpocket -root /srv/gitpocket invite |
| 75 | +sudo -u git gitpocket -root /srv/gitpocket admin you |
| 76 | +sudo -u git gitpocket -root /srv/gitpocket registration invite |
| 72 | 77 | ``` |
| 73 | 78 | |
| 74 | | -If you would rather let anyone sign up then `serve -open` skips the invite step entirely. `gitpocket user <name>` creates an account from the command line with the password read from stdin, and `-ssh-user` changes the account name that appears in clone URLs if you did not call yours `git`. The header shows `-name`, which defaults to gitpocket, while `-host` stays the hostname used in clone URLs and feeds. |
| 79 | +Registration starts open. `gitpocket registration` prints or sets the mode (open, invite or closed), `gitpocket invite` prints a single use code, and `gitpocket user <name>` creates an account from the command line with the password read from stdin. `-ssh-user` changes the account name that appears in clone URLs if you did not call yours `git`. The header shows `-name`, which defaults to gitpocket, while `-host` stays the hostname used in clone URLs and feeds. |
| 75 | 80 | |
| 76 | | -Mail is configured in `smtp.json` under the root directory and is read once at startup. Plain SMTP takes `host`, `port`, `user`, `password`, `from` and `starttls`, but many VPS providers block every outbound submission port, so `api` and `key` post each message as JSON to a Resend compatible HTTPS endpoint instead. |
| 81 | +Mail is optional and nothing needs it to sign up. Without it password resets go through `gitpocket passwd`. It is configured in `smtp.json` under the root directory and read once at startup. Plain SMTP takes `host`, `port`, `user`, `password`, `from` and `starttls`, but many VPS providers block every outbound submission port, so `api` and `key` post each message as JSON to a Resend compatible HTTPS endpoint instead. |
| 77 | 82 | |
| 78 | 83 | ``` |
| 79 | 84 | {"api": "https://api.resend.com/emails", "key": "re_...", "from": "gitpocket <noreply@git.example.org>"} |
| 80 | 85 | sudo -u git gitpocket -root /srv/gitpocket mail you@example.org |
| 81 | 86 | ``` |
| 82 | 87 | |
| 83 | | -`gitpocket feature ~you/a ~you/b` chooses the projects on the logged out front page in that order, `-~you/a` drops one again, and with none chosen the page falls back to the most recently pushed. `gitpocket backup file` writes a consistent copy of the database with `vacuum into`, which is safe to run from a timer while the server is live. |
| 88 | +`gitpocket feature ~you/a ~you/b` chooses the projects on the logged out front page in that order, `-~you/a` drops one again, and with none chosen the page falls back to the most recently pushed. `gitpocket backup <dir>` writes a dated copy of the database with `vacuum into`, keeps the last fourteen, and writes a `git bundle --all` for every repository whose refs changed since its last bundle, so a nightly run touches only what moved. Restoring a repository is a `git clone --mirror` of its bundle into the repos directory. |
| 89 | + |
| 90 | +``` |
| 91 | +go test ./... |
| 92 | +``` |
| 93 | + |
| 94 | +The tests drive the real handler through `httptest`. They sign up under each registration mode, try a cross site post, poke at someone else's private repository and an organisation you are not in, follow a moved repository, merge in a throwaway bare repo, and feed the Markdown renderer `<script>`, `onerror` and `javascript:` payloads to check none survive. |
| 84 | 95 | |
| 85 | 96 | ## Use |
| 86 | 97 | |
~james SHA256:KlzJ0P0gvJcnOYpaxD5yCUw52ImHttTKRjVAEnnYnKg