| @@ -1,113 +1,53 @@ |
| 1 | 1 | # gitpocket |
| 2 | 2 | |
| 3 | | -Small and lightweight git server for self hosting. Repositories are served over SSH and HTTPS, trees and logs and diffs and READMEs are rendered on the server rather than in the browser, and pull requests are handled without a working tree ever being checked out. The only JavaScript on the reading side is KaTeX, which is served only when a page contains mathematics. |
| 3 | +gitpocket is a self hosted git server in one static Go binary, serving repositories over SSH and HTTPS with a web interface rendered on the server. It idles at about 25 MB resident, so it fits comfortably on a 512 MB VPS next to the git processes it starts. |
| 4 | 4 | |
| 5 | | -## Features |
| 6 | | - |
| 7 | | -- Repos sit under `~user/name` and can be public or unlisted or private. Each one takes up to three tags, and the old `/projects` address now redirects to the search page with the matching tag qualifier |
| 8 | | -- Tree and blob and log and commit and refs views, plus a case insensitive search across file contents at any ref backed by `git grep`. Chroma does the highlighting and goldmark rendering the Markdown before bluemonday cleans it. Anything in `$x$` or `$$x$$` gets typeset in the browser |
| 9 | | -- The last commit shown against every tree entry comes out of one `git log --name-only` walk per directory instead of a separate process per file |
| 10 | | -- Profiles have a display name and a Markdown bio along with location and workplace and website, plus an avatar that you crop in the browser and the server then resamples down to a 256px PNG. Push a README.md to a public repository with your own name, `~you/you`, and it shows at the top of your profile |
| 11 | | -- Forks and pull requests from either a branch or a fork. Discussion comments and inline review comments on any line of the diff are Markdown, review threads can be resolved and go to an outdated section when the line leaves the diff, and a request can be closed or reopened or brought up to date from its source at any point |
| 12 | | -- Three merge strategies, a merge commit with an editable message, a squash authored by the request author, or a rebase that replays each commit onto the target. All three are built from `merge-tree` and `commit-tree` without a working tree and fail cleanly on conflict |
| 13 | | -- Collaborators per repository who can push, read it when private and merge pull requests, while settings and deletion stay with the owner |
| 14 | | -- Organisations, which are accounts that cannot log in and whose repositories every member owns. Anyone can create one from the dashboard, and members edit its picture, bio and membership from its settings page, so a project can outlive the person who started it |
| 15 | | -- A per owner disk quota that the push itself enforces. Each push over either transport runs `receive-pack` with `receive.maxInputSize` set to whatever the owner has left, so git refuses an oversized pack before writing it |
| 16 | | -- Signature verification for both SSH and PGP. Register the key in settings, sign commits with it, and they show `Verified ~you`. PGP keys are imported into a server keyring and matched by primary fingerprint, SSH keys through an allowed signers file matched by fingerprint. Keys are public at `~you.keys` and `~you.pgp` |
| 17 | | -- Issues per repository with Markdown bodies and comments, open and closed states, and an open count in the tab strip |
| 18 | | -- A site wide search at `/search` over repositories and people, ranked so an exact name beats a prefix beats a substring. Qualifiers narrow it the way they do elsewhere, `language:ocaml` (case insensitive, quoted when the name has a space), `user:name` and `tag:sel4`, and the language facets write the qualifier into the query box so the filter survives the next search. Each repository's languages come from the byte size of its files at HEAD by extension, skipping minified and vendored paths, and are cached against the HEAD hash so the language filter and the breakdown bar on the summary page cost one `git ls-tree` per push |
| 19 | | -- Stars, so a profile shows what someone found worth keeping, and a star count next to every repository title |
| 20 | | -- A year of activity on each profile, drawn server side as a grid with no script, with a tab for the past twelve months and one for each calendar year that has commits. Hovering or focusing a day shows its count in a CSS tooltip. It counts commits on every branch of every public repository whose author email matches the account's verified address, so work on other people's projects counts and nothing is attributed by name alone |
| 21 | | -- Blame, a compare page for any two refs, and tar.gz and zip downloads of any ref. The tar.gz is `git archive --format=tar` passed through a gzip writer with a zeroed header and a fixed level, so the same commit always produces the same bytes and `archive/<ref>.tar.gz.sha256` publishes the checksum that a package recipe can pin |
| 22 | | -- A branch switcher on every tree and file view, and a notice on the summary and pull request pages whenever a branch with commits the default branch lacks was pushed in the last three days, with a button that opens a pull request from it |
| 23 | | -- Every pushed revision of a pull request is kept under `refs/pull/<n>/v<k>`, and the request page links a `git range-diff` between any two of them, so a reviewer sees what a force push actually changed in each commit instead of rereading the whole series |
| 24 | | -- History search on the log page through git's pickaxe, either commits that change how often a string occurs (`-S`) or commits whose diff has a line matching a regex (`-G`), which answers when a line arrived or vanished without cloning |
| 25 | | -- Atom feeds of commits per branch, of issues and of pull requests per repository, and of every public commit by a user at `~you/feed`, which stand in for notifications when mail is not configured. Each page names its feeds in a `<link rel="alternate">`, so a reader finds them from the ordinary page URL |
| 26 | | -- Rename, transfer to another user and archive from the repository settings. The old path keeps working as a permanent redirect for the web, HTTPS clones and SSH, provided the visitor can read the repository at its new home, and an archived repository refuses pushes, pull requests and comments but stays readable |
| 27 | | -- An optional email address on each account, used for password resets and for notifications about pull requests, reviews and issues on your repositories when mail is configured. Every account gives one at sign up, and nobody has to confirm it before using their account |
| 28 | | -- TOTP two factor authentication with a QR code for the authenticator app and eight single use recovery codes, which is what a password alone cannot give you on something that accepts pushes |
| 29 | | -- SSH keys and access tokens live in settings. HTTPS auth is plain basic auth with a token or your password |
| 30 | | -- Relative links and images in a README resolve against the repository, so a `docs/diagram.png` reference renders rather than breaking |
| 31 | | -- Registration is open, invite only or closed, and an administrator switches between the three on the admin page or with `gitpocket registration`. Signing up needs only a name and password, and each address can create a handful of accounts every few minutes |
| 32 | | -- `/admin` exists for accounts you mark with `gitpocket admin <name>` and 404s for everyone else. Its overview counts accounts, repositories and disk use, and separate sections handle people (ban, unban and grant admin), sign up (registration mode and invite links), the front page order and storage with a usage bar per owner |
| 33 | | - |
| 34 | | -## Design |
| 35 | | - |
| 36 | | -Every read of git data is a call out to the plumbing with a thirty second timeout, which in practice means `ls-tree -l` for trees and `log` with a NUL separated format for history and `diff-tree -p --root -M` for commit pages and `cat-file` for blobs. Nothing is parsed out of packfiles in process, so a large repository costs exactly what git says it costs. A buffered call that tries to return more than 16 MiB gets killed. Only twice the core count may run together, or four on a single core box, and raw files and archives stream two at a time. In production the server idles at about 35 MB resident, so most of a 512 MB VPS is left for git. |
| 37 | | - |
| 38 | | -A little is remembered between requests, though deleting all of it loses nothing. Language bars and the history behind the activity graph are stored against the mtime of the refs, so after a push one `stat` notices and before one nothing is read at all. That history holds sixteen bytes a commit (a hash prefix, the day and which author address it was), and the list of people that search walks is refreshed at most twice a minute. |
| 39 | | - |
| 40 | | -Each client address gets a token bucket of five requests a second with a burst of 120. Search, blame, downloads, pickaxe and login draw from a second, much slower bucket because they are the pages that start real work in git. Pages are sent with a content security policy that only runs scripts served by the site, forbids framing and keeps the referrer inside the origin. Raw files go out as plain text unless they are images, audio or video, and those get a sandbox of their own, so a committed HTML or SVG file cannot run anything in a visitor's session. Go's cross origin protection rejects form posts from other sites before any handler sees them. |
| 41 | | - |
| 42 | | -When a pull request is opened or updated the source branch gets fetched into `refs/pull/N/head` of the target repository, and from that point on the diff and the commit list and the question of whether it merges cleanly all come from `merge-base` and `diff` and `merge-tree --write-tree` inside that single repo. The merge itself is `commit-tree` with both parents followed by `update-ref` carrying the old value it expects to find, so if someone pushes to the branch while you are clicking the button the merge simply fails rather than silently overwriting them. Conflicts are reported back to you and left for you to sort out locally. |
| 43 | | - |
| 44 | | -SQLite comes from the modernc port, which is pure Go, so a `CGO_ENABLED=0` build produces one static binary that will run anywhere there happens to be a system git next to it. |
| 45 | | - |
| 46 | | -## Install |
| 5 | +## Quick start |
| 47 | 6 | |
| 48 | 7 | ``` |
| 49 | | -CGO_ENABLED=0 go build -trimpath -ldflags='-s -w' -o gitpocket . |
| 8 | +git clone https://gitpocket.io/~gitpocket/gitpocket && cd gitpocket |
| 50 | 9 | useradd -r -m -d /srv/gitpocket -s /bin/sh git |
| 51 | | -install -m 755 gitpocket /usr/local/bin/gitpocket |
| 52 | | -``` |
| 53 | | - |
| 54 | | -Copy the systemd units and nginx site from the deploy directory and change the host name in them. The service unit sets `GOMEMLIMIT` to 192 MiB and gives systemd a hard 512 MB ceiling covering the server and every git it starts, and the backup timer fires once a night. |
| 55 | | - |
| 56 | | -``` |
| 57 | | -install -m 644 deploy/gitpocket.service deploy/gitpocket-backup.service deploy/gitpocket-backup.timer /etc/systemd/system/ |
| 58 | | -systemctl enable --now gitpocket gitpocket-backup.timer |
| 10 | +printf 'GITPOCKET_HOST=git.example.org\nGITPOCKET_UID=%s\nGITPOCKET_GID=%s\n' $(id -u git) $(id -g git) > deploy/.env |
| 11 | +docker compose -f deploy/compose.yaml up -d |
| 59 | 12 | ``` |
| 60 | 13 | |
| 61 | | -Put a reverse proxy with TLS in front of it. The proxy needs to forward `Host` and `X-Forwarded-Proto` and should set `X-Forwarded-For` to the client address. The server only believes that header when the connection comes from loopback, and then takes the last address in it, so a client cannot choose its own rate limit bucket. Turn request buffering off and raise the body limit as well, otherwise larger pushes over HTTPS will be cut short. The nginx site in the deploy directory does all of this and adds a coarse request limit of its own. |
| 14 | +The web interface is then on `127.0.0.1:3200` and wants a reverse proxy with TLS in front of it. Sign up, then make yourself an administrator with `docker exec gitpocket gitpocket admin you`. [INSTALL.md](INSTALL.md) covers the Docker and binary installs, nginx, SSH, mail, backups and upgrades in order. |
| 62 | 15 | |
| 63 | | -``` |
| 64 | | -Match User git |
| 65 | | - AuthenticationMethods publickey |
| 66 | | - AuthorizedKeysFile none |
| 67 | | - AuthorizedKeysCommand /usr/local/bin/gitpocket -root /srv/gitpocket keys %u %h %t %k |
| 68 | | - AuthorizedKeysCommandUser root |
| 69 | | - PermitTTY no |
| 70 | | - DisableForwarding yes |
| 71 | | -``` |
| 72 | | - |
| 73 | | -The `keys` command runs as root and opens the database read only and answers sshd with a forced command of `gitpocket shell <user>`. That in turn runs as the login user, checks what `SSH_ORIGINAL_COMMAND` is actually asking for against the database, and then execs straight into `git upload-pack` or `git receive-pack` so nothing is left running afterwards. |
| 74 | | - |
| 75 | | -``` |
| 76 | | -sudo -u git gitpocket -root /srv/gitpocket admin you |
| 77 | | -sudo -u git gitpocket -root /srv/gitpocket registration invite |
| 78 | | -``` |
| 79 | | - |
| 80 | | -Registration starts open. `gitpocket registration` prints or sets the mode (open, invite or closed), `gitpocket invite` prints a single use code, and `gitpocket user <name>` creates an account from the command line with the password read from stdin. `-ssh-user` changes the account name that appears in clone URLs if you did not call yours `git`. The header shows `-name`, which defaults to gitpocket, while `-host` stays the hostname used in clone URLs and feeds. A `logo.png` placed in the root directory replaces the header wordmark and an `icon.png` there replaces the favicon, both read once at startup. |
| 16 | +## Features |
| 81 | 17 | |
| 82 | | -Mail is optional and nothing needs it to sign up. Without it password resets go through `gitpocket passwd`. It is configured in `smtp.json` under the root directory and read once at startup. Plain SMTP takes `host`, `port`, `user`, `password`, `from` and `starttls`, but many VPS providers block every outbound submission port, so `api` and `key` post each message as JSON to a Resend compatible HTTPS endpoint instead. |
| 18 | +- Public, unlisted and private repositories under `~user/name`, with tree, log, blame, compare, history search and tar.gz or zip downloads of any ref |
| 19 | +- Branches, a branch switcher on every file view and a notice when a branch with new commits was pushed, linking straight to a new pull request |
| 20 | +- Pull requests from a branch or a fork with inline review, range diffs between force pushes, and merge, squash or rebase without a working tree |
| 21 | +- Organisations, collaborators, issues, labels, stars, forks, branch protection and per owner disk quotas enforced during the push |
| 22 | +- Profiles with a pinned repository list, an activity graph and a README from your `~you/you` repository |
| 23 | +- SSH and PGP commit verification, TOTP, access tokens and Atom feeds for commits, issues and pull requests |
| 24 | +- Site search over repositories and people with language facets |
| 25 | +- Open, invite only or closed registration and an `/admin` page for accounts, invites, the front page and storage |
| 83 | 26 | |
| 84 | | -``` |
| 85 | | -{"api": "https://api.resend.com/emails", "key": "re_...", "from": "gitpocket <noreply@git.example.org>"} |
| 86 | | -sudo -u git gitpocket -root /srv/gitpocket mail you@example.org |
| 87 | | -``` |
| 27 | +There is no wiki and no CI, and neither is planned. |
| 88 | 28 | |
| 89 | | -`gitpocket feature ~you/a ~you/b` chooses the projects on the logged out front page in that order, `-~you/a` drops one again, and with none chosen the page falls back to the most recently pushed. `gitpocket backup <dir>` writes a dated copy of the database with `vacuum into`, keeps the last fourteen, and writes a `git bundle --all` for every repository whose refs changed since its last bundle, so a nightly run touches only what moved. Restoring a repository is a `git clone --mirror` of its bundle into the repos directory. |
| 29 | +## Design |
| 90 | 30 | |
| 91 | | -``` |
| 92 | | -go test ./... |
| 93 | | -``` |
| 31 | +Every read of git data calls the plumbing with a thirty second timeout, so a large repository costs exactly what git says it costs and nothing is parsed out of packfiles in process. Ref tables, trees, summaries and languages are cached against the mtime of the refs, which means one `stat` after a push invalidates them and deleting the cache loses nothing. |
| 94 | 32 | |
| 95 | | -The tests drive the real handler through `httptest`. They sign up under each registration mode, try a cross site post, poke at someone else's private repository and an organisation you are not in, follow a moved repository, merge in a throwaway bare repo, and feed the Markdown renderer `<script>`, `onerror` and `javascript:` payloads to check none survive. |
| 33 | +Pull requests fetch the source into `refs/pull/N/head` of the target, and merges are `merge-tree` and `commit-tree` followed by an `update-ref` that names the old value, so a push racing the merge button makes the merge fail instead of being overwritten. SQLite comes from the pure Go modernc port, which is why a `CGO_ENABLED=0` build needs nothing beside a system git. |
| 96 | 34 | |
| 97 | | -## Use |
| 35 | +## Usage |
| 98 | 36 | |
| 99 | 37 | ``` |
| 100 | 38 | git clone https://git.example.org/~you/repo |
| 101 | 39 | git clone git@git.example.org:~you/repo |
| 102 | 40 | ``` |
| 103 | 41 | |
| 104 | | -The `.git` suffix works but is not required. Pushing over SSH needs a key added under settings, while pushing over HTTPS takes a token or your password as the basic auth secret. |
| 42 | +SSH pushes need a key added under settings, and HTTPS pushes take an access token or your password. To contribute to something you do not own, fork it, push a branch to the fork and open a pull request from that branch. |
| 105 | 43 | |
| 106 | | -To send changes to something you do not own, fork it from the button beside the title and push a branch to your fork, then open a pull request from the pull requests tab and choose that branch as the source. The owner merges it from the sidebar when they are happy with it. |
| 44 | +## Development |
| 107 | 45 | |
| 108 | | -## Limits |
| 46 | +``` |
| 47 | +go test ./... |
| 48 | +``` |
| 109 | 49 | |
| 110 | | -Raw files stream straight out of `cat-file` at any size, but the rendered view stops at 4 MiB and syntax highlighting at 512 KiB, beyond which a file is shown plain or offered as raw only. The quota limit counts the incoming pack, so a push that unpacks into loose objects can land slightly over it. Activity and search look at the 500 most recently created public repositories. There is no wiki and no CI, and there are no plans for either. |
| 50 | +The tests drive the real handler through `httptest`, covering each registration mode, cross site posts, private repositories, moved repositories, every merge strategy and Markdown injection payloads. |
| 111 | 51 | |
| 112 | 52 | ## Licence |
| 113 | 53 | |