mirror of
https://github.com/andwati/gitea-mirror-sync.git
synced 2026-10-18 03:12:53 +00:00
Trim credentials before use; fix redact on awkward tokens
A GITEA_TOKEN secret stored with a trailing newline makes curl emit `Authorization: token <value>\n`. The bare LF terminates the header block early, so Content-Type and everything after it is parsed as request body and Gitea rejects the call with 422 "Empty Content-Type". GETs were unaffected because nothing they need follows the auth header, which is why this presented as a proxy stripping the header. - trim CR/LF and surrounding whitespace off GITHUB_TOKEN, GITEA_TOKEN, GITEA_URL and GITEA_OWNER, warning when a value needed it - rebuild redact() on bash substitution; the previous sed s/// blew up with "unterminated `s' command" on a token containing a newline - add AGENTS.md / CLAUDE.md documenting the run flow and these traps
This commit is contained in:
@@ -0,0 +1,98 @@
|
||||
# AGENTS.md
|
||||
|
||||
Guidance for coding agents working in this repository.
|
||||
|
||||
## What this is
|
||||
|
||||
One bash script, `mirror-github-to-gitea.sh`, that makes every GitHub repo owned by the
|
||||
token's user exist as a **pull mirror** on a Gitea instance, plus the GitHub Actions
|
||||
workflow (`.github/workflows/mirror-to-gitea.yml`) that runs it daily at 00:00 UTC.
|
||||
There is no build, no test suite, and no package manager. Dependencies are `bash`,
|
||||
`curl`, and `jq`.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# syntax check (the only "build" step)
|
||||
bash -n mirror-github-to-gitea.sh
|
||||
shellcheck mirror-github-to-gitea.sh # if installed
|
||||
|
||||
# full run — needs all four env vars
|
||||
GITHUB_TOKEN=... GITEA_URL=https://gitea.example.com GITEA_TOKEN=... GITEA_OWNER=... \
|
||||
./mirror-github-to-gitea.sh
|
||||
|
||||
# smoke-test a single repo instead of all of them (do this before any full run)
|
||||
ONLY=blog ./mirror-github-to-gitea.sh
|
||||
|
||||
# dump request/response headers for every Gitea write, tokens redacted
|
||||
DEBUG_HTTP=1 ONLY=blog ./mirror-github-to-gitea.sh
|
||||
|
||||
# trigger the workflow, optionally with header debugging on
|
||||
gh workflow run mirror-to-gitea.yml -f debug_http=true
|
||||
gh run watch
|
||||
```
|
||||
|
||||
## Environment contract
|
||||
|
||||
| Var | Source in CI | Notes |
|
||||
| --- | --- | --- |
|
||||
| `GITHUB_TOKEN` | `secrets.GH_PAT` | PAT with read access incl. private repos; also embedded in the migrate payload so Gitea can pull |
|
||||
| `GITEA_URL` | `secrets.GITEA_URL` | trailing slash is stripped — `//api/v1/...` gets mangled by some proxies |
|
||||
| `GITEA_TOKEN` | `secrets.GITEA_TOKEN` | Gitea token, repo scope |
|
||||
| `GITEA_OWNER` | `secrets.GITEA_OWNER` | defaults to `andwati` |
|
||||
| `MIRROR_INTERVAL` | — | Gitea sync interval, default `8h0m0s` |
|
||||
| `ONLY`, `DEBUG_HTTP` | `inputs.debug_http` | testing knobs |
|
||||
|
||||
`GITHUB_USER` is hardcoded in the script while the Gitea side comes from env, so the
|
||||
two accounts are configured differently — don't assume they're the same value.
|
||||
|
||||
## How a run works
|
||||
|
||||
1. Page through `GET /user/repos?affiliation=owner` (not `/users/{user}/repos`, which
|
||||
is public-only) and accumulate the list with `jq -s`.
|
||||
2. Per repo, `GET /api/v1/repos/{owner}/{name}` on Gitea to decide what to do:
|
||||
- **200** → compare `private` on both sides; `PATCH` the Gitea repo if they differ,
|
||||
otherwise skip.
|
||||
- **anything else** → treated as "not mirrored yet", so `POST /api/v1/repos/migrate`
|
||||
with `mirror: true`. A transient 5xx therefore looks like "missing" and produces a
|
||||
failed create rather than a retry.
|
||||
|
||||
The script only creates and adjusts visibility. It never deletes, renames, or unmirrors,
|
||||
and repos that exist on Gitea but not GitHub are left alone.
|
||||
|
||||
## Gotchas worth knowing before editing
|
||||
|
||||
- **Credentials are trimmed at startup, and that is load-bearing.** A GitHub secret
|
||||
saved with a trailing newline makes curl write `Authorization: token <value>\n`; the
|
||||
bare LF ends the header block early, so every header after it — `Content-Type`
|
||||
included — is parsed as body and Gitea answers `422 "[]: Empty Content-Type"`. `GET`s
|
||||
are unaffected because nothing they rely on follows the auth header, which is why this
|
||||
looked like a proxy problem at first. The script warns and trims; the real fix is
|
||||
re-saving the secret without the newline.
|
||||
- **`redact()` uses bash `${var//old/new}`, not `sed`.** Interpolating a secret into a
|
||||
`s///` expression breaks the moment the secret contains a slash or a newline.
|
||||
- **All body-bearing Gitea calls go through `gitea_write`.** It exists because Gitea
|
||||
answers `422 "[]: Empty Content-Type"` when a reverse proxy drops the header in front
|
||||
of it — which happened from GitHub runners but not from a local network. The helper
|
||||
sends the body with `--data-binary @file`, disables `Expect: 100-continue`, sets an
|
||||
explicit charset, and retries once over `--http1.1` when that error comes back. Don't
|
||||
replace it with an inline `curl -d`.
|
||||
- **`gitea_write` echoes the HTTP status on stdout and callers capture it with `$(...)`.**
|
||||
Any progress or diagnostic output added inside it must be redirected to `>&2`, or it
|
||||
gets swallowed into the status variable.
|
||||
- **The migrate payload contains the PAT.** Response bodies and header traces are piped
|
||||
through `redact()` before printing; keep any new output on that path. Bodies live in a
|
||||
`mktemp -d` cleaned by a trap, and are passed to curl by file rather than on the
|
||||
command line so tokens stay out of the process list.
|
||||
- The workflow's preflight step POSTs a harmless body to `/api/v1/markdown` over both
|
||||
HTTP/2 and HTTP/1.1. Reading its two status codes is the fastest way to tell a broken
|
||||
network path from a broken payload.
|
||||
- The script relies on `mirror-github-to-gitea.sh` being committed mode `100755`; the
|
||||
workflow invokes it directly with no `chmod`.
|
||||
|
||||
## Commits and PRs
|
||||
|
||||
Do not add self-attribution. No `Co-Authored-By: Claude ...` trailer on commits, and no
|
||||
"Generated with Claude Code" line in PR bodies — the message ends at its last
|
||||
substantive line. If a trailer slips through, amend and force-push with
|
||||
`--force-with-lease`.
|
||||
Reference in New Issue
Block a user