0.1.93 — GitHub Sync push pipeline (JWT signing, repo picker, restore CLI)

Closes the experimental GitHub data-sync work scaffolded in 0.1.90. The push half is now end-to-end functional once the operator provisions the GitHub App env vars; the restore half ships as a stdlib-only CLI.

Landed across four scoped commits on main:

  • 6bae133 — backend deps + JWT signer + repo-list endpoint + tests.
  • 024bd85 — frontend repo-picker UI + per-app client wiring.
  • c332a27 — backend/scripts/github_sync_restore.py CLI.
  • (this round) — bookkeeping: VERSION + round log + TODO + docs.

Backend

  • backend/requirements.txt and backend/requirements-render.txt add PyJWT>=2.8,<3 and cryptography>=42,<46. The free-tier render build now ships with both since the data-sync feature is on the same backend.
  • creators/services/github_sync.py:
    • _normalize_pem converts the operator's single-line PEM (with literal \n escapes) back to the multi-line form cryptography expects. Idempotent for already-multi-line input.
    • _build_app_jwt builds an RS256 JWT with iat = now-60, exp = now+9min, iss = settings.GITHUB_DATA_SYNC_APP_CLIENT_ID. Per the GitHub App spec, exp must be ≤ 10 min in the future.
    • _refresh_installation_token now signs the JWT, POSTs /app/installations/<id>/access_tokens, persists the returned token + expires_at on the GithubIntegration row, and returns the token. Errors keep the AGENTS.md §1.8 three-component shape; the body of any 4xx response is truncated to 200 bytes so logs don't leak the JWT or PEM.
  • creators/api.py adds GithubSyncReposApiView (auth required). GET /api/v1/integrations/github/repos/ calls /installation/repositories with the installation token via _ensure_token, paginates per_page=100 (defensive 5,000-repo cap), and returns {repositories: [{full_name, default_branch, private}]}.
  • notechondria/api_urls.py wires the new endpoint.
  • creators/tests.py adds a GithubSyncTests block (10 cases): auth gate, status before/after callback, callback upsert by installation_id, disconnect, JWT round-trip against a freshly generated test RSA keypair, escaped-PEM normalize path, materialize produces every expected path for a seeded creator, and the disconnected-state error shape for /repos/ + /push/. All 10 pass under settings_test.

Frontend (shared widget + per-app wiring)

  • notechondria_shared/lib/src/components/mcp_skill_section.dart: GithubSyncExperimentalCard rewritten as a stateful widget with three states.
    • No callbacks (signed out): passive description card so anonymous users still see the feature.
    • Disconnected: "Install Notechondria GitHub App" button that redirects to the install URL via url_strategy.browserRedirect.
    • Connected: shows the account_login, fetches the install's repositories via onListRepos, renders a DropdownButtonFormField of full_name choices, persists the chosen repo via onConnect, exposes "Push now" (surfaces commit_sha) and "Disconnect".
    • All errors are surfaced via SnackBar with the AGENTS.md §1.8 shape (consequence + module/process + cause).
  • Each app (editor_app, portal_app, planner_app) adds five methods to its NotechondriaClient interface and HTTP impl: githubSyncStatus, githubSyncRepos, githubSyncCallback, githubSyncPush, githubSyncDisconnect. Editor's split into client.dart interface + http_client.dart impl; portal/planner keep both in client.dart per their existing pattern.
  • _SettingsPage gains a githubSyncCardBuilder prop on each app. app_shell (or editor's build_helpers.dart) builds the card with token-bound callbacks when authenticated; null when signed out. The settings pages fall back to the no-callbacks card so the experience degrades gracefully.

Restore CLI

  • backend/scripts/github_sync_restore.py walks a cloned data-sync repo and POSTs each piece back via the existing public REST API:
    • PATCH /api/v1/settings/ with the union of profile/creator.json, profile/settings.json, and profile/skill.md.
    • POST /api/v1/courses/ for each courses/<slug>.json, pre-fetching /courses/ first so reruns don't duplicate slugs.
    • POST /api/v1/notes/ for each notes/<uuid>.md + notes/<uuid>.meta.json pair. Strips YAML frontmatter, sends the body as content, the sidecar JSON as metadata_json + custom_meta, and uses the original client_draft_id (or restore:<uuid> when missing) so reruns are idempotent via the existing find-or-update path on /notes/.
    • POST /api/v1/planner-events/ and /api/v1/calendar-feeds/ for planner/events.json + planner/feeds.json rows.
  • Stdlib-only (urllib, json, argparse) so operators can run it in a recovery shell. --dry-run prints requests without contacting the server; --verbose prints per-section counters; default output is a JSON summary line.
  • Smoke-tested against a synthetic fixture in dry-run; settings, courses, notes (with frontmatter parsed), and planner events all fire the expected requests.

Docs

  • docs/integrations/github-sync.md known-gaps section updated: the JWT signing scaffold is gone, the repo picker is real, and a CLI restore script ships at backend/scripts/github_sync_restore.py. Static-asset re-bundling remains the open gap (avatars, attachments, cover images stay on the original CDN).
  • docs/TODO.md marks the "Experimental GitHub Sync — wire the actual push path" carryover done.
  • Root README.md + docs/readme.md adjust the data-sync section to drop the "wire-up gated" hedge — the push path is functional once env vars are set.

Operator action

Drop these into the backend env (.env, Render dashboard, Northflank service env, etc.) and rebuild:

GITHUB_DATA_SYNC_APP_NAME=notechondria-data-sync
GITHUB_DATA_SYNC_APP_CLIENT_ID=Iv1.<...>
GITHUB_DATA_SYNC_APP_CLIENT_SECRET=<from GitHub App settings>
GITHUB_DATA_SYNC_APP_PRIVATE_KEY=<single-line PEM with \n escapes>
GITHUB_DATA_SYNC_APP_INSTALL_URL=https://github.com/apps/notechondria-data-sync/installations/new

The GitHub App must request the following permissions:

  • Repository: Contents read+write (single repo per install).
  • Metadata: read (default).

Run migrations after deploy if the 0.1.90 schema isn't already on the server:

python manage.py migrate creators
python manage.py migrate notes

Verification

  • python manage.py test creators.tests.GithubSyncTests — 10/10 pass.
  • flutter analyze — clean across editor_app, portal_app, planner_app. Only pre-existing unused-* warnings remain.
  • Restore CLI dry-run smoke: synthetic fixture with profile + courses
    • notes (frontmatter) + planner events; emits the expected PATCH + POST sequence.

Carryover

  • Static-asset re-bundling. Avatars, attachments, and cover images are referenced by URL only; restoring into a fresh server with no original CDN access leaves those URLs broken. Next round should add an opt-in --include-assets flag that fetches and re-uploads each asset before re-creating its parent record.
  • Conflict resolution. The Contents API PUTs we use overwrite the remote copy. Multi-device users who edit on two devices between syncs can lose edits. Next round should fetch the existing blob, diff, and surface a "remote changed" warning before overwriting.