0.1.94 — GitHub Sync static-asset bundling (push + restore round-trip)
Closes the static-asset gap left open in 0.1.93. The data-sync feature is now genuinely server-loss-survivable for accounts whose total asset size fits inside the per-push budget — a user can push with assets, clone the resulting repo, and restore the full account (text + binary) onto a fresh server.
Landed across three scoped commits on main:
- 406fd2e — push side: opt-in inline of avatar / cover / attachment bytes with size guards + 3 new tests.
- d719c4c — restore CLI
--include-assetsflag + frontend toggle. - (this commit) — bookkeeping: VERSION + round log + TODO + docs.
Push side
creators/services/github_sync.py:- Module caps
ASSET_FILE_MAX_BYTES = 50 MBandASSET_TOTAL_MAX_BYTES = 200 MB. Per-file cap stays under GitHub's 100 MB Contents API blob limit; total cap keeps a single push from blowing the user's repo size budget. _read_field_bytesand_ext_from_namehelpers. Field reads are lossy by design — a missing avatar shouldn't kill a push._asset_files(creator, *, skipped)writes:assets/avatar.<ext>(single profile picture)assets/notes/<note-uuid>/cover.<ext>assets/notes/<note-uuid>/attachments/<attachment-uuid>.<ext>Files past either cap are recorded inskipped; the parent record's URL reference in the JSON sidecar is unchanged.
materialize(creator, *, include_assets=False)gains the flag. The manifest now carriesinclude_assets+skipped_assetsso the restore CLI knows what to look for.push_user_data(creator, *, include_assets=False)threads the flag through; log line records it for ops audit.
- Module caps
creators/api.pyGithubSyncPushApiViewacceptsinclude_assetsfrom query string or JSON body (truthy strings = 1/true/yes/on); response echoes the flag.- 3 new tests in
creators.tests.GithubSyncTests:test_materialize_skips_assets_by_default— default export contains zeroassets/paths.test_materialize_include_assets_inlines_avatar_and_cover— flag flips manifest + writes both avatar and cover bytes.test_materialize_skips_assets_over_per_file_cap— patched cap proves the size guard records the skip inmanifest.skipped_assets. All 13 GithubSyncTests pass.
Restore side
backend/scripts/github_sync_restore.py:- New
--include-assetsflag. RestoreClient.upload(method, path, *, field, filename, content, extra_fields)builds multipart/form-data with stdlib only so the script stays zero-dep._restore_notesnow populates auuid_to_idmap from each POST /notes/ response. The asset phase keys off it to target the right note's cover / attachment endpoints._restore_assetswalksassets/:PATCH /api/v1/settings/with multipartavatar.POST /api/v1/notes/<id>/cover/with multipartcover.POST /api/v1/notes/<id>/attachments/with multipartfile. Notes whose UUID wasn't restored in this run (already on server, never appeared) are tallied asskipped.
- Summary JSON gains an
assets: {avatar, cover, attachment, skipped}block when the flag is set.
- New
Frontend
notechondria_shared/lib/src/components/mcp_skill_section.dartGithubSyncExperimentalCardadds an "Include assets" SwitchListTile in the connected state. The flag is forwarded toonPushNowasincludeAssets.onPushNowcallback signature is nowFuture<Map<String, dynamic>> Function({bool includeAssets}).- editor / portal / planner client interfaces + impls update
githubSyncPush(token, {bool includeAssets = false})and POST{include_assets: <bool>}in the body. - Per-app
app_shell(and editor'sbuild_helpers.dart) bind the new callback shape.
Operator notes
- Once
0.1.94is deployed, users see the new "Include assets" toggle automatically. No new env vars; the existingGITHUB_DATA_SYNC_APP_*block from 0.1.90 is sufficient. - Asset reads pull from whatever
DEFAULT_FILE_STORAGEresolves to (local filesystem in dev, S3-compatible storage in prod viadjango-storages). The Cloudflare R2 path is already covered by the existingboto3setup — no extra wiring. - Per-push caps are intentionally conservative. If your power users
need higher limits, raise
ASSET_FILE_MAX_BYTES/ASSET_TOTAL_MAX_BYTESinbackend/creators/services/github_sync.pyand rebuild; both are module-level constants for easy patching.
Verification
python manage.py test creators.tests.GithubSyncTests— 13/13 pass under settings_test.flutter analyze— clean acrosseditor_app,portal_app,planner_app. No new warnings; pre-existing unused-* lints unchanged.- Restore CLI smoke: synthetic fixture exercises avatar + cover + attachment in the assets tree; dry-run prints the multipart PATCH /settings/ for the avatar and tallies cover/attachment as skipped (dry-run can't capture the note id assigned by POST).
Carryover
- Push-side conflict resolution. The Contents API PUTs we use overwrite the remote blob. A user editing on two devices between syncs can lose changes. Next round: fetch existing blob, diff against the materialized payload, and surface a "remote changed" warning before overwriting.
- Asset rotation / pruning. Pushing with assets repeatedly
accumulates orphan asset files for notes that have been deleted
client-side but where the GitHub commit history still references
the old path. A
--prune-orphansmode on the push pipeline can walk the Trees API and delete unreferencedassets/notes/<uuid>/subtrees in the same commit.