From 422560ec1090e21cde0addbaaa58ae66a293c490 Mon Sep 17 00:00:00 2001 From: Javier Cicchelli Date: Sun, 2 Aug 2026 02:57:43 +0200 Subject: [PATCH] Prompt the canonical site URL during bootstrap script to rewrite website crawlers. --- README.md | 8 +- Scripts/bootstrap | 45 ++++++++++++ Services/Website/README.md | 73 +++++++++++++++---- Services/Website/Resources/Static/robots.txt | 2 +- Services/Website/Resources/Static/sitemap.xml | 2 +- 5 files changed, 112 insertions(+), 18 deletions(-) diff --git a/README.md b/README.md index 63eb852..1fa7fe0 100644 --- a/README.md +++ b/README.md @@ -46,6 +46,11 @@ You'll be asked for: | --- | --- | --- | | **Site display name** (PascalCase) | `Loud` | Server name (`LoudWebsite`), `Loud.xcodeproj`, README title. | | **Project slug** (lowercase) | `loud-ams` | Container owner, database name/user, Compose project name. | +| **Canonical site URL** (scheme + host) | `https://loud.amsterdam` | The `Sitemap:` reference in `robots.txt` and the `` entry in `sitemap.xml`. | + +The canonical URL defaults to the reserved placeholder `https://site.example.com`; +leaving it there is allowed (the script warns), so you can bootstrap before the +domain is decided and set it later in the two crawler files. The script rewrites the placeholder occurrences in place, renames `Site.xcodeproj`, optionally starts a fresh git history, and then removes @@ -57,7 +62,8 @@ Bootstrap leaves reminders, but in short: 1. **Copy & localization** — `Services/Website/Sources/Library/Catalogs/Localizable.xcstrings` and the landing / 404 pages under `Services/Website/Sources/Library`. -2. **Static assets** — `Services/Website/Resources/Static` (CSS, JS, favicon, icons, manifest). +2. **Static assets** — `Services/Website/Resources/Static` (CSS, JS, favicon, icons). + The manifest's `name` / `short_name` are empty and are not rewritten by bootstrap. 3. **Secrets** — set a real `DATABASE_PASSWORD` in a git-ignored `Services/Website/.env` (the committed `.env.local` is an example only). 4. **Domain models** — the `Persistence` package ships a sample diff --git a/Scripts/bootstrap b/Scripts/bootstrap index 10aba68..8d44d58 100755 --- a/Scripts/bootstrap +++ b/Scripts/bootstrap @@ -24,6 +24,11 @@ fi XCODEPROJ="./Site.xcodeproj" +# The placeholder origin the crawler files ship with. It is an RFC 2606 reserved +# domain, so an un-bootstrapped copy can never point a crawler at a real site. +PLACEHOLDER_URL="https://site.example.com" +PLACEHOLDER_URL_PATTERN='https://site\.example\.com' + # --- Prompts ------------------------------------------------------------------ printf 'Site display name (PascalCase, e.g. Berlin) [Site]: ' @@ -35,9 +40,31 @@ printf 'Project slug (lowercase, used for owner/db/compose, e.g. loud-berlin) [% read -r SLUG SLUG="${SLUG:-$DEFAULT_SLUG}" +printf 'Canonical site URL (scheme + host, e.g. https://berlin.example.com) [%s]: ' "$PLACEHOLDER_URL" +read -r URL +URL="${URL:-$PLACEHOLDER_URL}" +URL="${URL%/}" # the rewrites append their own path, so drop a trailing slash + +# The URL is interpolated into a sed replacement (with `|` as the delimiter) and +# into the crawler files, so reject anything that is not a plain scheme + host. +case "$URL" in + http://* | https://*) ;; + *) + echo "error: the canonical site URL must start with http:// or https:// (got \"$URL\")." >&2 + exit 1 + ;; +esac +case "$URL" in + *[!-A-Za-z0-9:/._~]*) + echo "error: the canonical site URL contains unexpected characters (got \"$URL\")." >&2 + exit 1 + ;; +esac + echo echo " display name : $NAME (server name \"${NAME}Website\", ${NAME}.xcodeproj)" echo " slug : $SLUG (container owner, database name/user, compose project)" +echo " canonical URL: $URL (robots.txt sitemap reference, sitemap.xml entry)" echo printf 'Apply these values? [y/N]: ' read -r CONFIRM @@ -95,6 +122,14 @@ rewrite "$W/docker-compose.override.yml" \ rewrite "$W/Makefile" \ -e "s/),site)/),${SLUG})/g" +# Crawler files: the absolute origin they must carry. `|` is the sed delimiter, +# since both the placeholder and the replacement contain slashes. +rewrite "$W/Resources/Static/robots.txt" \ + -e "s|${PLACEHOLDER_URL_PATTERN}|${URL}|g" + +rewrite "$W/Resources/Static/sitemap.xml" \ + -e "s|${PLACEHOLDER_URL_PATTERN}|${URL}|g" + # Service README. rewrite "$W/README.md" \ -e "s/^# Site Website\$/# ${NAME} Website/" \ @@ -147,6 +182,7 @@ Next steps: - $W/Sources/Library/Catalogs/Localizable.xcstrings (copy) - $W/Sources/Library (landing / 404 pages) - $W/Resources/Static (css, js, favicon, icons) + - $W/Resources/Static/site.webmanifest (name / short_name) 2. Set a real database password in a git-ignored $W/.env (the committed .env.local defaults the password to the slug — do NOT ship that). 3. Point the git remote at your new repository: @@ -157,3 +193,12 @@ Next steps: The Persistence package still ships an ExampleRecord / ExampleRepository sample model — replace it with your real domain models when you add persistence. EOF + +if [ "$URL" = "$PLACEHOLDER_URL" ]; then + cat <` | The page's one-line description. | +| `canonicalURL` | `` | Absolute URL. | +| `socialCard` | Open Graph + Twitter `` tags | A `SocialCard` — title, summary, URL, site name, locale, share image. Scrapers require absolute URLs, so the page composes them from its own origin. | +| `structuredData` | `