mirror of
https://github.com/andwati/andwati.com.git
synced 2026-11-20 07:28:49 +00:00
Extend llms.txt to all content types; fix a gitignore bug
scripts/generate-llms.astro-site.mjs covers writings, portfolio, bookshelf, and blogs (unblocked now that site.config.ts exists from the feeds/meta work), wired into apps/site's build script. Also fixes .gitignore: the bare `public` pattern (meant to ignore the Zola build's root-level output directory) was matching at any depth, so apps/site/public/ and apps/cms/public/ were being silently excluded from every commit despite `git add` reporting them as staged for directory adds (the ignore-file warning only surfaces for an explicitly named path, not a recursive directory add — easy to miss). Scoped to `/public` so only the root output dir is ignored. Picks up robots.txt, the Astro scaffold's favicon, llms.txt, and Strapi's own public/robots.txt, none of which were actually tracked until now.
This commit is contained in:
+1
-1
@@ -1,4 +1,4 @@
|
|||||||
public
|
/public
|
||||||
node_modules
|
node_modules
|
||||||
manimations/*.*
|
manimations/*.*
|
||||||
/zola
|
/zola
|
||||||
|
|||||||
@@ -49,19 +49,27 @@ approved plan for full rationale on each decision referenced below.
|
|||||||
to the old "CASE FILE" hacker-terminal branding being fully replaced;
|
to the old "CASE FILE" hacker-terminal branding being fully replaced;
|
||||||
building this now would just be redone once the new visual identity
|
building this now would just be redone once the new visual identity
|
||||||
lands
|
lands
|
||||||
- [ ] `llms.txt` generation extended to cover all content types, not just
|
- [x] `llms.txt` generation extended to cover all content types — unblocked
|
||||||
posts — **blocked on site config**: `scripts/generate-llms.mjs` reads
|
once `site.config.ts` existed (built above for feeds/meta anyway):
|
||||||
site metadata (base_url, description, nav) from `zola.toml`, which has
|
`scripts/generate-llms.astro-site.mjs` reads all four content
|
||||||
no equivalent yet in the new site (Astro site config isn't decided)
|
directories directly, wired into `apps/site`'s build script
|
||||||
|
|
||||||
## Design system
|
## Design system
|
||||||
|
|
||||||
- [ ] New typography scale + color system (full reset, not inherited from
|
- [ ] New typography scale + color system (full reset, not inherited from
|
||||||
`sass/_theme.scss`)
|
`sass/_theme.scss`) — **needs your input**: aesthetic/brand decisions,
|
||||||
- [ ] Motion primitives: CSS + native Web Animations API spring utility
|
not something to guess at
|
||||||
- [ ] Base layout/components replacing the placeholder `Base.astro`
|
- [x] Motion primitives: CSS + native Web Animations API spring utility —
|
||||||
- [ ] Homepage redesign around the Three.js centerpiece as hero
|
`apps/site/src/lib/spring.ts` (damped-oscillator keyframes,
|
||||||
- [ ] New 404 page concept (terminal easter egg retired)
|
`prefers-reduced-motion` aware). Not wired into any component yet —
|
||||||
|
that comes with the design system below
|
||||||
|
- [ ] Base layout/components replacing the placeholder `Base.astro` —
|
||||||
|
**needs your input**: depends on the typography/color decision above
|
||||||
|
- [ ] Homepage redesign around the Three.js centerpiece as hero — **needs
|
||||||
|
your input**: depends on the constellation data model decision below
|
||||||
|
- [x] New 404 page concept (terminal easter egg retired) — "Page Fault",
|
||||||
|
a SIGSEGV/memory-fault framing; plain/unstyled pending the design
|
||||||
|
system, same as the rest of the current placeholder layout
|
||||||
|
|
||||||
## Three.js centerpiece
|
## Three.js centerpiece
|
||||||
|
|
||||||
@@ -108,12 +116,24 @@ approved plan for full rationale on each decision referenced below.
|
|||||||
|
|
||||||
## Migration from the legacy Zola site
|
## Migration from the legacy Zola site
|
||||||
|
|
||||||
- [ ] Script/process to convert `content/posts/*.md` into the new
|
- [x] Script/process to convert `content/posts/*.md` into the new
|
||||||
`content/writings/` schema
|
`content/writings/` schema — `scripts/migrate-posts-to-writings.mjs`,
|
||||||
- [ ] Preserve slugs/URLs or add redirects (mechanism TBD — Astro
|
additive only (`content/posts/` untouched), re-runnable/idempotent.
|
||||||
middleware vs. `static-web-server`/`sws.toml` rules)
|
2 of 20 posts (`from-c-to-machine-code`,
|
||||||
- [ ] Migrate `content/archive/`, `content/about.md` equivalents
|
`self-hosting-gitea-and-mirroring-github`) use note/tip/warning/danger
|
||||||
- [ ] Verify RSS subscribers and inbound links survive the cutover
|
callout shortcodes with no Astro equivalent yet — flagged by the
|
||||||
|
script, needs a real component once the design system exists
|
||||||
|
(**not blocking**, just noted for later)
|
||||||
|
- [x] Preserve slugs/URLs via redirects — the script generates 301s from
|
||||||
|
`/posts/<slug>/` to `/writings/<slug>/` into `sws.toml`, idempotently
|
||||||
|
- [ ] Migrate `content/archive/`, `content/about.md` equivalents — holding
|
||||||
|
off: the Astro site doesn't have archive/about pages built yet, and
|
||||||
|
"about" copy/framing is exactly the kind of thing you'd want to
|
||||||
|
review/rewrite for the new "digital legacy" identity, not port as-is
|
||||||
|
- [ ] Verify RSS subscribers and inbound links survive the cutover — can
|
||||||
|
only really be checked once this is deployed live (**needs your
|
||||||
|
input**: this is a production-cutover verification, not a build-time
|
||||||
|
one)
|
||||||
|
|
||||||
## Infra / deployment
|
## Infra / deployment
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,3 @@
|
|||||||
|
# To prevent search engines from seeing the site altogether, uncomment the next two lines:
|
||||||
|
# User-Agent: *
|
||||||
|
# Disallow: /
|
||||||
@@ -7,7 +7,8 @@
|
|||||||
},
|
},
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"dev": "astro dev",
|
"dev": "astro dev",
|
||||||
"build": "astro build && pagefind --site dist",
|
"generate:llms": "node ../../scripts/generate-llms.astro-site.mjs",
|
||||||
|
"build": "pnpm run generate:llms && astro build && pagefind --site dist",
|
||||||
"preview": "astro preview",
|
"preview": "astro preview",
|
||||||
"check": "astro check",
|
"check": "astro check",
|
||||||
"astro": "astro"
|
"astro": "astro"
|
||||||
|
|||||||
Binary file not shown.
|
After Width: | Height: | Size: 655 B |
@@ -0,0 +1,9 @@
|
|||||||
|
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 128 128">
|
||||||
|
<path d="M50.4 78.5a75.1 75.1 0 0 0-28.5 6.9l24.2-65.7c.7-2 1.9-3.2 3.4-3.2h29c1.5 0 2.7 1.2 3.4 3.2l24.2 65.7s-11.6-7-28.5-7L67 45.5c-.4-1.7-1.6-2.8-2.9-2.8-1.3 0-2.5 1.1-2.9 2.7L50.4 78.5Zm-1.1 28.2Zm-4.2-20.2c-2 6.6-.6 15.8 4.2 20.2a17.5 17.5 0 0 1 .2-.7 5.5 5.5 0 0 1 5.7-4.5c2.8.1 4.3 1.5 4.7 4.7.2 1.1.2 2.3.2 3.5v.4c0 2.7.7 5.2 2.2 7.4a13 13 0 0 0 5.7 4.9v-.3l-.2-.3c-1.8-5.6-.5-9.5 4.4-12.8l1.5-1a73 73 0 0 0 3.2-2.2 16 16 0 0 0 6.8-11.4c.3-2 .1-4-.6-6l-.8.6-1.6 1a37 37 0 0 1-22.4 2.7c-5-.7-9.7-2-13.2-6.2Z" />
|
||||||
|
<style>
|
||||||
|
path { fill: #000; }
|
||||||
|
@media (prefers-color-scheme: dark) {
|
||||||
|
path { fill: #FFF; }
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
</svg>
|
||||||
|
After Width: | Height: | Size: 749 B |
@@ -0,0 +1,54 @@
|
|||||||
|
# Thoughts and Musings
|
||||||
|
|
||||||
|
> Ian Andwati's personal digital-legacy site covering computer science, mathematics, life, and beliefs — writings, portfolio, and a curated bookshelf.
|
||||||
|
|
||||||
|
## Main pages
|
||||||
|
|
||||||
|
- [Home](https://andwati.com/): Introduction and latest writings.
|
||||||
|
- [Writings](https://andwati.com/writings/): Long-form essays and technical posts.
|
||||||
|
- [Portfolio](https://andwati.com/portfolio/): Projects and professional work.
|
||||||
|
- [Bookshelf](https://andwati.com/bookshelf/): Books and papers read, with notes.
|
||||||
|
- [Blogs](https://andwati.com/blogs/): Curated external blogs.
|
||||||
|
- [Tags](https://andwati.com/tags/): Writings grouped by topic.
|
||||||
|
|
||||||
|
## Writings
|
||||||
|
|
||||||
|
- [What is pwn?](https://andwati.com/writings/what-is-pwn/): No not that one
|
||||||
|
- [Export default and module.exports in Javascript](https://andwati.com/writings/javascript-exporting-modules/): Modules are the building blocks of modern JavaScript applications, allowing for the encapsulation of code into reusable components. Two primary systems govern the exportation of these modules: export default in ES6 and module.exports in CommonJS.
|
||||||
|
- [cognitive inertia](https://andwati.com/writings/inertia/): inertia
|
||||||
|
- [Self-Hosting Gitea and Automating GitHub Mirrors](https://andwati.com/writings/self-hosting-gitea-and-mirroring-github/): Self-Hosting Gitea and Automating GitHub Mirrors (and Everything Cloudflare Broke Along the Way)
|
||||||
|
- [From C to Machine Code: What Actually Runs](https://andwati.com/writings/from-c-to-machine-code/): Before diving into how to break a program, we need to understand what the computer is actually executing
|
||||||
|
- [Semantic Versioning(SemVer) and Conventional Commits](https://andwati.com/writings/semantic-version-and-conventional-commits/): Adopting best practices in your git and build workflows.
|
||||||
|
- [Integrating Netlify CMS](https://andwati.com/writings/integrating-netlify/): Integrating netlify cms
|
||||||
|
- [Install PostgreSQL on Arch | Manjaro | Garuda Linux](https://andwati.com/writings/install-postgresql-15-on-arch-manjaro-garuda-linux/): Getting PostgreSQL up and running on Arch Linux-based distros
|
||||||
|
- [A Fresh Coat Of Paint](https://andwati.com/writings/a-fresh-coat-of-paint/): Migrating to Zola
|
||||||
|
- [What say you?](https://andwati.com/writings/utterances-comments/): Adding a comment system by utterances
|
||||||
|
- [The One Thing Nobody Tells You About WiFi Hotspots on Arch Linux](https://andwati.com/writings/wifi-hotspot-arch/): A minimal Arch install is great until NetworkManager silently fails to find a binary you didn't know you needed
|
||||||
|
- [Setting up Obsidian sync in Windows](https://andwati.com/writings/obsidian-sync/): Syncing Obsidian in Windows with Windows Task Scheduler with git
|
||||||
|
- [The Single Byte That Kills Your Exploit: Understanding Endianness](https://andwati.com/writings/the-single-byte-that-kills-your-exploit-understanding-endianness/): Problem: Computers speak different languages, like people
|
||||||
|
- [Why This Site Exists](https://andwati.com/writings/example-first-post/): A short note on what this new site is, and why it's more than a tech blog.
|
||||||
|
- [mmuctf 2025 writeup](https://andwati.com/writings/mmuctf2025/): mmuctf 2025 challenges writeup
|
||||||
|
- [Configure Vue.js app With Tailwind CSS](https://andwati.com/writings/vuejs-tailwind/): Tailwindcss in vuejs
|
||||||
|
- [Subresource Integrity](https://andwati.com/writings/subresource-integrity/): Understanding Subresource Integrity
|
||||||
|
- [ScriptCTF 2025 Writeup](https://andwati.com/writings/scriptctf2025/): ScriptCTF 2025 Writeup
|
||||||
|
- [Django Secret Key Tutorial](https://andwati.com/writings/generate-django-secret-key/): Managing secrets
|
||||||
|
- [Hello World](https://andwati.com/writings/hello-world/): Hello world
|
||||||
|
|
||||||
|
## Portfolio
|
||||||
|
|
||||||
|
- [Example Project](https://andwati.com/portfolio/): Established the portfolio content schema.
|
||||||
|
|
||||||
|
## Bookshelf
|
||||||
|
|
||||||
|
- [Attention Is All You Need](https://andwati.com/bookshelf/): Ashish Vaswani, Noam Shazeer, Niki Parmar
|
||||||
|
- [Structure and Interpretation of Computer Programs](https://andwati.com/bookshelf/): Harold Abelson, Gerald Jay Sussman
|
||||||
|
|
||||||
|
## Blogs followed
|
||||||
|
|
||||||
|
- [Example Blog](https://example.com)
|
||||||
|
|
||||||
|
## Feeds and discovery
|
||||||
|
|
||||||
|
- [RSS feed](https://andwati.com/rss.xml): Full writings feed in RSS format.
|
||||||
|
- [JSON feed](https://andwati.com/feed.json): Full writings feed in JSON Feed format.
|
||||||
|
- [Sitemap](https://andwati.com/sitemap-index.xml): Machine-readable index of public pages.
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
User-agent: *
|
||||||
|
Allow: /
|
||||||
|
|
||||||
|
Sitemap: https://andwati.com/sitemap-index.xml
|
||||||
@@ -0,0 +1,127 @@
|
|||||||
|
// Generates apps/site/public/llms.txt for the new Astro site, covering all
|
||||||
|
// four content types. Run before `astro build` (wired into
|
||||||
|
// apps/site/package.json's build script) so it's included as a static file.
|
||||||
|
//
|
||||||
|
// Site metadata is duplicated from apps/site/src/site.config.ts rather than
|
||||||
|
// imported — this script runs as plain Node outside Vite/Astro's pipeline,
|
||||||
|
// and the config is small enough that duplicating it beats adding a
|
||||||
|
// cross-module-system import.
|
||||||
|
import fs from "node:fs";
|
||||||
|
import path from "node:path";
|
||||||
|
import { fileURLToPath } from "node:url";
|
||||||
|
import { parse as parseToml } from "smol-toml";
|
||||||
|
|
||||||
|
const baseUrl = "https://andwati.com";
|
||||||
|
const siteTitle = "Thoughts and Musings";
|
||||||
|
const summary =
|
||||||
|
"Ian Andwati's personal digital-legacy site covering computer science, mathematics, life, and beliefs — writings, portfolio, and a curated bookshelf.";
|
||||||
|
|
||||||
|
// Resolved relative to this file, not process.cwd() — this script runs as
|
||||||
|
// part of apps/site's build script, whose cwd is apps/site, not the repo
|
||||||
|
// root where content/ lives.
|
||||||
|
const root = path.join(path.dirname(fileURLToPath(import.meta.url)), "..");
|
||||||
|
const contentDir = path.join(root, "content");
|
||||||
|
const outFile = path.join(root, "apps", "site", "public", "llms.txt");
|
||||||
|
|
||||||
|
const FRONTMATTER_RE = /^\+\+\+\r?\n([\s\S]*?)\r?\n\+\+\+\r?\n?([\s\S]*)$/;
|
||||||
|
|
||||||
|
function walk(dir) {
|
||||||
|
if (!fs.existsSync(dir)) return [];
|
||||||
|
return fs.readdirSync(dir, { withFileTypes: true }).flatMap((entry) => {
|
||||||
|
const full = path.join(dir, entry.name);
|
||||||
|
if (entry.isDirectory()) return walk(full);
|
||||||
|
return entry.isFile() && entry.name.endsWith(".md") ? [full] : [];
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function slugFor(dir, file) {
|
||||||
|
const rel = path.relative(dir, file).split(path.sep).join("/");
|
||||||
|
return rel.endsWith("/index.md")
|
||||||
|
? rel.slice(0, -"/index.md".length)
|
||||||
|
: rel.slice(0, -".md".length);
|
||||||
|
}
|
||||||
|
|
||||||
|
function readEntries(typeDir) {
|
||||||
|
return walk(typeDir)
|
||||||
|
.map((file) => {
|
||||||
|
const raw = fs.readFileSync(file, "utf8");
|
||||||
|
const match = raw.match(FRONTMATTER_RE);
|
||||||
|
if (!match) return null;
|
||||||
|
const frontmatter = parseToml(match[1]);
|
||||||
|
return { slug: slugFor(typeDir, file), frontmatter };
|
||||||
|
})
|
||||||
|
.filter(Boolean);
|
||||||
|
}
|
||||||
|
|
||||||
|
const writings = readEntries(path.join(contentDir, "writings"))
|
||||||
|
.filter((e) => !e.frontmatter.draft)
|
||||||
|
.sort((a, b) =>
|
||||||
|
String(b.frontmatter.date).localeCompare(String(a.frontmatter.date)),
|
||||||
|
);
|
||||||
|
const portfolio = readEntries(path.join(contentDir, "portfolio")).filter(
|
||||||
|
(e) => !e.frontmatter.draft,
|
||||||
|
);
|
||||||
|
const bookshelf = readEntries(path.join(contentDir, "bookshelf"));
|
||||||
|
const blogs = readEntries(path.join(contentDir, "blogs"));
|
||||||
|
|
||||||
|
function link(title, url, description) {
|
||||||
|
return `- [${title}](${url})${description ? `: ${description.replace(/\s+/g, " ").trim()}` : ""}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
const lines = [
|
||||||
|
`# ${siteTitle}`,
|
||||||
|
"",
|
||||||
|
`> ${summary}`,
|
||||||
|
"",
|
||||||
|
"## Main pages",
|
||||||
|
"",
|
||||||
|
`- [Home](${baseUrl}/): Introduction and latest writings.`,
|
||||||
|
`- [Writings](${baseUrl}/writings/): Long-form essays and technical posts.`,
|
||||||
|
`- [Portfolio](${baseUrl}/portfolio/): Projects and professional work.`,
|
||||||
|
`- [Bookshelf](${baseUrl}/bookshelf/): Books and papers read, with notes.`,
|
||||||
|
`- [Blogs](${baseUrl}/blogs/): Curated external blogs.`,
|
||||||
|
`- [Tags](${baseUrl}/tags/): Writings grouped by topic.`,
|
||||||
|
"",
|
||||||
|
"## Writings",
|
||||||
|
"",
|
||||||
|
...writings.map((e) =>
|
||||||
|
link(
|
||||||
|
e.frontmatter.title,
|
||||||
|
`${baseUrl}/writings/${e.slug}/`,
|
||||||
|
e.frontmatter.description,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
"",
|
||||||
|
"## Portfolio",
|
||||||
|
"",
|
||||||
|
...portfolio.map((e) =>
|
||||||
|
link(e.frontmatter.title, `${baseUrl}/portfolio/`, e.frontmatter.outcome),
|
||||||
|
),
|
||||||
|
"",
|
||||||
|
"## Bookshelf",
|
||||||
|
"",
|
||||||
|
...bookshelf.map((e) =>
|
||||||
|
link(
|
||||||
|
e.frontmatter.title,
|
||||||
|
`${baseUrl}/bookshelf/`,
|
||||||
|
(e.frontmatter.authors ?? []).join(", "),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
"",
|
||||||
|
"## Blogs followed",
|
||||||
|
"",
|
||||||
|
...blogs.map((e) => link(e.frontmatter.title, e.frontmatter.url)),
|
||||||
|
"",
|
||||||
|
"## Feeds and discovery",
|
||||||
|
"",
|
||||||
|
`- [RSS feed](${baseUrl}/rss.xml): Full writings feed in RSS format.`,
|
||||||
|
`- [JSON feed](${baseUrl}/feed.json): Full writings feed in JSON Feed format.`,
|
||||||
|
`- [Sitemap](${baseUrl}/sitemap-index.xml): Machine-readable index of public pages.`,
|
||||||
|
"",
|
||||||
|
];
|
||||||
|
|
||||||
|
fs.mkdirSync(path.dirname(outFile), { recursive: true });
|
||||||
|
fs.writeFileSync(outFile, lines.join("\n"));
|
||||||
|
console.log(
|
||||||
|
`Generated ${path.relative(root, outFile)} (${writings.length} writings, ${portfolio.length} portfolio, ${bookshelf.length} bookshelf, ${blogs.length} blogs).`,
|
||||||
|
);
|
||||||
Reference in New Issue
Block a user