Convert to Eleventy. #836

davidlehn · 2024-03-19T03:27:29Z

Status:

Second try at converting to a static site generator.
- First try was to Jekyll in 2017: Jekyll-based json-ld.org #461.
- Much of the discussion there applies here after many years.
There are many many many SSG choices. This conversion is using Eleventy. If the community wants to use another system, it's likely these conversions are a good start with similar patterns.
What's happening in the initial commits here:
- Handles pass through of all useful (and more) files that were served before.
- Converts the runtime php indexing code to build time js code.
- Some tweaks for uniformity.
The plain apache+php hosting we use now could probably host this output with a few operational changes.
Templates and various upgrades and improvements are not yet done. Solving issues one step at a time.

Issues:

Basically same ones as in the jekyll issue.
How to handle minutes/. It's gigantic and inappropriate to mix into a static site directly.
- It is being updated on the main site AND on gh pages at https://json-ld.github.io/minutes/, but the logs note that 1.8G is beyond the 1G max and deploy might fail and it can take 20min+ to deploy. Another solution would be better. Also the "pre-wg" link is easy to not notice.
How to handle the playground and playground php proxies.
How best to host and handle updates.
- May need content type and cors fixes depending on hosting platform.

Notes:

One approach for many issues is a subdomain using gh pages or something else for minutes, cloudflare for hosting, cloudflare workers for the proxy. I think Benjamin likes this idea.
We also need to decide how to improve the playground. No one at all wants to work on it as is. Integrating a web app build step into the static site process is doable but perhaps not the best approach. I'm pondering the site code here being more content based and (ab)using a CDN or gh pages to host built playground app bundles and resources. Suggestions welcome on that.

Testing:

npm install
npm run serve
# open http://localhost:8080/ (or whatever URL it gives you)

- Copy and ignore files as needed. - Comment out old `prettify.css` usage. - Move `learn.html` to `learn/index.html` and update links. This could break external links, but is aligned with the pattern of other files and dirs. Eleventy will put it in `learn/` by default either way without an override. - `specs/`, `primer/`, and `requirements/`: - Generate sub-dir listing data with JS. - Switch from PHP to LiquidJS.

gkellogg

Thanks for work on this, Dave! Most of the space the minutes consume is due to audio captures of the early meetings, which we haven’t kept going in some time. Do logs indicate if they’re ever actually accessed? Maybe there’s a way to move those someplace else off of the main (or minutes) site.

BigBlueHat

At last!

BigBlueHat

Looks great overall. Just some non-blocking recommendations.

BigBlueHat · 2024-04-17T18:35:30Z

.eleventy.js

+  eleventyConfig.ignores.add('presentations');
+  eleventyConfig.ignores.add('scripts');
+  eleventyConfig.ignores.add('spec/tools');
+  eleventyConfig.ignores.add('spec/LICENSE.md');


These ignores can be moved to .eleventyignore to make this file more sane.

What's wrong with having them here? At least for now. It's a bit confusing as is due to eleventy processing various things, having to pass through others, and ignore particular files. And for drafts, that's done computationally. I thought it made more sense to keep the details in one place.

I get the single place to look. I mostly just like data in inert places whenever possible vs. buried among API calls.

package.json

cloudflare-workers-and-pages · 2024-04-17T18:43:14Z

Deploying json-ld-org with Cloudflare Pages

Latest commit:	`eac8e81`
Status:	✅ Deploy successful!
Preview URL:	https://f886401a.json-ld-org.pages.dev

View logs

- Convert README from reStructuredText to Markdown. - The reStructuredText format still works, and was a fine choice back in 2010(!) but Markdown is more common for READMEs like this today.

- Add `404.html` for Cloudflare Pages.

davidlehn · 2024-05-09T16:52:24Z

Another issue with this conversion is how to handle all the .htaccess rules. If the target host is Apache then of course these will still work. But our likely new target is Cloudflare Pages, which does things differently.

$ git ls-files | grep .htaccess
.htaccess
contexts/.htaccess
images/.htaccess
spec/ED/.htaccess
spec/latest/.htaccess
test-suite/.htaccess
test-suite/tests/.htaccess
utils/.htaccess

CF has _redirects and _headers files, which are simpler and limited. Some of the .htaccess rules can be handled with those files. (Untested, but looks promising)
The content type detection is trickier and I believe would require another _functions worker for some special cases. Effectively writing some js to do what apache rules do.

And while pondering these things, I'm wondering where this is headed.

A good dev env is now rather complex.
11ty has the nice auto update dev server. But it's unaware of these htaccess or cloudflare rules. Making it difficult to actually test the entire site. To use the playground proxy (see other PR), you need to have a 11th proxy to a cloudflare wrangler dev server.
Cloudflare wrangler could also serve the static dev site with its redirects and headers and functions. But then the auto update feature doesn't work. There may be some extra proxying to make that work. But then that would require two dev servers to do dev work, which seems a bit awkward.
Should we make apache or cloudflare a build option? I suspect it will just flip which files get copied to the output. But that's two things to maintain and keep in sync.

BigBlueHat · 2024-07-29T15:12:04Z

@davidlehn per your concerns above, I think we can keep using 11ty separate from the hosting platform. Having 11ty in place makes working on (and redesigning) the site much simpler--where it's hosted can be a separate matter because 11ty generates static sites.

Cloudflare hosting can easily get more complex/tangled (just like the build tool nightmare that is modern JavaScript vs. ye olde <script> tag). It really depends on what we want to manage, pay for, support, etc. Most/all of the Cloudflare stuff, though, can also be defined descriptively (or at least via JS in the same repo) to avoid "blind"/hidden configurations available only via the Cloudflare Dashboard.

All that to say, how this is hosted is orthogonal to using 11ty.

README.md

BigBlueHat · 2024-09-24T22:46:23Z

@davidlehn how close do you think this is to shippable? I know #840 is part of the consideration here too. Just curious how we can get this over the finish line.

Since the WG is getting rechartered, it'd be super to get this in a space we can continue to improve, redesign, etc.

Happy to help finish it off...just not sure where we're stuck.

- Remove PHP proxy code. - Move proxy URL to /playground/proxy. - Use eleventy@3 alpha for `onRequset` dev server support. - Add Eleventy dev server `onRequest` proxy to local wrangler server proxy. Due to current limited API, emulate a basic playground request. - Reimplement proxy as a Cloudflare Pages Function. - Basic proxy designed for limited use and abuse. - Add various checks to limit useful scope. - Update README. - Ignore .wrangler.

BigBlueHat · 2025-04-02T15:24:12Z

11ty has the nice auto update dev server. But it's unaware of these htaccess or cloudflare rules. Making it difficult to actually test the entire site. To use the playground proxy (see other PR), you need to have a 11th proxy to a cloudflare wrangler dev server.

Cloudflare wrangler could also serve the static dev site with its redirects and headers and functions. But then the auto update feature doesn't work. There may be some extra proxying to make that work. But then that would require two dev servers to do dev work, which seems a bit awkward.

To untangle this, I've added a npm run pages option which can be used alongside npm run watch. That should provide the handy auto-building from 11ty while serving a localhost Cloudflare-like environment.

Should we make apache or cloudflare a build option? I suspect it will just flip which files get copied to the output. But that's two things to maintain and keep in sync.

I don't think we should do this as it'd be a good deal of overhead to maintain/juggle both. Consequently, I've removed the .htaccess files in #855.

Once #855 is merged into this PR, I think this one can be marked "ready for review", and we can get a preview URL on Cloudflare sporting all the things for further testing from everyone here.

TallTed

My earlier concerns were due to misinterpreting Markdown. They're resolved

davidlehn added 2 commits March 18, 2024 21:49

Remove empty file.

19883af

davidlehn added the website label Mar 19, 2024

gkellogg approved these changes Mar 19, 2024

View reviewed changes

davidlehn marked this pull request as draft March 19, 2024 03:50

BigBlueHat approved these changes Apr 17, 2024

View reviewed changes

davidlehn added 4 commits April 25, 2024 21:03

Use installed eleventy.

ba19a52

Remove unused fields.

386eb14

Convert README to markdown.

b0fa2a8

- Convert README from reStructuredText to Markdown. - The reStructuredText format still works, and was a fine choice back in 2010(!) but Markdown is more common for READMEs like this today.

Add 404 page.

a8a8b0c

- Add `404.html` for Cloudflare Pages.

TallTed requested changes Jul 29, 2024

View reviewed changes

README.md Show resolved Hide resolved

README.md Outdated Show resolved Hide resolved

davidlehn added 2 commits March 27, 2025 11:54

Ignore .backup files.

d366038

BigBlueHat requested a review from TallTed April 1, 2025 16:07

TallTed approved these changes Apr 3, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Convert to Eleventy. #836

Convert to Eleventy. #836

davidlehn commented Mar 19, 2024

gkellogg left a comment

BigBlueHat left a comment

BigBlueHat left a comment

BigBlueHat Apr 17, 2024

davidlehn Apr 26, 2024

BigBlueHat Jul 29, 2024

cloudflare-workers-and-pages bot commented Apr 17, 2024 •

edited

Loading

davidlehn commented May 9, 2024

BigBlueHat commented Jul 29, 2024

BigBlueHat commented Sep 24, 2024

BigBlueHat commented Apr 2, 2025

TallTed left a comment

Convert to Eleventy. #836

Are you sure you want to change the base?

Convert to Eleventy. #836

Conversation

davidlehn commented Mar 19, 2024

gkellogg left a comment

Choose a reason for hiding this comment

BigBlueHat left a comment

Choose a reason for hiding this comment

BigBlueHat left a comment

Choose a reason for hiding this comment

BigBlueHat Apr 17, 2024

Choose a reason for hiding this comment

davidlehn Apr 26, 2024

Choose a reason for hiding this comment

BigBlueHat Jul 29, 2024

Choose a reason for hiding this comment

cloudflare-workers-and-pages bot commented Apr 17, 2024 • edited Loading

Deploying json-ld-org with Cloudflare Pages

davidlehn commented May 9, 2024

BigBlueHat commented Jul 29, 2024

BigBlueHat commented Sep 24, 2024

BigBlueHat commented Apr 2, 2025

TallTed left a comment

Choose a reason for hiding this comment

cloudflare-workers-and-pages bot commented Apr 17, 2024 •

edited

Loading