Hi! I’m Isaac Mills, I’m the guy managing the infrastructure behind Compute! In this article, I’d like to talk about just that: the infra behind this media outlet, how it all works, and why it is the way it is. Plain text is as simple as it gets, yet it can be an extremely powerful tool. It’s capable of being anything, and can also be transmuted into anything. Its infinite extensibility makes it a formidable tool that every developer should have in their arsenal. For our case, we use a lot of plain text. In fact, the article you’re reading right now is written in plain text, not with some web UI. A while back, I found a markup language called djot. It was created by the same person who created CommonMark, a flavor of markdown, to be easier to parse and more featureful. Below is some example djot The benefit of using djot is that it compiles directly to HTML, thus the jorunalists who have joined Compute don’t need to learn HTML to write articles. They also don’t need to learn a clunky slow website editor like Wix or Squarespace. This is another superpower of plain text, if we used Wix for our website, our journalists would need to learn how to use the Wix UI, and how to write articles for that UI. If we needed to change our tooling at any time, they would need to re-learn everything for that tool. Not only that, but we would need to port the entire website (every article), to use that new tooling. This is not so with plain text, if things change in the pipeline, or even if you’re just joining our publication, there’s no need to re-learn how to write text. At worst, you just need to convert the plain text to another format of plain text (djot to HTML for example). All our journalists need to know how to do is write their articles in djot, and submit it to the team via the pipeline Pictured above is the full pipeline that articles go through in order to reach you readers at home, it goes like this Basically, this is just the workflow you would use for code, but adapted for journalism. In other words, no learning curve for our journalists! And if they do need to learn it, then this is information that they should know anyway. The workflow you see above has been in the making since git was created in 2005, with the sole purpose of efficiently moving code from development, into production. In the places where this workflow is used, more overhead in the pipeline means time, and time means money. Thus the workflow used in those places should be as simple as possible to reduce overhead, and I see it no differently for our publication. If this workflow has worked for nearly 2 decades for a pathologically huge project like the Linux kernel (which git was tailor-made to handle), the better. Consider the following: If I’m accepting untrusted code from the public into my open source project, and I need that code to be production ready, how can I ensure that the code I accept is actually production ready. The answer is with continuous integration, or CI. The idea is that every piece of code submitted to an open source project would undergo automated testing, linting, and checking to ensure that nothing will break upon merging the code into the production code base. For a project like Their pipeline checks if the library with your new code compiles to every platform it’s compatible with, with every feature enabled. It also makes sure that your code is well-formatted, contains no conflicts of license, uses no libraries banned by the project, and contains no security advisories. The only way this many checks can be done on every git commit, is through CI, GitHub Actions in egui’s case. The way I’ve described CI so far has probably made it seem like the least simple thing you could add to your project, but CI is actually quite simple, and it’s an excellent example of how capable of a tool you can make by just building simple, on top of simple, on top of simple. Really, CI is just shell scripting with extra steps. Despite it’s inherent simplicity, CI can serve not only as a means to filter bugs out of pull requests, but it can also be a simple way to communicate to open source developers what a project wants out of their code. Instead of having to read a big Fortunately, the level of CI I’ve described above isn’t required for journalism. Our CI simply compiles our journalists’ unreviewed articles, and serves them on an un-indexed (not visible on production) web page so that they and the team can preview their work before merging it. Our CI is also responsible for indexing and publishing finished articles onto our production website. Deployment is the most complex part of our pipeline; getting the written, production ready articles, onto the website you are reading this on. As I said earlier in the article, we use CI to compile and index finished articles. The CI tool we use is called Woodpecker CI, a self-hosted docker-based CI tool. Self-hosted basically means that we can run the CI tool on the same server we use to serve our website, making deployment to there as easy as moving the generated files into the directory that our web server is serving (NGINX in our case). What’s important to know is that when a pull request is opened on this journal’s git repository, and when a pull request is merged into production, Woodpecker CI will run a custom program that I wrote in Rust to… Because our CI tool is running this code, we can know which articles need compilation, who wrote those articles, and if any articles need to be deleted. Our CI tools will put our code in the working directory of our git repo. And via environment variables, the CI tool will tell us which files have changed, how the pipeline was triggered (via pull request, code push, etc.), as well as which git branch production resides in, and which branch our code is currently in. Our code can then use this information to… Everything I’ve described that the program can do so far can be done in just ~400 lines of Rust. This is the effect of buidling simple on top of simple, and making an effort to retain simplicity by making the most out of the simple building blocks. (P.S: Our Rust code now also generates a title card image for each article, so they look better on platforms like Twitter and Discord. By using the same stradegy, I only needed to add ~230 lines to our Rust code) My Rust code makes heavy use of git, a very simple yet amazing tool for adding functionality to plain text. By taking advantage of the vast functionality that a simple tool like git gives us, our code can stay lean, while also being very mean. Speaking of staying lean… There is no better example of the cons of complexity than the modern web. Most webpages are not only bloated with ads, cookie banners, autoplaying BS, popups and the like; but they are also inundated with copious amounts of JavaScript. We only use JavaScript in 2 places Other than that, the actual article pages, such as this one, depend on nothing but the JavaScript that Webflow bundles in. And for our homepage, it’s built and optimized so it can be served statically with it’s one dependency. Basically, I wanted to make our website as suckless (as lightweight, and as free from bloat) as possible. I say I wanted to because our founder wanted to use Wix originally, yuck. Instead of that, I used Webflow to design our webpages. For a one-time fee of $24, you can have access to the Webflow editor for 1 month, and then export your web pages to HTML/CSS/JS. Webflow is very different from your average Wix/Squarespace, those editors are designed for non-programmers who don’t know and don’t want to know HTML or CSS. Webflow is an editor for developers that know what they’re doing. It gives you the full power of HTML and CSS in a responsive, visual editor; making it an incredibly flexible tool capable of generating very based and performant webpages. Unlike the latter tools which generate bloated and obfuscated garbage. In just HTML, CSS, and JS: The homepage of this journal compressed is around 152.42 KB large, whereas our old Wix website’s homepage compressed is around 861.9 KB large. That’s nearly a 6x size difference. That’s the effect of building your website in a UI that attempts to match up to the flexibility of HTML and CSS… without actually exposing you to the HTML and CSS. This is where Webflow thrives, as it is literally just a simple graphical editor for HTML and CSS. It’s another excellent example of what building simple around simple can do! Computers have an inconcievable amount of potential, but they’re only as smart as their programmer. When you’re building something with a computer, it’s often much better to do more with less, than less with more. Don’t use 17 different JavaScript frameworks with your hypervisor GPU WEB2.0 interface-driven scripting framework to drive your map/reduce-aware proxy-oriented software API. Just start simple, build simple, and if you need complexity, build it with more simple. The point I’m trying to make here is that bloat is your enemy, and it’s often better for you, your team, and your users to just KISS (Keep it simple, stupid!)Plain text
# Heading
paragraph
*bold* _italic_ _*bold italic*_ {-strikethorugh-} {+underline+}
- list
- list
1. list
2. list
3. list
The Pipeline
CI
egui, their CI pipeline contains 19 checks.CONTRIBUTORS.md file to get an idea of that, developers can know that their code is good quality if it just passes CI.Deployment
Webpages (and staying based)
In conclusion
