Bundling ClojureScript to leverage Modern JavaScript

Bundling JavaScript like it’s 2015

Ellis Kenyő

24th February 2024

8 min read

Writing ClojureScript is nice, until we need something from “modern JavaScript”. If only there was a better way…

I’ve recently found myself leaning more on ClojureScript when it comes to writing web applications. It brings a lot of pros to the mix, a few of which include:

• A much more succinct syntax
• Related to the previous point, it’s a really good fit for component-driven design since you can trivially nest and compose web components (functions)
• A macro system, kind of ¹
• The ability to write cross-domain code²

Overall, it’s really great. Coupled with something like uix it makes writing React applications simple.

…that is until you rely on a library like react-query (for the purposes of this article, you don’t need to understand or even be aware of what it does).

A brief tangent

So, in order to explain the problem we need to go over some context with ClojureScript. If you’re already familiar with the internals of ClojureScript, feel free to skip this section.

ClojureScript relies on the Google Closure compiler to emit its code, and as such any and all features from upstream JavaScript also have to be implemented there. The problem that we have is that some features that “modern JavaScript” rely on are not implemented yet in Closure, e.g. Class fields.

There’s nothing inherent “we” can do (except implement the code and see it through of course), so while we wait for these features to be added we have to make do.

Surely there’s a way we can leverage these modern features?

Leveraging these modern features

Among shadow-cljs’ numerous options is the ability to specify a :js-provider which includes a number of options, the one we care about here is :external.

What this option in particular does is instructs shadow-cljs to not handle the various require statements and just emit a single index file. With this index file, we can pass it to some other JavaScript bundler (webpack, for our purposes) and have that handle our bundling.

By adding a small overhead (more on that later) and an “extra compile step” (welcome to the web), we can write ClojureScript as we normally would including using 3rd party libraries that rely on these “modern features” just as we would in regular JavaScript.

Setup

For these purposes, I’ll run through a simple ClojureScript app I was tinkering around with.

shadow-cljs.edn

{:dev-http {3000 "./resources/public/"}
 :source-paths ["src"]
 :dependencies
 [[com.pitch/uix.core "1.0.1"]
  [com.cognitect/transit-cljs "0.8.280"]
  [com.pitch/uix.dom "1.0.1"]]
 :builds
 {:main
  {:target :browser
   :output-dir "resources/public/js"
   :asset-path "/js"
   :js-options {:js-provider :external}
   :compiler-options {:source-map true}
   :devtools {:preloads [snippets.preloads]}
   :modules {:main {:init-fn snippets.core/init
                    :entries [snippets.core]}}
   :release {:target :browser
             :output-dir "dist/js"
             :asset-path "/js"
             :modules {:main {:entries [snippets.core]}}}}}}

The highlighted lines are the only ones we care about. The first one is where our JavaScript is produced, which we’ll need later. The other lines are the only setup we need to get this working. Once we have this and we build our project, we instead produce a file which by default lives in target/external.js and looks like the following

// WARNING: DO NOT EDIT!
// THIS FILE WAS GENERATED BY SHADOW-CLJS AND WILL BE OVERWRITTEN!

var ALL = {};
ALL["react-dom/client"] = require("react-dom/client");
ALL["react-refresh/runtime"] = require("react-refresh/runtime");
ALL["@uiw/codemirror-theme-atomone"] = require("@uiw/codemirror-theme-atomone");
ALL["@uiw/codemirror-extensions-langs"] = require("@uiw/codemirror-extensions-langs");
ALL["react-dom"] = require("react-dom");
ALL["@uiw/codemirror-extensions-color"] = require("@uiw/codemirror-extensions-color");
ALL["highlight.js"] = require("highlight.js");
ALL["@uiw/react-codemirror"] = require("@uiw/react-codemirror");
ALL["react-router-dom"] = require("react-router-dom");
ALL["react"] = require("react");
global.shadow$bridge = function shadow$bridge(name) {
  var ret = ALL[name];

  if (ret === undefined) {
     throw new Error("Dependency: " + name + " not provided by external JS. Do you maybe need a recompile?");
  }

  return ret;
};

shadow$bridge.ALL = ALL;

Great! We have all our dependencies without any transformations applied. This isn’t wasted on this project, as we have a down-the-line dependency that breaks thanks to this issue (JavaScript is a great ecosystem), so without this setup we can’t build.

Now that we have this, what’s next? Well, we need to point webpack-cli at this file and have it produce its output to the same place shadow-cljs emits your code, and then our index.html can just include it before anything else and we’re away.

We can take care of webpack-cli with the following:

webpack-cli watch \
--entry ./target/external.js \
--output-path resources/public/js/libs \
--target web \
--mode development

(for simplicity I encourage you to make this an npm task)

Lastly, we have to include this new bundled JavaScript in our index.html as below (rest of the file omitted for brevity):

<body>
    <div id="root"></div>

    <script src="/js/libs/main.js"></script>
    <script src="/js/main.js"></script>
</body>

Make sure that the bundled JavaScript comes first.

I use a subdirectory to ensure that the main.js file it produces doesn’t wipe out my build, and it makes it clearer in the index.html which does what. If you’re using async on the script tags, I wouldn’t recommend it here.

And that’s it!

How does it handle in a real app?

Okay, that’s not the full story.

I’m sure someone is sat there thinking “adding another compile process sure seems like that’s gonna introduce a lot of complexity and slow everything down”, so let’s try and answer those two points.

Complexity

In terms of code we need to introduce and thus maintain, it’s very minimal. You can tweak the build setup as much as you want, if you need your external js file to end up somewhere else or have other transformations applied etc.

The fairest point here is the introduction of webpack. Anyone that’s done web development for a few years is well aware of webpack, JavaScript bundling and all the um … fun that introduces.

For our needs here, we just need webpack to emit JavaScript. The complex things it would normally do like loading JavaScript, ClojureScript, CSS, images and the like are already taken care of and handled in your code by shadow-cljs. The only things it has to do is process JavaScript dependencies, and by reducing the amount of code it’s responsible for; you reduce the chance of issues. Further reduced if you use a versioned lock file; if you dependencies don’t change the code produced won’t change either.

The watch process doesn’t technically have to be a watch since it’s only taking care of dependencies. You could thus just have the file produced when shadow-cljs starts up (since you need to restart to load a new dependency usually anyway).

To summarize then, there is a very valid argument around the complexity introduced, but due to the way in which we use webpack I think that the pro can outweigh the con; though I don’t think this approach is needed in every ClojureScript project or even most projects.

Opt-in when you need it.

Performance

Another fair point on paper, the idea that introducing another compile step will increase the time it takes to compile is a no-brainer. More things = more time.

And yeah, I can’t refute that. But what I can claim is by how much.

So let’s run the tasks and see how long they take in isolation using my AMD Ryzen 7 7700X.

npx shadow-cljs compile main

shadow-cljs - config: /home/lkn/build/snippet-share/shadow-cljs.edn
[:main] Compiling ...
[:main] Build completed. (86 files, 0 compiled, 0 warnings, 1.24s)

1.24 seconds to produce the target/external.js file as well as the project’s code. Not bad.

Now for webpack

npx webpack-cli build --entry ./target/external.js --output-path resources/public/js/libs --target web --mode development

assets by chunk 390 KiB (id hint: vendors)
  asset vendors-node_modules_codemirror_legacy-modes_mode_sql_js.js 113 KiB [compared for emit] (id hint: vendors)
  asset vendors-node_modules_codemirror_legacy-modes_mode_css_js.js 102 KiB [compared for emit] (id hint: vendors)
  asset vendors-node_modules_codemirror_legacy-modes_mode_javascript_js.js 96.7 KiB [compared for emit] (id hint: vendors)
  asset vendors-node_modules_codemirror_legacy-modes_mode_clojure_js.js 42.4 KiB [compared for emit] (id hint: vendors)
  asset vendors-node_modules_codemirror_legacy-modes_mode_python_js.js 36.5 KiB [compared for emit] (id hint: vendors)
asset main.js 15.7 MiB [compared for emit] (name: main)
asset node_modules_codemirror_legacy-modes_mode_mllike_js.js 22.6 KiB [compared for emit]
asset node_modules_codemirror_legacy-modes_mode_ttcn-cfg_js.js 20.9 KiB [compared for emit]
asset node_modules_codemirror_legacy-modes_mode_asn1_js.js 19.7 KiB [compared for emit]
asset node_modules_codemirror_legacy-modes_mode_rpm_js.js 8.88 KiB [compared for emit]
runtime modules 8.08 KiB 12 modules
modules by path ./node_modules/ 6 MiB
  modules by path ./node_modules/highlight.js/lib/ 1.46 MiB 194 modules
  modules by path ./node_modules/@codemirror/ 1.89 MiB 125 modules
  modules by path ./node_modules/@lezer/ 763 KiB 16 modules
  modules by path ./node_modules/@uiw/ 44.9 KiB 11 modules
  modules by path ./node_modules/react/ 127 KiB 4 modules
  modules by path ./node_modules/@replit/ 224 KiB 4 modules
  modules by path ./node_modules/react-dom/ 1000 KiB 3 modules
  modules by path ./node_modules/react-refresh/ 20.6 KiB 2 modules
  modules by path ./node_modules/@babel/runtime/helpers/esm/*.js 764 bytes 2 modules
  modules by path ./node_modules/@nextjournal/ 15.9 KiB 2 modules
  modules by path ./node_modules/scheduler/ 17.3 KiB 2 modules
  + 10 modules
./target/external.js 986 bytes [built] [code generated]
webpack 5.89.0 compiled successfully in 1099 ms

Much noisier output, but it runs in about the same time. And that’s just the builds, any changes in both of these are then incremental builds than run in several milliseconds.

These aren’t isolated examples either, I ran these commands several times and replaced the times with the average (rather than having 3 code blocks).

If you’re spinning up the build process a lot, and a second is really important, then you’ll unfortunately have to look into alternative avenues.

Closing

I aimed to give some setup on using webpack with shadow-cljs, and some of you might be wondering why shadow doesn’t just use webpack to begin with. Well, this has already been answered by someone much smarter than me. Twice actually.

I encourage you to try this out on your own projects, but if you find yourself relying on more and more of the node ecosystem; you might be better off by just using JavaScript. It’s possible to import JavaScript directly, though I wouldn’t recommend this in production-level apps.

As with most things in software, use the best tool for the job.

References/footnotes