Hi! As I posted elsewhere, this also bit me really hard. Let me see if I can offer my viewpoint on this issue — and OP is definitely not alone in his pain, here; in fact, I’m pretty sure everybody using BuckleScript to write libraries is probably feeling some of this pain (and I’d guess that the only reason it’s only showing up in these discussions now, is that Reason is really disproportionately being used for “apps” and other end-user products at the moment.)
So, since y’all askd for input on Discord regarding this and use-cases you don’t understand; in no particular order, here are my thoughts:
I, personally, absolutely did not assume BuckleScript would maintain ABI-compatibility between major versions, or even a modicum of that. The team is way too small for that! That’s not the point or argument really worth discussing in this thread, at least in-my-humble-opinion. Totally happy with some of the ABI-breaking changes, like the 5.2 modules-as-objects, and the 7.0 records-as-objects. Awesome work!
As with @ryb73, I found it very surprising behaviour that the compiler sometimes takes into account compilation-artifacts shipped with npm packages: I completely assumed that …
either consuming-clients’ compilers would fail at runtime if artifacts were missing from the
node_modulestree (i.e. the fact that
require("bs-thingie/thingie.bs.js")shows up in the output, implied to me that BuckleScript treated dependencies as black-boxes)
or the consumer’s compiler would completely ignore
.bs.jsfiles shipped in npm modules, and build all intermediate
bs-dependencies’ ML source-code into
Personally, despite now understanding better how this works, I still think this behaviour is surprising. It may be too late to change this, but I believe one of the two above behaviours is far more intuitive: either always expect ML-build-products to be pre-compiled, or completely ignore them and build the entire tree.
Similarly, speaking of surprising behaviour, I am shocked to discover that
bsbwill write into node_modules. Wow. Please don’t do that! If building is necessary, copy tertiary files into
lib/bsor similar, and produce build-products there.
(Hey, this part is actually something I can help with!) Again, same as @ryb73, a big reason that all of this tripped me up, is the lack of documentation catering to library/module-authors, and more details about build-system-npm-interop. Once I understand best-practices for this, I think it’ll be a very good usage of my time to go write that page, heh, and submit a pull-request to the documentation-site. That said, some of the above needs to be addressed before any of this really makes sense to document, you know? It’s pretty broken, at the moment.
Let me explain, first in the abstract, and then by presenting one of the package-trees that the v5.2 release broke for me, in the real world, in an unfixable way.
Our primary product is the
bs-thingieshows up in the npm
cool-app— they aren’t involved with our team, have no idea BuckleScript exists, and expect
task-doerto Just Work.
As things currently stand,
.bs.jsfiles, but the downstream consumers don’t care about that); that source-code includes lines like:
var Thingie = require("bs-thingie/thingie.bs.js");
Of course, this is all fine and dandy, because
task-doerhas correctly declared its’ dependency upon
bs-thingie/was downloaded and unpacked by npm, and contains
thingie.bs.js, itself compiled by the author thereof, archived, and published to npm. Our client,
cool-app, can just
require("task-doer")like any other npm library, and away we go, everything is fine … riiiiiiiight up until v5.2 or v7.0 of BuckleScript lands, and either the we (the authors of
task-doer) or the authors of
bs-thingieupgrade our BuckleScript versions. (Obviously! Yes, that’s how ABIs work, I know.)
The above suggestions boil down to ‘solving’ this situation in, basically, one of two ways:
.npmignoreor similar. However, doing so means every consumer, anywhere in the npm dependency-tree, needs to install a version of
bs-platform, and somehow work it into their build-system: in our above example, this would involve informing
cool-js-library? They have to do the same thing to their consumers!)
task-doerauthors would, in this case, basically need to remove
bs-thingieand all of their other dependencies from the actual semantic dependency-tree, and add build-tooling outside of
Both of these have huge downsides, obviously, and ones I won’t get into here; the downsides of vendoring are well-studied elsewhere, as is the breaking of dependency encapsulation. In particular both are absolutely untenable for my own use-cases — let’s study one in particular.
I publish a library,
bs-uchar(details unimportant; but it shims the
Uchar.ttype in a version-independent manner, to help library-authors publish code that’s usable on both BuckleScript v4/5, and v6/7.) This is intended for wider public consumption, as is all of my work; it must stay a stable product that isn’t tightly coupled to my other work.
Next, I’ve a much more involved effort,
bs-uchar, for obvious reasons; again, this is intended for general public consumption, and must not be tightly coupled to the above or following.
The last of my own involved projects,
excmd.js(a parser library) depends both on
bs-uchardirectly, as well as on
Finally, the downstream project to which I am contributing here, Tridactyl (a Vi-mode total-interface-overhaul for FireFox) depends on
excmd.js, and has no idea that BuckleScript exists.
Let’s look at what the two existing solutions would involve, to implement in my own projects:
.bs.jsfiles from any involved library that I control; and then convince the Tridactyl developers — who don’t care, and shouldn’t! — to add BuckleScript to their (already mind-bendingly complicated, as it turns out) build-system. (I’m not even sure how you’d invoke
bsbto compile dependencies in
.bs.jsfiles that any additional downstream consumers of
excmd.jswould be taking on the responsibility to build.
excmd.js. So, to implement this solution, I’d have to move all of the above-mentioned dependencies into
devDependenciesor similar. Then, add a Rollup stage to my build-system, and bake a single
lib.jsfile or similar, post-BuckleScript
bs -make-worldcompilation, with every dependency vendored into it, into my npm archive. (I am, of course, now throwing away all of the advantages of npm and SemVer, of a dynamic dependency tree, of semantic security-analysis, of Yarn resolutions, and making debugging a nightmare for my consumers. Woo! Vendoring deps!)
Neither of these are great options, in ways that I hope are, by this point, obvious!
This got really long-winded; so I’m gonna cut it off here — but, well, I clearly have quite a lot to say on the topic. My biggest concern is that there may not be much we can do about this, even if y’all accept my point that it’s a big problem; it may simply be too late.