What is holding you back from upgrading to the latest OCaml compiler?

#1

Greetings ReasonML Developers,

The maintainers of core tools such dune, odoc, etc. are committed to supporting as many OCaml versions as possible. Unfortunately, this can be quite a burden on new development and a significant overhead on maintenance. To improve this situation, we’d like to have a better idea on when to drop support for older versions of OCaml, and just how far back our support should extend.

As a first step, we’ve started a thread in the OCaml discourse to gather data about which versions of the OCaml compiler are in active use today. I’m extending this exercise to this reason forum as well.

The maintainers team is particularly interested to hear from users who are unable to upgrade to recent versions of the compiler. If you’re using OCaml and you’re stuck with anything but the latest 3 versions, please share your experiences with us here.

#2

Well, as the thread in OCaml discourse server said, bucklescript is the only thing that’s holding me back from using newer version. But more seriously, even if 4.06 support comes out today, I wouldn’t have any incentives to upgrade it. Because bucklescript provides an alternative ecosystem that suits me with frontend development

#3

While this is true for some people, I don’t want to agree with it. Having BuckleScript be on a recent version of OCaml makes it easier to use OPAM packages with it (sure, you have to write the bsconfig.json, and people do).

#4

BuckleScript will fairly soon be on 4.06 going forward so really the only thing I see keeping people on 4.02.3 is if they’re using some PPXs and the PPX authors haven’t gotten around to upgrading/releasing them for 4.06.

#5

Is there a definite timeline for “fairly soon”? If it matches our timeline, the dune team could drop support for 4.02 around 3-6 months after BuckleScript transitions to 4.06.

#6

No definite timeline AFAIK. in a recent blog post, they encouraged experimenting with 4.06 in 6.x.x release branch but not recommending it for production usage.

They will tell us if 6.x.x is ready for production in another post

#7

Hi @anmonteiro, i’ve heard about writing bsconfig.json files before for opam packages, but I’m not entirely sure what it entails. Could you describe a bit how it works? In particular, I’m wondering the following: when you are in the bucklescript world and use opam packages, is dune involved at all? If dune is not involved at all, then dune dropping 4.02 support shouldn’t affect bucklescript users.

But on the other hand, it probably means that something could be improved so that you don’t have to write such bsconfig.json files by hand.

#8

What I meant was that having Dune support for 4.02 (or any old version that BuckleScript will decide to be on) means that people who maintain packages built with Dune can still make sure they’re compatible with that version of the compiler, even though BuckleScript would take care of building it.

To your point, though, I wonder if bsconfig.json files could be generated somehow, but I defer to people more experienced with BuckleScript here. I don’t think either the OPAM package or the dune files would have all the information that BuckleScript needs, but there could be an approximation.

#9

I think Elliot Cable would be the right person to ask, as he has ported quite a few opam packages over, e.g. https://github.com/ELLIOTTCABLE/bs-cmdliner . I’ll ping him on Discord.

Normally, opam packages declare the minimum required OCaml version, right? I’m thinking this can be used to check for BuckleScript compatibility?

#10

This is not what I meant, though. My point was that if Dune drops support for 4.02.3 now there’s a class of packages that is not known to support 4.02 and for which builds might fail.

#11

Oh, you mean the package itself might declare its minimum OCaml version 4.02.3 but since Dune dropped support, we can’t be sure because we can’t build it with that version?

#12

So, the dune language is completely versioned. This means that even if Dune dropped support for 4.02.3 now, a package could keep compatibility with OCaml 4.02.3 by not using new Dune features. We could even check that: if a package uses newer features but claims to be compatible with OCaml 4.02.3, then Dune would complain and ask the user to make a choice between 4.02.3 compatibility and using new Dune features.

However, I feel like it would be easier for Reason developers if Dune kept compatibility with 4.02.3 while the official OCaml version supported by bucklescript is still 4.02.3, as otherwise you would have to use an older version of Dune for which we couldn’t provide support for. We’ll check with Hongbo what are is plan regarding officially supporting 4.02.

@yawaramin indeed, I have seen Eliot’s activity on github related to building opam packages with bucklescript. I’ll check with him.

#13

'allo, fam.

Can you expand on exactly what Dune dropping support for 4.02 would entail? Unless Dune actually shipped that compatibility-shim that was discussed, and projects depend on that at runtime, I’m not clear on what support is being lost.

Meanwhile - yep, I can’t see this negatively affecting projects that build only with BuckleScript’s build-system, bsb; I suppose the only people negatively affected would be those of us supporting dual-build-system projects that need 4.02 for legacy (pre-6.0.0) BuckleScript projects. And that might literally be a set of just one person … me. :rofl:

Relatedly, though, does this mean the plans for Dune to support bsc directly (and thus possibly supplant bsb for some uses) are DOA? Because I was hella looking forward to that!

#14

It would just mean that new releases of Dune would no longer support 4.02. This is indeed not a huge deal. It simply means that if you want to keep an OCaml project compatible with 4.02, then you couldn’t use the latest Dune features, you’d have to stick with the feature set of the last version of Dune that supported 4.02. Dune makes it easy to do that, i.e. it errors out if you use a feature that is more recent that the version of dune written in the dune-project file.

Hmm, yh I suppose it does :confused: Well, the plan would have to move to the bucklescript based on OCaml 4.06.

#15

Sorry, I spoke unclearly. Let’s say I tried to use a post-support-drop Dune version in a 4.02 project; what exactly would break? Are you simply planning to stop addressing new issues that pop up, or is there a particular change y’all’re waiting to make that will immediately somehow immediately and completely prevent 4.02 projects from building with the newer Dunes? If the latter, is it something that will definitely affect all 4.02 projects, or a specific subset of 4.02 projects with ‘property X’ that newer Dunes will no longer be able to build?

While I’m here, something worth pointing out, is that JavaScripters are fairly averse to upgrading with abandon. Past experience (in an untyped language, with a hugely diverse selection of shared libraries that spread across a spectrum of quality …) has taught most of us to move carefully. I wouldn’t A. count on 6.0.0 being declared stable too soon (the ‘six months’ above sounds pretty accurate; although I’d consider that more of a lower-bound), nor B. count on all BuckleScript projects everywhere switching in any timely fashion. Backwards-compatibility is important to us, so 4.02-life is going to be a reality for us for quite a while!

That said, I don’t think losing Dune support is going to be a big problem for the sort of project stuck on 4.02. I suspect that most projects that care about Dune, at all, are libraries — and care about 4.02 transitively, not directly; i.e. they themselves need to test on and support 4.02 for the sake of their users. If they’re holding back from new OCaml features to test on 4.02, they can probably hold back on new Dune features, too. ¯\_(ツ)_/¯

#16

Well, there are few places in Dune’s code that look like this:

if <compiling for OCaml 4.02> then
  <do this to workaround a bug in 4.02>
else
   ...

These are additional things to think about, but there are not a huge burden so we could keep them in the code. Currently, we test every PR against several versions of OCaml, including 4.02.3. This gives us good confidence that Dune can build with 4.02.3 and can build projects with 4.02.3. Dropping support for 4.02.3 in Dune would mean stopping this testing and then we wouldn’t notice when we break the 4.02.3 world.

I completely agree with you regarding backward compatibility. That’s why we completely versioned the Dune language. The latest version of Dune is able to understand projects written against all past versions of Dune and Jbuilder for instance, even though we changed quite a lot of things since the original release of Jbuilder.

Anyway, the most frustrating aspect of supporting old compilers for us is that we can’t use new OCaml features in Dune itself. Dune can build projects using a different version of OCaml it was compiled against, so in theory we could require OCaml 4.08 to build Dune and use the resulting binary to build projects using any version of OCaml. That would actually be ideal for us, except that this is not supported at all by opam :frowning:

#17

Well, notably, the thing we’re discussing here doesn’t demand opam-support, does it? You could publish to opam indicating >= 4.03, and require that to build; but continue to support building projects that use 4.02 — as long as the dune-binary is prebuilt. Given that soon-to-be-legacy BuckleScript projects are using an external compiler, it’s perfectly fine if opam installs 4.08 and builds Dune using 4.08!

Biggest problems I see with that approach: complicated documentation (“depending on how you look at it, we support either 4.02+ or 4.03+ …”), and complicated testing (your aforementioned opam-constraint-constraints come into play when you’re trying to get opam to build against 4.0X and run the damn tests against 4.02 … ruh-roh.)

Anyway, I’m gonna dip out of this thread, as it seems like you have the information you need! Also v impressed with y’all’s attention-to-detail, here. <3

Edit: I would add that decoupling buildtime-requirements and runtime-requirements might be a good idea moving forward — it’s just so much easier to drop older versions from the former, and that’d theoretically give you a lot more freedom to use even newer features than 4.03’s. Maybe worth opening an issue against opam, to give you the freedom to include that dualistic support-matrix in the opam constraints?