Intersection with OCaml Ecosystem


#1

Hi everyone!

I’ve been using ReasonML for about a year and really love it.
Lately, I started working on a side project (native app) and decided to do it in ReasonML/OCaml. When comparing the syntaxes, I preferred OCaml and thus wanted to try out the syntax instead of ReasonML.

As mentioned almost everywhere, “ReasonML is just a syntax for OCaml” so I thought switching the syntax would be a breeze, but I am starting to realize that there is actually a huge gap between ReasonML and OCaml ecosystems:

  • The package managers are not the same (Esy for ReasonML, Opam for OCaml)
  • The libraries are not the same (e.g. Rely for ReasonML, Alcotest or Ounit for OCaml)
  • The standard libraries are not the same (Real World OCaml and a lot of OCaml repos use Janestreet’s standard library, I haven’t seen any repo in ReasonML using it)
  • The Discourse, Discord, Websites, Resources are all different.

Although ReasonML is just a syntax for OCaml, choosing one or the other is actually choosing two completely different ecosystems.
Of course, there are exceptions: the (really awesome!) graphql_ppx is used by both ReasonML and OCaml ecosystems, but I feel like it’s more of an exception than the norm.

I’d really like to know if this separation is voluntary or maybe a side-effect of another decision (e.g. building libraries that are more familiar to JS developers).

It would be totally legitimate for the separation to be voluntary. I would not agree with it, but if it’s the direction of ReasonML, it is what it is.
If it is not, however, I think the community would benefit from having a plan to unify the ecosystems.

  • Do you think OCaml and ReasonML ecosystems should unify?
  • If so, what can be done to go in this direction?

#2

I think there is a misunderstanding here. Reason is a syntax, toolchain, and community for OCaml. Not just syntax. You can use OCaml syntax with Reason tooling like Esy, libraries like Rely, and of course the JS compiler BuckleScript. Also nothing stops you from using OCaml libraries like Jane Street libraries, with Esy. Esy and Reason are backward-compatible with pretty much the entire OCaml ecosystem.

There are various good reasons to have a distinct Reason ecosystem and community from the OCaml one, but there is no artificial barrier between them at least in the direction of Reason stuff using OCaml stuff.


#3

Indeed! To add to @yawaramin’s points

  1. Opam can be used with Reason projects. In fact, during my early days I used opam for a native Reason project at work. (Reasons why I switched to esy are irrelevant here, but would love to discuss)

  2. As mentioned, you can mix and match the testing libraries. Infact, I love expect_ppx by Janestreet for my unit tests in a Reason codebase

  3. Standard libraries: this an interesting area there are a lot of competing libraries out there! Nevertheless, they can all be used to Reason. Unfortunately, I don’t have an example to show Base/Stdio from Janestreet used with OCaml. For pesy, I used tablecloth that works with bucklescript and native Reason/OCaml

  4. Website: primarily because of syntax and priority of concerns

I have mention this incident: a couple of months back, opam changed it’s directory layout. It was working great for the opam ecosystem, but broke esy and it’s consumer. One of the maintainers reverted the commit and waited for things to stabilise for esy. Unfortunately, I couldn’t find the github issue or the twitter link.

Also, dune supports Reason syntax out of the box! There are many things like that’ll go to show that the two communities, despite different approaches and priorities want to work closely :slight_smile:


#4

Agree that there are no artificial barrier to mix the two ecosystems, which is part of the reason the separation is confusing to me.

For instance, nothing is preventing me from using Rely with OCaml, but it would be a very odd choice given that literally no one do this:

  • There would be a translation overhead every time I read the doc, or read people’s code
  • It would create inconsistencies with the community practices, which I think would be detrimental.

To me, the separation of the ecosystem is a pity because it prevents the OCaml ecosystem from benefiting from advances of the ReasonML ecosystem, and inversely.
For instance, Esy is an amazing project and I think lots of OCaml folks would benefit from using it, but I have never seen any OCaml repository use it.
Not that there is a barrier, but it’s just not part of the community practices. As you point out, @prometheansacrifice, before Esy people were using Opam for native development, but the community made a switch. The same applies to Rely. And by doing so, it distanced itself from the OCaml ecosystem.

There is clearly some initiatives to bridge the two ecosystems (dune, odoc, etc.), but there are also lots of decisions that widen the gap between the two, which, again, is totally fine if this is where the community wants to go, but if not, we might need to build more bridges before the gap is too large.

Here are some ideas:

  • Using Core as the recommended native ReasonML standard library.
  • Expose OCaml community to Esy and see if this could become a recommended package manager for OCaml.
  • Have a common Discord.
  • Contributing to existing libraries instead of creating new ones (dbuenzli/logs vs reason-native/console, mirage/alcotest vs reason-native/rely)

Thanks for taking the time to answer, really appreciate getting people’s insights on this!


#5

It would create inconsistencies with the community practices, which I think would be detrimental.

Fair point. This can be likened to the JS ecosystem - when there are a lot of options, it looks like there will be inconsistencies. A vague guideline I follow - if the target audience is some one from OCaml, use the tools from there. If they are both, use both! Example H2 It can be used with both Esy and Opam. Also, a good example for a case that an OCaml project that uses esy

before Esy people were using Opam for native development

Not sure about this. But I should clarify, that I did not intend on implying this. It could be true though. Earlier there I was citing a personal example - at work I was using Reason for a native project and decided to use opam because it was more mature. I switched later on for reproducibility issues.

Using Core as the recommended native ReasonML standard library.

I’m not sure even OCaml users will agree to this. A common complain I have heard about the RWO book it that it uses too many Core examples. To my understanding, there are a lot of good competing stdlibs/containers out there.

Expose OCaml community to Esy and see if this could become a recommended package manager for OCaml.

To an extent OCaml users are aware of Esy. I love what you are suggesting. At the same time, I also see a lot of questions/try outs at #esy channel on the OCaml servers. So I believe, to an extent this is already happening. Also, I’m not of the opinion that everyone should switch - opam and related tooling has been around for some time now. Older (as in experienced) users are quite comfortable with it and have adapted their setup to it. Those coming from JS background might find esy convenient. (Similar to yarn/npm divide? Maybe I’m too off)

Have a common Discord.

I love this idea! Because infact, on Reason channels, experienced OCaml user’s come and help all the time! Which btw I’m super thankful for them for!

Contributing to existing libraries instead of creating new ones (dbuenzli/logs vs reason-native/console, mirage/alcotest vs reason-native/rely)

I think I disagree here.
I love dbuenzli’s libraries today. But when I started, I had a really hard time using those libraries. Application operators, combinators, monadic patterns confused me a lot. So, I’m fine with alternatives. Anything that helps newcomers get started quickly.


#6

Esy is very, very new. It hasn’t even reached v1 yet :slight_smile:

And, it’s really difficult to get OCaml folks to change their build tooling. There are still people who don’t want to use dune, because they’re using make or something.

Plus the fact that it requires npm to install right now, may not make it very attractive to pure OCaml devs.

But I’m sure these issues can be overcome in time. I think we can benefit from Esy right now while waiting for the uptake that will come in time.


#7

Shameless plug, we’re looking for contributors :smiley: Would love some help in getting to v1

Also, agreeing with every point made by @yawaramin


#8

For the “split” with Rely, I think you’re seeing a couple of things:

  1. Syntax is more important than you initially would expect because it even influences the kinds of APIs that would be more readable. Rely’s API is designed to have a callback as the final argument, and the community has even built special formatting for this use case - and this matches what the JS community is used to with Jest and Prettier.
  2. Reason appeals to the JS community even when it is supporting the native toolchains. So some of the libraries we’ve developed and released for native, appeal to a different audience than existing OCaml programmers. The existing libraries and testing frameworks in the OCaml ecosystem are great. But by making everything as familiar as possible within the Reason libraries and toolchains, it acts as a way to get more people exposed to the language, because the JS ecosystem is where all the developers are. (I would recommend making things as frictionless and superficially familiar as possible - but not to the extent that the technology is worse of course).

For Console: Console does something different than the logs library you mentioned. Console is designed to be equivalent to the console.log of node/browsers, and is totally polymorphic, requiring no ceremony or static typing. It’s really just for log debugging. But again, it is designed to be immediately familiar to the largest number of developers - and coming from the JS community.

For esy: Like the others, esy is designed to be immediately usable by the largest number of developers (and they happen to come from the JS community). It’s true that we need to make it installable from places other than npm, and it would be great to get more people from the OCaml community to check it out and contribute. Many have already contributed feedback or fixes!

One thing that we could and should do better, is release more of the packages we’ve made with esy, onto the opam repository (once we’ve tested them sufficiently). We’ve made some progress here, and I’ve created some build steps that automatically generate opam files in the reason-native libraries.
My original assumption was that developers in the OCaml ecosystem were content with the opam workflow (especially since it does so many great things with CI and its testing matrix). Npm developers have a higher expectation of project isolation. But I have been pleasantly surprised that many OCaml programmers have appreciated esy as well. Some of the projects you see on Github might have opam packages, but their contributors might still be using esy because esy can build opam projects too.

Docs is one place where the difference is not intentional - we just need a little more help. For example, is anyone willing to help rebuild the OCaml manual/standard library in Reason syntax? That would be helpful - please get in touch if anyone’s interested. (It’s an automated process that is generated via a build - but we need to inject the reason parser/top level into it).

Having separate Discourse/Discord has some benefits in that Reason has a lot of newcomers to the language, and a lot of them have large overlap with the compile-to-js workflow (BuckleScript). There are a lot of cross-discussions across the different Discord servers though.

Similar to esy, I would also like to point out one amazing tool built by someone in the Reason community that integrates well with both Reason and OCaml: sketch.sh.

I think if you dig a little deeper, you’ll come to the realization that the Reason ecosystem in the native world, is actually not so far diverged from the OCaml native ecosystem on all the important parts. Andrey Popp (architect of esy) went to great lengths to make esy support opam packages for example. Dune works with Reason (as mentioned by others). For everything else that seems like an unnecessary difference, I think it’s best to ask whether or not that difference accomplishes the goal of appealing to a wider audience(which means JS) without compromising on quality. If the answer is “no it doesn’t help with wider reach”, then we should probably revisit some things and do a better job of integrating with the OCaml ecosystem. If the answer is “yes”, I’d say it’s worth it - but that’s just me.


#9

Really good and inspiring points, thanks for writing this up!

My original assumption was that developers in the OCaml ecosystem were content with the opam workflow

I must say, Esy, https://reason-native.com/ libs, Revery, and others in Reason Native ecosystem especially shine for application development. When I’m writing a library I think I would still prefer to publish to Opam where possible (or simply use git+https://...), I don’t totally see the advantage of npm registry when Esy works so well to tie all of this together when building products. Are there any advantages to npm registry you could think of when in the end you’re going to be using Esy? For example, the publishing workflow?

I’m also curious about what the end goal for you would be, if not limited by available resources or time.

  • Would I use Esy instead of both npm & opam, and build multi platform apps with it? Right now it can (and does) replace opam tool, but I don’t know if I should start considering it as a replacement for npm & yarn as well.

  • Is Dune a good fit and is it something Esy team intends to treat as the main build tool for the foreseeable future?
    If so, is it likely that at some point we’ll be able to build software exactly like you outlined, but with BuckleScript as one of the compilers supported by Dune? Similar to how js_of_ocaml user experience is right now.


#10

Esy doesn’t make you use the npm registry. It’s just that npm registry is the most frictionless way to publish versioned packages. No one needs to accept your request to publish to npm. On the down side, publishing packages there doesn’t give you the same kind of stability guarantees because it lacks opam’s (awesome) matrix builds for reverse dependencies. I think npm is a great place to publish new packages that you’re still experimenting with and want rapid iteration for. There’s other reasons why it’s good that esy can pull from npm even if you don’t actually publish to npm. You can depend on existing JS packages.
Please feel free to publish to opam. It’s great too.

Would I use Esy instead of both npm & opam, and build multi platform apps with it? Right now it can (and does) replace opam tool, but I don’t know if I should start considering it as a replacement for npm & yarn as well.

Esy does pull regular JS dependencies from npm repository as well, but it is currently limited to using yarn’s pnp style dependency loader. We would like to lift that restriction in the future, but we could use some assistance and testing.

Is Dune a good fit and is it something Esy team intends to treat as the main build tool for the foreseeable future?

Dune is a great build system, but esy isn’t just for Reason and OCaml - it also builds C programs and dependencies. So really there’s two concerns going on here. One is pulling down packages and resolving dependencies (that’s what esy does). The other is actually building each package and that is a concern that is build system specific (that’s what dune does). However, Dune has provided some great hooks for esy to be able to perform out of source builds (which is one thing that empowers its awesome cross-project caching feature). Also, pesy is a project written by @prometheansacrifice that creates an esy driven workflow that generates Dune files for your project in an npm require style (feedback welcomed).

like you outlined, but with BuckleScript as one of the compilers supported by Dune?

That’s just a completely different topic. Right now, esy is native-first, which means everything native should work and jsoo is currently the compile to javascript tool that works in native first workflows. Being able to consume packages from Bucklescript when those packages were authored in native-first workflows (not bucklescript) is hard because Bucklescript wants all of its dependencies to be written with Bucklescript. Instead of the more difficult task of making Buckelscript work with things that weren’t written in Bucklescript, we have instead experimented with allowing esy to power pure Bucklescript workflows. Help appreciated there too. It won’t make it so that you can mix and match Bucklescript packages with native-first packages that used Dune, but it will allow you to have one package manager that can power two separate workflows - one for native first, and one for Bucklescript-exclusive.


#11

I’ve recently written up a doc called navigating-reason which helps explain what the various tools are, and how they might overlap. This can serve as a way to coordinate consolidation of any tools where possible.


#12

I’m willing, not sure if I’m qualified, having next to zero experience with OCaml toolchain outside of using bs-platform and utop (for exercises). But anyway, I don’t mind learning a few things, and by the by, I’m specifically interested in rtop: I was wondering how to get Belt working there, and I’m curious about repl-driven development (not sure if it’s all that relevant when we have type-driven development, but).

So anyway, why not :slight_smile:. Should I PM or mail someone? Should I come to the Discord channel?


#13

Concerning the OCaml’s user manual, if there is any help needed on the OCaml side, please don’t hesitate to contact me. Also a quite recent good news is that the manual has been relicensed to Creative Commons Attribution-Sharealike (4.0), which may help translations and redistribution.


#14

Yes, everyone please ping me on Discord. The process for building the manual is a bit of a pain because it uses the top level to generate results in the document. If we can just get it to be build as part of Reason’s CI steps, it could automate everything and produce a pdf manual and website for each commit.

Once we get Reason plugged into the manual, then I could also imagine a sketch powered interactive manual as a second step!

As for rtop interactive top level. I recently landed the diff that will generate a package on every commit!
Click on the Azure pipelines badge, then go to recent builds for master, then click on artifacts, then download the release:

Extract the contents and then run npm install -g ./Release to get a global rtop.
To help out, you can help us make it so that Azure publishes the format in tar.gz instead of .zip, and then somehow get the url of the latest release artifact to be printed in the build log, or even published as a badge on the main README of Reason. That way people don’t have to download it and can just do npm install -g that-url-to-tar.gz.


#15

Thanks. I hope I’ll get back to you once I figure out how the whole building process even works :slight_smile: