How to build with esy without relying on npm for distribution?

#1

Hello.
Today I decided to go esy a go. I just cloned the hello-reason repository, build, executed with esy x Hello and everything went fine.
The problem is that I can not find any reference in how to distribute the resulting binary without npm. In fact, i can not even run the generated binary under _release folder.
Is there a way that I can build a standalone binary that can be run from anywhere?
Thanks in advance

#2

What command are you issuing, in what setup, and what happens when you do it?

Apart from distribution, your project (or binary) might not depend on npm in the first place — for example, if it depends only on packages from opam, or only on packages specified by repository URLs (or slugs).

If you are targeting native code, there is no way to build one binary that will run on all platforms and architectures. However, for each platform and architecture separately, the binary built by esy should already be standalone (except in the rare case where you depend on some system library that is dynamically linked; you will usually know about that). You should be able to simply copy out the built binary and distribute that. You have to do that once per platform.

For reference, in one of my packages, which I distribute through npm, I include prebuilt binaries for Linux and Mac, x86_64. The installer script checks the platform and whether it can run one of the prebuilt binaries. If not, the installer builds binaries from source. This should give some idea of the impossibility of a binary running everywhere, and one strategy for distribution despite that.

#3

Sorry if my post was not clear enough. Of course I’m not trying to say that I want the same executable for all platforms, I’m not that naive :smile:
What I mean is that, the only installation method explained by esy is by deploying to npm and then doing npm i yourpackage. I don’t want that, I want to be able to, for example, download the binary for your platform and run it.

I’m just executing the build steps described by esy:

esy build
esy npm-release

then, for executing just run ./_release/bin/Hello and get the following:

Fatal error: exception Unix.Unix_error(Unix.ENOENT, "execve", "/Users/danielo/hello-reason/_release/3/i/hello_reason-1e09868a/bin/Hello")

Thanks for you ranswer

#4

Ah, I’ve never used esy npm-release (I generally package everything manually). You can find the binary after esy build. It should be at _build/default/bin/Hello or _build/default/bin/Hello.exe, or so.

So, I would recommend to do esy build, take the binary out, and then use any distribution method you want. Examples are:

  • npm with prebuilt binaries, as mentioned before
  • tell people to depend on your repo directly, perhaps on a branch that has prebuilt binaries available, with a build system that checks for the existence of these binaries
  • upload as artifacts to GitHub
  • etc.

So, perhaps part of the answer is that esy is not itself a distribution method. You need to use opam, npm, GitHub repo URLs, GitHub artifacts, AWS, ordinary web pages, etc., for distribution, and you may need to write an installer script, depending on what you are doing. esy does make it easy to depend on packages in most of these, however.

#5

Ok, now I see how my original title was confusing.
My actual question would be something like “how to build distributable binaries with esy” but I let the esy docs make me think the only way was using npm and hence I adapted my title for that.
So, I was not expecting esy to be any kind of distribution system, but just a way to build binaries for the platforms I may want. On the esy docs there is no mention to build artifacts, and every tutorial out there relies on npm for even installing the local output of the build, that’s where my confusion comes from.
Now, I realized about a file named hello-reason.install where the actual generated paths are. In myc ase, the output is way more convoluted and obscure than what you wrote:
hello-reason/_esy/default/store/b/hello_reason-a85a496f/install/default/bin/Hello
I understand that such information is used when you distribute through npm, but I would appreciate an easier way to just consume the build artifacts locally or to use an alternative/manual distribution system.
I tried copying the executable to other places on my system and it worked perfectly, so I guess that is the way to go.
Thank you for your help.

#6

In case you want a more predictable path for automation or other purposes, you can add "buildsInSource": "_build" to your package.json/esy.json. Then you will get a local _build directory, and the binary will be somewhere in there, either at a path like the one I suggested, or somewhere around _build/install/default/bin/Hello.

#7

That did the trick!
Awesome!
Now it’s time to dig into more native stuff.
If you want to point me to any good resources for CLI tools I will read them all, but I got enough help from you today, so thanks anyway.

Regards

#8

The main thing I’d suggest is using Cmdliner for the command line (you may already be aware of it). It’ll give you nice help output, etc. AFAIK esy uses it, as does Dune. You can pull it in as"@opam/cmdliner".

#9

I didn’t knew about it. Thanks! I guess that, in order to do anything useful in reason-native I will have to rely on opam packages for almost everything.

#10

Yes, opam remains overwhelmingly the source of native packages.

Also, there is an incentive for new packages to publish in opam, if they are relevant to the rest of OCaml, since both the opam front-end and esy can consume the opam package repository.

It actually works out nicely with “isomorphic” packages that can be used for both reason-native and BuckleScript. You can publish the native version to opam, and the JS version to npm.

Of course, you can still always publish a single isomorphic package to npm, because package.json or esy.json will be chosen for the build by npm and esy respectively, if you have a separate esy.json, so you can issue different commands according to what is consuming the package.