Ocaml-migrate-parsetree for bucklescript

#1

Is there anyone who ported ocaml-migrate-parsetree (omp) to bucklescript? I am investigating the usage of ppx that are common in the native world but with bucklescript. Most of the native ppx now depend on omp. So it would be very convenient to be able to compile it with bucklescript.

Then the same question would apply for ppxlib, but it seems to support only “recent” versions of ocaml. So it will have to wait for the port of bs to 4.06.

#2

I’ve been trying to make omp work with BuckleScript, but got stuck early in the process. I’m documenting it here with the hope it helps others, or someone can point out what I’m doing wrong.

The first thing I couldn’t make work is the preprocessing part of omp, that removes the (*IF_CURRENT *) comments in the Ast_40x.ml file that corresponds with the version of OCaml being used.

I managed to make bsb use omp pp binary with

  "ppx-flags": [
    "./tools/pp.exe 4.02"
  ],

But pp.exe is expecting to have the filename passed as argument, and with BuckleScript I only get arguments like /var/folders/ss/97gmv_jn031f29dsvb600f280000gn/T/camlppxcbb8b1, which doesn’t contain the original file name. I’ve tried several things but not sure how to fix this. :thinking:

The other main problem that I haven’t been able to solve is how to import compiler-libs in a BuckleScript project. I have tried hacking around it by adding:

    "sources": [{
      "dir": "node_modules/bs-platform/vendor/ocaml/parsing",
      "files": [
        "ast_helper.mli",
        "ast_helper.ml",
        "asttypes.mli",
        "docstrings.mli",
        "docstrings.ml",
        "location.mli",
        "location.ml",
        "longident.mli",
        "parsetree.mli"
      ]
    }]

to bsconfig. But this just keeps erroring because dependencies are missing, until I reach a point where it asks for config.ml but the vendor/ocaml/utils folder has a config.mlp which seems to need some preprocessing as well.

There must be easier ways to accomplish this, I wonder if @bobzhang has any ideas?

I’ve also been thinking this would be much easier if dune could be used with a BuckleScript project. I found out there is some work being done by @zploskey in https://github.com/zploskey/hello-esy-dune-bsb/ that seems to be starting, but is very promising :slight_smile: See also https://github.com/ocaml/dune/issues/140.

#3

The fundamental thing is that if you have a ppx, what’s your plan to distribute it via NPM

#4

One could use esy + the ppx sources (that’s what I did when experimenting with ppx_bs_css for example). Or the ppx author can generate binaries for each platform, like graphql_ppx does (builds binaries on travis/appveyor and bundles them into a single npm package).

But there are cases where omp can be needed as a dependency, and they don’t even relate to ppxs. For example, with rebind I was generating a Parsetree AST and I wanted to use refmt as a library to emit the Reason code in the browser. The only way to do so currently is with jsoo (because of the dependency of refmt on omp), but I would rather use BuckleScript.

#5

Then we are mostly ignoring Windows users which is not the direction I wish we advance…

#6

There are ways to generate Windows binaries.

graphql_ppx uses appveyor to create Win binaries: https://github.com/mhallin/graphql_ppx/blob/master/appveyor.yml#L26

genType does something similar at well to provide Windows binaries.

I’m not suggesting at all to ignore Windows users. I must be missing something, because I don’t see the connection between binaries distribution and the compilation of omp in BuckleScript? :slight_smile: As mentioned above, compiling omp is useful for more cases besides ppxs. And even if it was, I think binaries distribution is something that would need to be solved after figuring out how to actually build omp in BuckleScript for at least one platform.

#7

Ah, I see you are trying to use compiler-libs in the browser, that is indeed more convenient to use jsoo.
What I did before is using bspack to pack all the compiler-lib files into a single file so that I can use bsc to compile it, but it is less convenient. Currently if you have a large dependency on existing ocaml libs, jsoo maybe make your life easier

#8

Thanks @bobzhang. Do you have any public repo where that packing with bspack is shown? It would help me a lot to understand how it can be done.

Also, do you think it would be feasible in the future to have an option in BuckleScript so users can include compiler-libs in bsconfig.json, and somehow have the compiler expose the required files in that case?

#9

Here is a test I wrote: https://github.com/BuckleScript/bucklescript/blob/master/jscomp/test/Makefile#L327
But it is not recommended to use more than testing purpose…

#10

@bobzhang Thanks. I have thought more about this approach (bundling all the compiler libs and exposing them separately) and I’m not sure about the downsides. Ideally the exposed compiler modules would be the same as the ones used by bsc, right? It seems to me that having a separate module could lead to issues of duplicated types, etc.

I have created a repo to share the approach I was trying, and I mentioned above (include node_modules/bs-platform/vendor in bsconfig). I have it almost compiling, except for this config.ml file, which seems to need some pre-processing but I’m not sure how / what tool I should use:

Do you know if this approach could make sense, and how I could generate the missing config.ml file?

#11

I got the project to compile by vendoring the config.ml file produced after compiling the original bucklescript repo: https://github.com/jchavarri/explore-bs-compiler-libs/commit/06f40b2641949bf88761b9ee62a9bbded0ff9c73

The next step would be to fork ocaml-migrate-parsetree to include a bsconfig.json file so it can be added as a dependency, include it in the project, and see what happens… :slight_smile: