BuckleScript 8.1: new syntax option

#61

Good thoughts @idkjs.
My two cents:
It wasn’t clear from the Blog post title (hopefully that can be fixed), but this new syntax is an opt in syntax specifically for BuckleScript, and it is distinct from Reason syntax. The Reason Syntax will continue to support the OCaml ecosystem, and I believe in the future, having a different file extension for a BuckleScript specific syntax/parser, vs. the Reason Syntax, will allow the toolchains to manage expectations better regarding what works in which environments. I also believe it will allow Reason Syntax to do a better job of being compatible with the OCaml ecosystem than it already does, and could also allow the BuckleScript specific syntax to do a better job of being compatible with the BuckleScript ecosystem. Right now, there is already a fragmentation, but it isn’t clear from the docs or file extension - so you have fragmentation plus confusion. I believe that if we do this well, it could be an opportunity to remove a lot of the confusion, and provide room to improve quality for both workflows independently(OCaml ecosystem compatible and BS ecosystem compatible), even if it doesn’t address the fragmentation directly.

#62

So it sounds like you will be able to use the old syntax with Bucklescript. Is that the case?

#63

You can ask @bobzhang about that, but from what I understand this is not a forced immediate migration. I bet the fact that the BuckleScript syntax has a different file extension really opens up options (otherwise it would be a pretty invasive change that would make it hard not to break people without forcing an upgrade).

#64

Ultimately, there is a fragmentation in direction and leadership. BuckleScript is going in one direction, ReasonML another, and OCaml another. I am not upset about this, more disappointed that leadership are not on the same page moving in the same direction.

#65

I’ll try to organize my thoughts on this.

First of all, this is an amazing feat of engineering. The parts I like best:

  • Amazing syntax errors, especially things like properly warning that React component parameter names start with ~
  • Simplified syntax, especially attributes which make FFI bindings look almost simple
  • And, overall, the sheer impressiveness of being a tiny outfit and trying to seriously go after TypeScript.

That said, I do have some feedback and questions.

  • As others have pointed out, .res syntax is increasing the fragmentation in the community, which is already fragmented between OCaml/Reason/BuckleScript/native. Paul Biggar ironically talked about this fragmentation issue at ReasonConf US. Looks like we’re not over the hump yet.
  • Allowing only a few infix ops, meaning we know that both the parser and the base language support all infix ops in general but we’re not allowed to use them, is a difficult sell for me. Languages should not impose arbitrary restrictions.
  • It’s not clear what the long-term maintainability of this new syntax is. Based on what we know today–I’m not sure how sustainable it is as a separate project. If the upcoming Res syntax converter works both ways with high fidelity, that will be more reassuring. People need an escape hatch. That’s what we tell prospective users about BuckleScript from day one–if you want to eject, check in the JS output and delete bs-platform.
  • Will BuckleScript continue to upgrade to new Reason Syntax versions as those are released (e.g. bugfixes, new features), or is it frozen at the current version of Reason Syntax?

Overall, I get that this is an exciting direction for BuckleScript. As Jordan made clear though, this is not Reason Syntax. It is great to experiment with it, but I think everyone agrees we should not be productionizing it right now.

#66

I agree with @yawaramin. What I don’t understand is why it needs to be separate from the Reason parser and printer. From the surface of it it seems like this can all be part of the v4 of the reason syntax. But probably it has to do with a next iteration that is more specific to Bucklescript?

It would be great to be able to share some compiler code so Reason will benefit from the better errors etc.

#67

Hi, thanks for the feedback.
Note that I am not involved in the early phase of the new syntax, but I did the integration, from the technical point of view, it is much easier to integrate due to zero dependencies for the new syntax, and it is a hand written parser so that it is easier to tweak high quality error messages.

As others have pointed out, .res syntax is increasing the fragmentation in the community, which is already fragmented between OCaml/Reason/BuckleScript/native. Paul Biggar ironically talked about this fragmentation issue at ReasonConf US. Looks like we’re not over the hump yet.

This is still the early phase, in the long term, the new paser will be a subproject of bucklescript, bucklecript will be properly rebranded, you can think of .res as a dedicated front-end to bucklescript.

Allowing only a few infix ops, meaning we know that both the parser and the base language support all infix ops in general but we’re not allowed to use them, is a difficult sell for me. Languages should not impose arbitrary restrictions.

Indeed, I need check with Maxim what’s the rational behind this.

It’s not clear what the long-term maintainability of this new syntax is. Based on what we know today–I’m not sure how sustainable it is as a separate project. If the upcoming Res syntax converter works both ways with high fidelity, that will be more reassuring. People need an escape hatch. That’s what we tell prospective users about BuckleScript from day one–if you want to eject, check in the JS output and delete bs-platform.

Maintainability actually is a good story of the new architecture, I tried to do some bug fixes for refmt but it is really hard to set up and have its dependencies installed, for the new architecture, it is zero effort, I will ask Maxim to make the repo public so people can try it.

Will BuckleScript continue to upgrade to new Reason Syntax versions as those are released (e.g. bugfixes, new features), or is it frozen at the current version of Reason Syntax?

We will provide a conversion script in the next release.

Bug fixes releases are always welcome. In the mid term, we will do our best to support it, it’s hard to predict things in the long term. One possibility is to make refmt configurable, it is already there today but not wildly advertised, you can customize the version of your refmt to pick up the latest version.

#68

Thank you Bob. That’s good to hear. As always, your work is highly appreciated :slight_smile:

#69

Reason’s main competition and blocker to adoption has always been Typescript. This release fixed up a lot of the small quirks of the language and made Reason truly competitive not in just functionality but also the way it looks to newcomers.

While reason certain felt nicer in many areas comparing to JS/Typescript syntax, I have always felt slightly embarrassed when showing people the syntax of bindings/decorators, list/array, string interop, polymorphic variant, excessive paren overload, weird semi-colon rules and etc. These changes will make me feel proud to show people what we got.

And the new syntax errors is just something I have never experienced in another language.

Amazing release.

#70

This would actually be very nice. Perhaps we can even require refmt to be a separate package, so we can choose the version we’d like to use.

@bobzhang I was wondering if BuckleScript will always be compatible with vanilla OCaml? I guess so because the AST is shared between Reason/BSS/OCaml, but is this also the long term goal?

#71

Hi @Jfrolich It is hard to predict things in the long term, note below is my point of view ( I don’t make decisions alone, will reach consensus among team members too) – we don’t make any decisions in the short term, so it will stay as is in the near future

OCaml is composed of two parts, the language itself and its libraries, toolchains.

The advantages of being compatible with full OCaml language

  • Maintainability
    We currently reused OCaml’s type checker (with some patches) and pattern mach compiler (some patches too) – this is the main concern, if we do deviate from OCaml language some day, we will put the maintainability into the highest priority, a non-invasive way to do this is pick an opinionated subset with some minor patches.
  • Learning
    This could be a significant advantage, however, the learning tutorial has to be adapted to alternative syntaxes

The disadvantages

  • Legacy
    OCaml is a language of long history, it has to maintain backward compatibility
  • Objects
    OCaml’s objects does not translate well to JS semantics, the maintenance overhead is non-trivial, essentially we are putting 10% of time in maintaining 0.1% used features.

The advantages of supporting OCaml stdlib and its ecosystems

  • Reuse existing OCaml libraries
    This sounds amazing, however, to achieve this, we have to do much more work, like being compatible with dune (assume dune is the only native build system which is not) and provide lots of c stubs polyfills.

The disadvantages

  • Philosophy
    We encourage people to do more shipping use the right tool instead of playing with tools itself.
    For example, it is fun to play with ppx, however, it is a broken abstraction in my opinion and should be considered only when you really need it
  • Native and JS has different memory model/performance model
    It is impossible to have a library performs best in both native/JS, they have different models.
#72

Sorry one more question: will BuckleScript continue to parse .ml/.mli files? I.e. can people continue to rely on using ML syntax in the future?

#73

see my post above, nothing changed in the short/mid term

#74

@bobzhang Thanks for your lengthy reply. I agree with most of it.

I don’t think it’s necessary to share the standard library of OCaml per se. Indeed it is probably better to have a universal library that is optimized for both target. The standard library of OCaml isn’t that great and many people use replacements for it anyway.

I do think it is nice to keep ml/mli files (and as reason is just a syntax, also reason). As that will allow for sharing some code between native/bs, that is actually a supercool thing that can enable many future innovations (especially when reusing components between web and REAL native). And it is also one of the reasons why people love Bucklescript. Often performance is not really that critical for a lot of code, and hot code paths can be optimized between platforms.

As for PPX’s I agree that it is not best situation at the moment. But I want to emphasize that metaprogramming of some kind is very important especially for a typed language. For instance graphql-ppx gives a way to use GraphQL with all types generated. Bucklescript probably would not be worth it to use in combination with GraphQL if you’d have to write all the types yourself (and almost all apps use something like GraphQL). So I hope that is not removed from the language. At the same time, I think it would be fantastic if we can have something resembling real macros in the language itself!

#75

I’d like to emphasize one more time that no matter which decision we make, your existing code will keep working. I feel that we’ve proven in the past that we took backward compat seriously. So subsequent discussions don’t need to worry about a hypothetical situation where your existing code breaks hard.
There can potentially be some small regressions but those would be bugs to fix. And if we ever need to intentionally break some small things, we’ll have migration scripts as we’ve always had, etc.

#76

I’ve been having a difficult time thinking about how to express my concerns with what’s going on, especially since I really like most of the syntax changes that were done and have a really deep respect for the team/community and all of the hard work that it takes to move things forward. But I’ll try my best to express what I think troubles me.

When I first started using ReasonML, an elevator pitch for it could be: “Use a strongly-typed, battle-tested, and pragmatic FP language with an approachable syntax while still having complete access to an incredible ecosystem and not needing to rewrite your entire codebase”.

And that is one hell of a sales pitch! And a non-trival part of it was that ReasonML would be an alternative syntax to OCaml but maybe lagging behind upstream but still a 1-to-1 AST for whatever version of OCaml it supported. Over the last few years I think that has mostly held true and it contributed to a lot of excitement for this language/community/ecosystem. But with some of the new syntax changes it seems that the maintainers want to fork off and start removing features that OCaml has support for.

To me, that changes what I thought I bought into and I think I feel a little-bit baited-and-switched here, especially since there was no real mention of this until it landed today.

And it’s also continuing a trend of how open people are to fragmenting the community. The OCaml community has definitely felt like ReasonML has been fragmenting them. The push to incorporate the -> as the preferred way to pipe things is still today, 2 years later, a hot-button topic that feels like it was released very similarly to how these syntax changes were released today. And now we’re looking at having 2 divergent syntax options which in the short/mid term can co-exist but how long can that realistically be maintained?

So I guess my concerns really are 2 things:

  1. We are okay with continuing to fragment the community
  2. The removal of a key part of the language has had significant effort put into it in stealth-mode and it going away completely in the long-term is even an option

Those things really shake my faith in what we’re doing here and for the first time has really made me second guess my decisions for buying into ReasonML so heavily.

Now, with all of that being said, I want to end this with a note of gratitude. I do really appreciate everything you all have done though, regardless of any of the decisions that may come to pass. You are all incredibly talented people and have built something really amazing. Using ReasonML has been one of the best programming experiences I’ve had in almost 30 years of coding and I’m deeply grateful for that. Thank you all so much.

#77

I share the same concerns.

Grammar should be decided at language level not at compiler / toolchain level.
There is already the case of pipe first but it’s quite explainable because of Belt and it’s an option, you can still use pipe last with function that have main parameter right.

I like some points (remove parenthesis, remove ;), some are explainabled at toolchain level (include Js module in pervasive), some other are totally weird regarding what is ReasonML (diamons for polymorphic parameters, dereference, list/array syntax)

This new syntax, at long or mid term, drives either ReasonML to converge with BS which will drop OCaml ecosytem or BS to be a different langage from ReasonML.

ReasonML is still young, community is amazing but small … it’s still hard to convince stakeholder to dive in, this kind of initiative will make the work harder.

#78

I do like the direction in which BuckleScript is going - a strong break from OCaml conventions and specializing towards Javascript is a good outcome for me. I would’ve liked to use the same code unchanged on both native and front-end, but for that we first need a web server and an ecosystem that can compete against Rails, Laravel and the like. That’s a long road. No point in hamstringing front-end adoption for a distant dream.

However, if I had reservations on the syntax change (which I have about array vs list), preserving backward compatibility wouldn’t be any comfort. I would want to be always on the latest syntax, and not say on the deprecated Record syntax for ReasonReact. I think that’s shared by folks I know who use Reason in production as well.

However a perfectly reliable AST mapping from the old syntax to new one would be super useful. Ideally there should only be one way of doing things and so even if people don’t buy into a particular way wholesale, it is still better to travel with community for better docs and better ecosystem support.

#79

I love the syntax. Much more lightweight than what Reason is now. I want this to become Reason.

Lack of communication is staggering, and the absent leadership because of “lack of time” effectively causes us all to waste a lot of energy. There’s been no plan and no real definition of napkin here. You don’t have to make democratic decisions but not doing so should allow you to communicate and plan better (because you don’t have to consult with the community). It doesn’t happen.

And it’s a recurring pattern. We see some random mentions of different kinds of weird stuff on Discord like occasionally Bob hinting at Bucklescript becoming another language in the future.

Edit: reassurances and listing tradeoffs is not a plan and neither a “definition” (of napkin in this case).

#80

I think this also ties into the other thread regarding Chat vs Forum. I often feel like there are things happening in the chat that’s only accessible to an inner circle (who’re rightly there because they contribute heavily, participate in issues, help newcomers, be a sounding board for core contributors and so on). But moving more things to forum would help folks who’re mostly consumers of the language like me, and want to see it gaining adoption.