RFC: The future of Bs-Express

#1

Update to update: Still getting useful input, so keeping topic open for a few more days

Update: I’m going to keep this topic open for 3 more days to make an even week. After that I’ll make a summary of the opinions contained within this thread and subsequently attempt to work the feedback into the proof of concept I’ve started at https://github.com/ncthbrt/reconstruct. This work is likely to be a long march to reach a point where it could be considered acceptable for broader usage, but bs-express will not be going anywhere and will be maintained in the interim.

I have held the view that express while widely popular is a many senses a poor design. Largely for two reasons:

  1. Its mutable api makes it easy in larger applications to accidentally open up security holes or introduce unpredictable request flows (have experienced this in practise).
  2. Express also makes it possible to leave requests hanging.

Bs-express corrects deficiency number 2, but doesn’t help to correct 1. Right now this project is being actively maintained but is more or less in a holding pattern (for example it isn’t really getting new features nor a nice set of documentation). This is largely because I’m not very motivated to improve a design which I consider to be based upon an unsound foundation.

I would like to propose that this project (or a successor to this project) be broken into two parts:

The first is a layer which allows users to describe their web application in a more declarative/functional manner. This layer would probably take its cue from frameworks like Freya, Suave, and Giraffe which are all written in F#. This layer would be platform independent.

The second layer would be where all the platform specific/moving parts live. This layer would take the abstract description of the web server, “compile” and then “execute” it. This could use Node.js or if using native, be run using something like LWT.

This approach means that native and node ecosystems can cross pollinate and provide a stronger, more idiomatic approach to writing web servers.

Keen to hear thoughts…

#2

For what it’s worth it looks like you’ve really thought through this problem and I really like a more idiomatic approach to web servers in Reason, especially when it has the possibility to compile to native in the future. I know in the past OCaml has been a poor server-side language due to a lack of proper threading? The bigger issue might have been that it just wasn’t popular enough for people to just make something good.

Although this is perhaps outside the scope of your project I’ve loved the Phoenix framework on Elixir, which is a sort of functional rails. It’s really well put together.

Overall: Keep up the good work :clap:

#3

Thanks for starting the discussion. I think Reason could indeed do very well on the backend, even though it’s a bit lacking right now. Express most definitely has bunch of flaws but it’s something that a lot of people know and use to build smaller scale projects. I think there’s tremendous value in having an express-like API just to attract NodeJS developers.
There are 2 things people want to do when they first try Reason:

  • make a react app
  • make a mostly static backend (hey, one promise of Reason is to be usable anywhere right)

Right now they can do #1 but when they ask about #2 I don’t want to point to something super different from express. Actually I’d literally tell them “you can use express! Look at bs-express” and that reassures them and makes them wanna build a full-stack reason website.

So tl;dr I think there’s a lot of value in blindly binding / copying NodeJS APIs for the sake of familiarity for newcomers.

EDIT: That said, the same way ReasonReact is working on fixing the problems of React, you could make the Express API evolve to be better / faster / more secure and that’d be a HUGE selling point.

#4

@bsansouci: That is an interesting point. I agree with your general sentiment. However I feel we’re stuck in a bit of a local maxima with BsExpress as we’re constrained by express’s underlying design.

Having studied express’s docs to work on the current bindings, I believe being able to provide a familiar express-like API wouldn’t be inordinately hard to do on top of a lower level foundation.

Perhaps I’m wrong and better bindings can shore up the majority of rough edges. But it would also be nice to be able to compile to native Ocaml as well using bsb-native.

@justgage: Thank you! Taking a look at Phoenix. I’ve vaguely heard of it before. Reading the docs now.

#5

I second the express-like API style.

Alternatively, I think I might even like a HAPI-style of API more? I’ve written JS for both, and the HAPI performance was worse than express, but I felt like it was more expressive. It would be even better if it had something nice like Swagger that came with it.

Maybe Reason can combine better-than-express performance with the niceness of HAPI?

#6

Lot of ocaml web stuff has been done, for example, http://ocsigen.org/. Matter of porting maybe…

#7

That is a good point @idkjs. I haven’t searched though projects in Ocaml that would be suitable for porting.

Ocsigen’s work on web servers is interesting but it is quite a departure from what I’d say people are typically used to in a web server. A project like opium feel a little bit more approachable: https://github.com/rgrinberg/opium. Though that just may be the way ocsigen presents and documents their projects.

#8

I’d go the route of making it feel like express while fixing up some of it’s shortcomings. If it’s any guide, ReasonReact is a bit of a departure from what people are used to with JS React (reducerComponent, etc.). I think devs just need to know that there is energy/activity in a module, and that it’s familiar enough.

As an express user myself starting to look hard at Reason, you’d connect with me if it was just as easy to set up an app and register routes in a similar way. I’d more than tolerate differences if the design improvements were clearly explained. In fact, it would just make a stronger case for why I’d want to build an app wth Reason.

#9

I’m super impressed by rust’s rocket web framework – I’d love to come up with something as usable & fast & safe out of the box!

#10

@jaredly Rocket looks very cool.
Q: What is the thing about rocket that impressed you the most? The typesafe routing engine?

#11

For what it worth, we are using ocaml as server side language at very big scale at Ahrefs, including http server, and it works pretty well. So I don’t think you will be blocked unless you are Facebook or Google.

#12

@Khady Out of interest. What are you using as your webserver at Ahrefs currently?

#13

We are using https://github.com/ahrefs/devkit/blob/master/httpev.ml
It is probably too low level for what you would like to do. But I think it’s a good example that “simple” stuff can reasonably scale up.

#14

Yeah – I love that it feels as easy to use & get started with as express (maybe easier?) but it’s much safer

#15

One thing that might would help in applying ocaml tested stuff would be to maybe roadmap how one might use @Khady’s suggestion in a reasonml project. What would it take to do that? Doesnt seem to hard. Maybe build it with esy? I need to see that work flow one time. Would be useful in all kinds of other projects as well and would accelerate the proliferation of ocaml work into reasom land.

#16

@idkjs: Would you mind explaining a little more about what using @Khady’s suggestion means? I’m not quite clear as to what you’re suggesting.

#17

@ncthbrt i guess I mean, what would a quick port look like. ahrefs doesnt seem to be using it in public reason project yet though they are writing bindings so they might be.

#18

@idkjs Looking at the code, feels like it would be something that would be quite difficult to port. Which is why I would rather attempt to create an abstraction layer which could be easily ported to different backends.

#19

i love it when you talk that way. I dont know what an abstraction layer is but I can guess what you mean. So what would an ‘abstraction layer’ look like then on that project?

#20

I agree with having a familiar interface for newcomers to ReasonML.

I also think that there is probably a lot of room for improvement, so the ideal would be to do both IMO.