End of the rails

I started working with Ruby on Rails full-time in 2007. I discovered it in 2005, when I was researching how to create an easy-to-use database layer for a desktop application I had been working on. And after finding ActiveRecord (and hence Rails), after a couple of weeks of playing with it, I was sold. Why copy this open-source project and translate it into another language (Delphi), when I could just use this. Unfortunately, my bosses didn't agree, so I made some plans, found a couple of clients and quit. I then spent the next fifteen years as a freelancer, building Ruby on Rails business applications for small businesses around the UK (as well as working on some really big projects that I got contracted on to).

And I still found time to do the startup thing, taking a load of seed funding for an online advertising company that was a lot of fun, allowed me to work with some really good friends, taught me a whole load about scaling applications, but ultimately failed. The sad thing was our USP was that we were privacy-first, with no tracking, and we used machine learning to tailor our adverts. If we'd just launched ten years later, I'm sure the company would be a huge success, but back then, no-one apart from us cared about that stuff.

Throughout a lot of the last ten years, especially with the rise of front-end javascript frameworks, I've seen the question asked - "is it still worth building a system on Ruby on Rails?". And my answer has always been the same. "Yes - there's no better way to build a database-backed web application".

But I think that answer is no longer valid.

The trouble with Rails

Currently, I still have a few freelance clients, as I don't want to leave them in the lurch. But I'm also working full-time as the CTO of Collabor8Online, a system designed for managing documentation in the construction industry. I rewrote the original Collabor8Online system (Rails 4.1, Ruby 2.3) from scratch, using modern techniques, such as Hotwire. And compared to building a front-end in Javascript with the back-end in Rails, it's been a breath of fresh air.

At the moment, according to bin/rails stats, the C8O app has 184 controllers and 118 models. We have around 30 customers and the audit trail table has about 40 million rows - not huge by modern standards - but it's approaching the point where just slapping together an ActiveRecord model, with a controller in front of it, takes a bit of thought and planning to avoid some serious performance issues.

As you can imagine, keeping track of almost 200 controllers and over 100 models also takes a bit of work. I admit that, as with all projects, when you're under pressure to get something shipped as quickly as possible, I made some design mistakes (quick example - I wish I'd forced shallow routes everywhere from the start). And those decisions make things even harder; but you'll be hard-pressed to find a project that doesn't suffer under similar pressures.

In addition, a lot of the operations on those 100+ models necessarily take a bit of time (there's a lot of shuffling files around between different storage areas, analysing the contents, then returning those results to the user). So, at first, I thought Hotwire's TurboStreams, with the background broadcast functionality could be perfect for this. I could kick off an ActiveJob (that might in turn fire off further ActiveJobs) to do the work. Then, when the jobs are done, they update the model, which broadcasts its changes back to the front end and the user knows what's going on.

And in simple cases, this works fine. But in more complex cases, it causes problems.

The world has changed

I have two major problems with Ruby on Rails at the moment. Well, three, if you include the attitude of our not-so-benevolent dictator for life (if you're interested in the trouble with DHH, there's been a few interesting discussions on the ruby.social mastodon instance).

But Rails was designed in the age when a web-application was a simple request-response thing. You might have an application with accounts that contained many people. So you could represent this as two models and two controllers. The AccountsController would list accounts (accounts#index) and show an individual account (accounts#show), the PeopleController would list people (people#index) and show an individual person (people#show). And you would probably nest the PeopleController under the accounts controller, so you can select an account, then see the people for that account. The operations you might want to perform would be to add a new account (accounts#create), edit an existing account (accounts#update) or delete an account (accounts#delete), with equivalents for people too.

Because of this generally simple structure, Rails recommended that you follow a RESTful style for your controllers. This mirrors the general model that the web was built on - you are dealing in documents and you operate on those documents by GETting, POSTing, PATCHing and DELETing them.

This is fine for many applications, but it's fast falling out of date because you no longer have a single view of a person.

You have their avatar, their name badge, the profile page that's visible to other users, the profile page that's visible to you, the badge showing whether you're online or offline, the badge showing how many unread notifications you have, the card that shows as a summary when looking at a group, the list item that shows when looking at an account, the card that appears in search results and many, many more.

Rails tries to deal with these through partials. But partials are slow to render, plus they use instance variables with no way of knowing (without reading the partial itself) which variables are required. Again, this is fine in a simple 2007-style Rails application where you display a person page and use those partials, in a single context, to build the page. However, when you've got different "aspects" of a person represented in multiple places across multiple different pages, rendered by multiple different controllers, it quickly becomes unwieldy.

Throw in Hotwire, and TurboStream broadcasts and it gets worse.

Firstly, you are tying user interface updates to your models, when the general Rails way is that views are tied to controllers. This is why instance variables from controllers are available in views.

Suddenly, when the model is responsible for the view, it needs to know those same instance variables. Plus, with multiple partials, all representing different aspects of the same model, the model itself needs to make sure it handles the broadcasts for all those different partials. Every time you add a new view to your application, the model might need to be aware of it.

Secondly, the model knows nothing about users.

In Collabor8, permissions are vital. You might be looking at a document, in a given folder, but you have permission to perform some operations on it (comment on it, download it), but not other operations (move it to a different folder, delete it). So that "list-item" view of the document needs multiple variations, taking account of the current user's permissions, so it can show the correct toolbar buttons. But the model is just broadcasting the update blindly, with no idea who will be receiving it. And because the partial is being rendered outside of the context of a controller, it doesn't know who the current user is, meaning the partial cannot conditionally render itself (not drawing the buttons you don't have permission to use).

Rails is trying to deal with this by adding the Current object, but that is, in effect, a giant collection of global variables that you have to hope a controller, earlier in this request, has set up correctly for you. Your Current object won't help you if you are inside a background job. And it still doesn't deal with the fact that a single broadcast could go out to hundreds or thousands of different users, each with different permissions.

The future of web development

I've been using ViewComponents to try and deal with the issues around partials.

They render faster than partials, they are an object with a constructor, so you know every time, which variables you need to supply. And, by definition, a component is showing just one aspect of a particular model.

But it doesn't solve the fundamental problems.

• If you use broadcasts, you can (with a bit of work), render a ViewComponent instead of a partial. But your models are still responsible for generating your views, even though views really should be generated in the context of a controller.

• If a component (some aspect of a model) is lazy-loaded (using a TurboFrame) then it needs a controller and route in order to load itself. This is fine if you have a couple of different components representing a person, but when you have twenty, it becomes unmanageable.

• The set of operations you want to perform on a complex application are far larger than just "create" and "update". In fact, the promise of Object Oriented programming is that the operations for a "thing" are defined alongside that thing - they are the messages your object accepts (in other words, the methods that it defines).

  • For example, in Collabor8, the AccountMember model has a field account_owner: boolean. But just setting this to true or false is only part of the story. As soon as you set it to true, the system then needs to perform a whole load of other operations, cascading through the hierarchy, to ensure this account owner has the correct permissions.

  • The simplest solution to this, in Rails, is to attach a callback to your model. But once you have lots of callbacks (as I have in this large application), it becomes almost impossible to trace through what is actually happening, as things get triggered across lots of different models. Plus a model a long way down the hierarchy doesn't know who the current user is, or why a particular permission is being changed. This means that, when a customer asks why such-and-such has happened, it's really difficult to figure it out. And, in my experience, a lot of "bug reports" that take up a huge amount of support time, aren't really bugs, just the complexity of the system resulting in unexpected outcomes. Traceability really helps here.

  • The alternative recommendation, in Rails, is to add in new routes. In your AccountMembers controller, you might ensure that the account_owner field cannot be changed. Instead you might have extra routes for an AccountOwnersController with actions for AccountOwners#create and AccountOwners#destroy. This extra controller will not only set the account_owner field but also do the cascade of permissions and other operations that come with it.

  • But, once you have ten or twenty specialist operations per model, you get into building a lot of routes. And finding names for all those routes gets really tricky, because RESTful routes are nouns but operations are verbs. The one that really bugs me in one of another client's apps is TimesheetDiscrepancyResolvers#create which is just a crappy way of saying Timesheet#resolve_discrepancy.

The way we build web applications, and the expectations of the people who use those applications, have changed greatly in the last fifteen years. We expect to see things updating in real-time, no matter where it is on the page, no matter if it is on a page directly relating to that model or not.

But Rails is still built around the fundamental assumption that working on the web is similar to working with documents, with a tiny number of operations to be performed and an equally tiny number of representations of those documents to be displayed.

Front-end javascript deals with this by using components.

Each component represents an aspect of a model, it can pull the data for the model in its entirety and just show the parts that are relevant. It has its own set of operations that work locally or on the server (depending on your choice of framework). The component deals with updates by using what Javascript people call "reactivity" (which is where React gets its name). Reactivity, to Object-Oriented people, is known as "the observer pattern" and what, all the way back in the 1970s, Smalltalk called "the dependency protocol" . A view is a dependent of a model, so when a model updates, the view updates too; but the model knows nothing about its dependents.

None of this is new.

TurboStream broadcasts are an attempt to add manage this reactivity, but they are normally built from partials and require the model to know every single view that needs to be updated. This ties your model to your user-interface in a way that breaks all the normal Rails conventions and makes your application much harder to maintain.

Compare that to the equivalent Sveltekit application which automatically knows every component that needs to be redrawn when you update the name on a Person model. No matter which page they are on. And none of those components need a new route adding in to deal with the redraw, even though they may be rendered both on the server and on the client.

The way forward

What is needed is a way for Ruby applications to publish multiple different aspects of a given model, in the correct context, without generating a ton of busywork around routes and controllers and without requiring the model explicitly track who is viewing it.

That's my plan for Cromulent (coming soon).

You define your model, specify the different ways you can represent that model on-screen as well as defining the various operations you can perform on it. Then, when an operation updates the model, the views automatically update, no matter where they live. All without requiring you to make up nouns for your verbs and add in extra routes and controllers to deal with every facet of your application.

Given the situation with Rails and DHH, I'm leaning towards making it something that doesn't depend on Rails itself, but, of course, can be easily embedded within a Rails application. However, it's an ambitious goal and I'm still just starting out with it.

The Promise

The promise of Rails, back in 2005, was you could get a web application reading and writing from your database in fifteen minutes, without having to write a ton of XML mapping code. Today, that mass of XML has been replaced by writing a ton of ruby, defining your controllers, routes and manually managing your user interface's reactivity.

I want to take things back to how they used to be, and let Ruby once again embiggen your web-app productivity.