Tuesday, August 21, 2012

I Further Predict the Death of Your Web Framework

In my last post, I philosophized how new technology is going to change the bottleneck of web (and other) systems (which despite everything else, have remained surprisingly stable for awhile).

This spelled the demise of many systems that relied on a given system bottleneck, specifically, slow runtime systems.

But there is another technological shift conspiring against many web frameworks that isn't focused on performance, but instead focused on "ease of use" - which in many cases may hit far closer to home.

That shift is the reorganization of MVC.

MVC stands for "model-view-controller" which loosely means you have a datastore/database (the model) which is retrieved and manipulated (by the controller) such that it can finally be shown to a user (the view).

That is, pretty much always the flow. Well - kinda. The first thing you might notice is that "MVC" is an out-of-order acronym per the dataflow. In that case it would be "MCV". And happily, given that dataflow is paramount to my story - I'll use that in the rest of this article (that might irk you if you're CDO (which is like "OCD", except in alphabetical order, like it SHOULD BE)).

A predecessor to MCV was a simpler idea of simply "client/server" where client was the view, server was the controller and model (or, some or all of the controller could be in the client too). However, the client in that case actually implied it was a real client - that is a program that received the data and showed it.

In the web, the browser is the client, but interestingly in things like Rails, Jails, Nails, Grails, Struts, Play!, PHP, ASP.Net, and many others the "view" is on the server which then renders HTML and sends that to the browser. As far as the programmer is concerned, the whole MCV is on the server. The browser is often just a dumb terminal.

In the last year or two however, the popularity of a new type of framework is changing all that.

That change is coming from libraries such as backbone.js and ember.js (and many, many others).

Those libraries allow you to render views (not just show, actually render) in the browser itself. In addition, they let you leverage a lot of javascript magic in the browser. This is pretty awesome for several reasons.

The computing power of rendering is moved to the client's machine. Rendering isn't probably your biggest computing expense, but take off that computing cost from your server (times every web request you get) and its measurable.

And as you can imagine, if the "V" of MCV actually migrates to the client, all that's left on the server is "MC" (to be fair, sometimes even part of the "C" goes to the client).

What thousands and thousands of Rails developers discovered upon moving to backbone is that they no longer needed their fancy template views. Their backend became a system that pushed JSON over HTTP.

Very clean and very simple. At my new company Refresh (we're hiring!), our backend pushes the exact same JSON to our webpage as it does to our IOS app. And that same system will someday seamlessly become our API too.

For me, using Rails for webapps over Java (where I spent plenty of time years ago) was a simple decision. ActiveRecord was beautiful and elegant (especially compared to things like Java's hibernate). Also, the view layer was simple, well-laid-out, and standardized. If anything, Java had too many choices.

But these days, I tend to use NoSQL on the backend. And ember on the front-end. All I need in the middle is something to manipulate and push JSON. Why was I paying the Rails tax? (again insert any language that is a multiple slower than Java in that sentence).

I'm not particularly picking on Rails - it is just a full MCV solution that I no longer need. There are plenty of those.

And if you're thinking this is a win for Node.js - you're probably right. With much more javascript coding entering your web framework as a whole, using Node on the backend is probably the winner of all this on the usability front. Javascript on the server isn't the fastest, but it's pretty darn good at manipulating JSON (and thank you to whoever it was that shot XML dead).

So my not-so-amazing prediction is that in a few short years time full web frameworks from any languages disappear. Node picks up some of that slack but so do less feature-ful frameworks (and maybe performant ones). Even non-frameworks altogether get more use.

There's surely no mourning required here. Web frameworks change every few years no matter how you slice it. But between this post and my last, I see two converging fronts out to kill some our most popular ones right now.

Personally, I'm hoping to never server-side render HTML again. I'll let your browser do my rendering while I sit back, chill, and push some JSON.

And yes, Mailinator is in rewrite now to use ember, much to the chagrin of web scraping programs everywhere! (but much to the happy of JSON receivers)

22 comments:

KautionTape said...

I do completely agree that there will be a movement for all the View handling will be done client-side using JavaScript. However, does that necessarily mean the death of full web frameworks?

Web frameworks are capable of morphing to meet the needs of the users. If those needs are "NoSQL + ember.js" then web frameworks may move to that direction. There is still a use of Models to map stored information to "objects" and the controller to handle all that authentication and CRUD work. And I don't always want to write all of that from scratch. Especially when you look at what frameworks like Django are trying to solve with the "Give us the basics, we'll do everything else." Perhaps this will mean a shift away from the "end-all-be-all" solutions like RAILS to minimal solutions like CodeIgniter which may just include ember.js or backbone.js from the start. Just a thought. I completely agree to be rid of server-side HTML rendering, but I don't know if that would really mean the end of web frameworks.

That said, I love your blog. I eagerly anticipate your posts, and find them very awesome reads. Keep them coming!

Dobes said...

This may be a long term trend but at this time it is still faster to render server side HTML. One can buy a faster server but you cannot buy your users a faster computer.
This is because the HTML for a piece of view is often smaller than the json data plus javascript code to render it the first time AND it renders faster. And the first render is the one that makes the biggest impression.

Andy C said...

Do you think that the same should apply to mobile web sites? These are usually much more cut down than their desktop counterparts and the end device has much more limited processing capability.

I would have thought that in these situations that server-side rendering is still best.

What do you think?

Anonymous said...

I am not too convinced at the moment. Reason for that is the lack of a fallback solution for non-JS users. Your attitude is right in environments where you strictly need JS - but what about a page like Amazon? If they exclude potential customers by excluding non-JS browsers, then something is wrong in their business model...

Anastasiosyal said...

An interesting and informative read indeed. I feel that web frameworks are here to stay for much longer than you predict. The reason being that SEO with static text output from the server is superior to ajax based SEO.

How many sites do you see at the top of the search engine rankings whose content is rendered by client side javascript? I know there is support for it by google, but in reality, I do not see sites up there in the first search results.

Unless such a movement gains enough critical mass, search engines will not take serious note of it and plain old html served by the server will be the way that business is conducted, simply because it ranks higher in search result listings.

Of course, business applications or applications where SEO isn't a major requirement can take this route without taking a hit.

Just my 2c. Thanks for an informative article!

Anonymous said...

Interesting, but how about search engines? What they see is the HTML sent from the server...

luis_cornejo said...

Interesting,

but what are your thoughts on what the 37signal guys are doing? They are moving away from the heavy client side js frameworks back yo rendering in the server. Check out their blog.

Anonymous said...

Well it is not that special. I did most of the rendering in the browser from the moment IE introduced the xmlhttprequest object.
At that time it was just waiting until the other browsers caught up.
It took a long time.
JSON is another of those 'technologies' that is bound to fase out as well. Structured text is much more compact and de/serializes incredible fast.
The last 12 years i did not have to change much client and server side. I just ignored all the 'technologies' that made it slower and using ever more resources. The IDE's are where most of the gains were made. Color coded and intellisense helps a lot.
The biggest problem of the web will always be the DOM and its interpretations and implementations. Instead of just making one standard in 1999, IE5 was at that moment the standard as it was most used. But no instead specifications were written that contradicted the way ms used it and created the difficult 12 years folowing that.

omegaluz said...

Antiquating frameworks like MVC is not going to happen. Even if you offload the HTML rendering to the browser, your server side view effectively becomes the JSON that you are sending to the client. The pattern still remains.

If you are developing simple apps like Commercial web sites that don't have a ton of logic or data being computed or passed around, then I can see a Backbone like framework being ideal. However, once you move in to the realm of Enterprise apps, the thought of the client (the browser) doing ALL of the UI work, I get frightened that developers like yourself are purporting such nonsense.

Anonymous said...

So, how would I get started in learning how to do this type of web development? I've been "toying" with Rails for years now but haven't done anything really interesting with it. Is there a "phrase" or something that I can look up for this? In other words, if I researched the "Rails Way" by searching out MVC (or MCV) as you put it, is there something I can search out to learn techniques for this new method? Is this method just inherent in the way people use ember.js and other tools? I'm interested, but not a "seasoned web dev" who can figure all this out on my own... :(

Jeffrey Yasskin said...

Man, get with the times. "Web framework" now refers to the thing like Ember that runs in the browser. ;)

The one thing I wonder about with the new paradigm is how to optimize the initial latency of loading the page. Once the application is running, of course, a client framework with a json backend is exactly the right thing to use to optimize latency, but it's guaranteed to penalize the initial load by at least one round trip, in addition to the time needed to download any javascript. (Versioned URLs with infinite caching might be enough to eliminate the JS download time for all but the first request, but I haven't tested that.) Maybe that extra round trip isn't significant for 90% of web sites?

Jeffrey Yasskin said...
This comment has been removed by the author.
Perry Butler said...

"our backend pushes the exact same JSON to our webpage as it does to our IOS app. And that same system will someday seamlessly become our API too."

This is probably the greatest consideration for using JSON/XML over AJAH - the ability to push that JSON/XML data straight to iOS native apps for parsing where HTML would be too messy to deconstruct. I myself refuse to cater to iOS native so I only develop for the web and make sure that it works cross platform.

I have been programming custom frameworks using an AJAH approach and it works extremely well. It's a hybrid - we render the full page at the server (synchronously) on initial page load (it gets cached!) and then do partial updates asynchronously (these also get cached!). This eliminates the need for the client to do several calls to the server on initial page load in order to fetch all of the content. Basically if we can do an update from the client we can do that same update from the server synchronously. This optimization dramatically reduces the number of HTTP requests made to the server. I look forward to trying Ember or Backbone one day but so far I have been able to do everything rather elegantly with jQuery alone. Thanks for writing this article.

Anonymous said...

It's a great moment when the penny drops that you can eliminate all presentation logic from the server-side forever. No templates, no HTML, just send JSON to a web GUI developed in 100% JS. Better segregation, fewer bugs, more flexibility, much less network traffic, agnosticism between client and server, etc.

Been doing it for years now. Never looked back. In fact, I like it so much that I develop desktop apps the same way via WebKit embedded in a Qt wrapper.

Anonymous said...

Problem with moving rendering part on the client is that client itself can be very dumb device with just a HTML renderer or switched JS off.
But Web itself is a stupid self-limiting technology: invented just to SHOW TEXT, not it used everywhere for anything and... suxx. Rich windows apps has way more beauty interface and response, including many hidden tasks which cannot be done from browser. So yes, framewroks gonna die and one normal .NET should solve all that "multiplatform" issue.

rmurray said...

Twitter for example, moved from client side (browser) rendering back to server assembled templates because their app was too large and the long load times meant user experience was hampered. But then thats the scale of twitter...

Anonymous said...

Paul,

I sort of see what your saying, but I'm not sure how this will result in the death of web frameworks.

A framework (e.g. rails) has routes, controllers, models, validations, helpers, etc. etc. When using something like backbone.js, that takes out of view layer but you still need a framework to do all the other backend work (albiet you are now returning json).

Am I missing something?

jmarranz said...

(sorry previous comment was anonymous)

One thing I appreciate of you is you don't like to follow the "mainstream approach" whether you're not sure it's the right approach.

Regarding to server side web framework going to die:

http://engineering.twitter.com/2012/05/improving-performance-on-twittercom.html

"To improve the twitter.com experience for everyone, we've been working to take back control of our front-end performance by moving the rendering to the server. This has allowed us to drop our initial page load times to 1/5th of what they were previously and reduce differences in performance across browsers."

Reasoning is simple:

Getting JSON data from server is basically the same in performance than getting the same data decorated with the final HTML and modifying the page just with a simple innerHTML.

Apparently there is much more work in server: not at all, rendering JSON or HTML is not going the same very different and the most important thing:


"UI rendering and logic to JavaScript running on our users’ browsers and consumed the Twitter REST API directly... it lacked support for various optimizations available only on the server."

That is in a pure centric you're going to ask several times to server because your code in client is not going to be smart enough and
the original source of truth and decisions are... in server.

Good luck with your Refresh.io adventure.

Anonymous said...

This is a completely personal and subjective opinion, but I fell it is more manly to render HTML server side... It is just a feeling really, not even a proper argument, but the thing is most browsers still offer the possibility to turn off javascript, so someone can still walk to your website and have...hmmm... no view at all! Why choose something as the main technology for your site when it can be turned off... This "trend" actually started with google and adsense?!?! I know, terrible arguments, but think about it! Plus, I believe the people who come up with this kind of trends, for using javascript, json, use javascript for the backend and stuff like that. They never think about security , people security, not user security AND they think that they are doing something SO new, the whole browser is client side mate! A huge amount of technologies run on the client side, this is not new! The terminal is not dumb AT ALL! Another thing is that sometimes, some people give the argument that "Oh, we are going to do the heavy lifting on the client side, so front-end developers are helping the company save lots of money on hardware and new infrastructure", c'mon! Computers were never as powerful as now, as cheap as now, and they always get more powerful and cheaper and with more memory. We are even inventing new ways to do programming THAT ADD nothing really new to programming beyond complex syntax and the need to have more powerful computers to run... What is really important these days, is to turn the internet more secure! Families are training their children to NOT trust the internet, children are receiving classes in school about not trusting the internet, religious parties are teaching their crowd on TV shows why not to trust the internet, soon enough, there will be no market for us to do heavy lifting for, no matter wich side it is gonna be.

Anonymous said...

You talk about MVC as though you invented the pattern, but you did not. That honor is reserved only to the authors of the Smalltalk system, where the controller did not have a say in how the view rendered or otherwise interacted with its corresponding model.

I don't know what it is with all these people trying to bastardize MVC into Java, Javascript, servlets, or whathaveyou, but it's emphatically not MVC. Useful? Absolutely! MVC? No way. At best, it's some combination of Document/View (remember that one?).

Alas, it's much too late to change history now.

Alexey said...

Google has moved to MPV a while back within their GWT framework so you can say all GWT developers are using MVP already and not MVC:

https://developers.google.com/web-toolkit/articles/mvp-architecture

Kamil Rakowski said...

Hey Paul! I made something for you and I also left an email at support@manybrain.com ;) My email is kamil@webclip.tk by the way.
Anyway, I made an alias generator for Mailinator! But not a random letters and numbers generator, but one that makes actual words. You can download it and see the screenshots, as well as the VirusTotal results, at webclip.tk/mailinator.