Web application implementation step 6: make it fast!
RT @thibauld Web application implementation step 6: make it fast!To continue in the "how to implement great web apps" series, here the step 6: make it fast! This is a vast subject and I'll not try to be exhaustive, I'd rather make a list of checkpoints with clues on how to improve each point. Making its web app fast is key for user adoption, as there is nothing more frustating than a slow web application.
Just to be clear, when I say "fast", I mean "fast from the user perspective" because what's really important in the end is how responsive your app will feel from the user perspective. To illustrate this point, let's take the example of a web page that requires 5 secondes to load, it could be acceptable if the web page becomes usable after 1 second (from the user perspective). What would be unacceptable is a page that would require 5 seconds to be finally usable by the end user. Now what should you do to make your app fast?
Slowness can come either from :
- the backend: the server takes too much time to process the request
- the network: transporting the response from the server to the client (the user web browser) takes too much time
- the rendering: actually displaying the response received from the server takes too much time
Today we'll tackle the backend part:
Profile your application. It is necessary to measure how fast is your app and where lie the problems if it is too slow. If you're developing on a LAMP stack, there is an awesome tool for that called Xdebug. Once you have installed xdebug, all you need to do is to enable it in /etc/php5/apache2/conf.d/xdebug.ini, here's my configuration:
zend_extension=/usr/lib/php5/20060613+lfs/xdebug.so
xdebug.profiler_output_dir = "/tmp/"
xdebug.profiler_enable = Off
xdebug.profiler_enable_trigger = 1
The output dir must be writable by apache and I encourage you to enable it only via trigger. Now if you want to profile a screen of your web application, just append the argument XDEBUG_PROFILE=1 to the list of arguements. Example:
http://dev.domain.com/a/screen/path?XDEBUG_PROFILE=1
Now, when you access the above url, a file gets created in the output dir (here /tmp/). Just open this file with kcachegrind and you'll be able to visualize exactly what has happened behind the scene to process your request.
It will be easy then to answer the following questions:
- what is being called? how many times?
- how much time does my request take?
- is my caching system working as expected?
- why is this request so slow?
- are my sql requests fast enough?
- is the framework I use adding too much overhead?
- etc....
Kcachegrind provides an incredible value to profile your web application. In the next article, we'll stay in this 'make it fast' step and talk about how to make your web app network efficient.
Stay tuned!
Web application implementation step 1: choose your weapons!
RT @thibauld Web application implementation step 1: choose your weapons!During the next series of posts, I will describe the different steps involved in building a web application. These posts will only deal with the technical aspects of implementing a web application that has already been designed. Web application design is another subject which I'll develop in another series of posts.
I'll take the new FBC website as an example. It is a good example as we almost had to start from scratch to build the new version. Indeed, the architecture in place was extremely inflexible, the worst being :
- Webpage layout relied on field values in SQL tables
- 1 url <=> 1 php page containing everything from HTML to SQL
Functionally speaking, their website was working well but it had been at the expense of countless hours spent testing and testing and retesting that every process was functional. But now they were stuck... Only changing the graphical chart would have needed to go change *every* php file and restest the whole thing entirely... When I saw that, it was obvious that it would take us less time to rebuild the entire architecture from scratch and then add the new features than trying to work on this extremely rigid and non evolutive code organization. Of course, we would need to reutilize pieces from the old code in order not to re-implement the business process, but it was pretty much everything we would be able to reutilize.
So the first step when building a web application (and not a website) is choosing your weapons, that is the technology stack you're going to build your application upon :
- Which OS are you going to use ?
- Which webserver ?
- Which database ?
- Which programming language and framework ?
Choosing the OS
I'll not elaborate on this one, use Linux. The only reason I can see for not using Linux are :
- you know nothing about Linux and don't want to learn
- you were asked explicitely not to use Linux
- there are obvious reasons which require your app run on another system
If none of the above matches, choose Linux. And among all Linuxes, if you have no preference, I'd personally go for a Ubuntu Server LTS (Long Term Support) which is free and provides security updates during 5 years. Just check that the LTS does not end next year 
The current Ubuntu Server LTS is version 8.04 (code name Hardy Heron).
Choosing the webserver
Again, I'll not elaborate much: Apache 2 as it has proven capabilities to run high traffic websites, great documentation (official and non official), lots of modules availables... etc... There are other webservers but, unless you want to restrain it to very specific tasks, use Apache to serve your dynamic pages.
Choosing the database
I assume you want an Open Source database here. It's hard to choose here between MySQL and PostgreSQL . PostgreSQL is probably more powerful but it is also probably a little harder to get into. For example, the fulltext search in MySQL in a lot easier to use than in PostgreSQL but PostgreSQL's fulltext search allows you to perform more complex queries. In the same spirit, MySQL provides handy but limited functions (ex: GROUP_CONCAT) where PostgreSQL offers you less handy but more powerful equivalents (ex: array).
Both database has a good official documentation but I think MySQL benefits from a lot more unofficial documentation compared to PostgreSQL. In terms of proven capabilities, both have examples of usage in very demanding contexts...
Finally, the choice is up to you BUT if you use MySQL, do not forget to use InnoDB tables instead of MyISAM tables. By default, tables are created using MyISAM. Using MyISAM table can be a good choice but it must be a known choice. Indeed, MyISAM table does not support FOREIGN KEYS nor does it support sql transactions so it is very limited and will inevitably result in added programming overhead if you build a complex web application upon MyISAM tables.
Choosing a framework
Rule 1: be skeptical
First, you should not use a framework for the sake of using a framework. Instead, you should really think at what you expect from your framework. I'm personally a bit skeptical about a lot of frameworks because, in my opinion, a lot of them end up being bloated, trying to reinvent the wheel and with finally a limited number of really useful functionalities. Keep in mind that using a framework is a binding investment: First it needs time to learn how to use it efficiently and, second, it generally deeply affects your code. So you'd better choose your framework well if you don't want to loose a lot of time !
Rule 2: know why you need a framework
Beware of frameworks that try to provide a solution for every aspect of your application as they are likely to belong to the "bloated" category! I personnally prefer a framework which focuses on doing 1 thing right than a framework that tries to simplify everything. For each functionality, ask yourself : "Do I really need my framework to do this ? Will it really help me be faster in my work ?
Rule 3: Beware of marketing crap
Here are examples of marketing functionalities I always consider with skepticism :
- i18n support (internationalization): In which way is it better than gettext ?
- RESTful URLs: You mean Apache URL Rewriting (and optionally a routing index.php) ?
- Easy Ajax: I did not know Ajax was hard to implement on the server side! Are you looking for a Javascript framework ?
- Full database abstraction: This one must be considered really really carefully. Database design is one of the most crucial part when you build a web application. In the same manner, how you query your database has an enormous impact on your application performance. This is why I am really really skeptical when it comes to delegating the database management to the framework. Some frameworks even create the database for you! Unless you're talking of a very simple web app, I think it just does not make sense... The only explanation I can think of is that a lot of web developers have a very little knowledge of SQL and, for them, it's easier/better to have the framework handle the database. On the other hand, it is also true that that database abstraction (especially ORM) are useful to handle all the simple queries. As a conclusion, concerning database abstraction, I would only use one if I'm sure that it's flexible enough to :
- Work on top of the database schema I designed
- Let me perform raw SQL queries when needed
I see a framework as a toolbox to help you concentrate your efforts on implementing real business code. So what can you expect from a framework ? A templating system, a caching system, a testing / debugging framework... All of these are very useful functionalities that a framework can offer because coding them would sure be a waste of time!
Choosing a programming language
There are a lot of very good languages for web programming (PHP, Python, Ruby...) and I can see a few differences:
- Framework compatibility: Once you've chosen your framework, you probably already know the programming language you'll be using.
- Quality and quantity of documentation: I think PHP has the best and most comprehensive documentation, followed by Python and Ruby.
- Library exhaustivity: Again, in my opinion, PHP has the deepest library available, followed by Python and Ruby. You should check beforehand that a language has all the required library for the app you want to implement. For example, if you plan to use SOAP based webservices, you probably want to avoid Python as (last time I checked) it has one of the most limited SOAP library. PHP, on the contrary, has a good gsoap compatible SOAP implementation.
- Familiarity with the language: One of the most important criteria of course! How productive are you with each language ? Even if all of these languages are relatively easy to get into, mastering a language takes time... so if you're a Ruby guru and have never used PHP, then you're probably better off with Ruby...
- Language beauty: Sorry guys but this is unfortunately NOT a good criteria. Had it been, then I would have rated Ruby first, then Python and then.... far behind, PHP
To conclude this long post, I think the most important rule is :
Use production ready technologies which have proven to be performant in similar contexts
Each time you have to choose a technology stack, no matter what it is and what bloggers and/or developers tell you about it, find relevant examples of web applications using this very stack to be sure, provided you do things well, the chosen stack will be able to fit your needs (performance, scalability...).
Stay tuned for the next post where I'll talk about the technology stack I've chosen for FBC!