Web application implementation step 4: Copying is not reusing
RT @thibauld Web application implementation step 4: Copying is not reusingIt is often said that "good coders code, great reuse" (see an example here). I definitely agree with this... unfortunately, too often, developers think that reusing code means copying code. How often did you see developers looking for code snippets on google to find a way to implement any given functionality, convinced that it will save them time... Unless the code snippet you're looking for is yours (= one you developed and, by extension, master), this behaviour will not help you much... Worse, it will even slow you down and here's why:
- you spend time on google, blogs, groups, forums... trying to find the code snippet that fits your needs
- you find something that is supposedly doing what you want but how to know exactly if it's ok ? It would require to carefuly examine the code... but it takes time... and you wanted to find a code snippet to save time so you decide it must be ok.
- you include the snippet in your own code but you have to spend some time adapting it because it can't be included "as is".
- finally you think it's ok and test your code. Not bad but it's not exactly what you expected so you're back hacking the snippet. As it is not your code, you're reluctant to study it in depth (and you're a bit lazy) so you treat it as a black box: you change a value here, a value there... "it should do it". And you spend time fine tuning it.
- you finally have the functionality you wanted. Too bad, the guy who wrote the snippet was not an expert and you're now facing crashing, security and/or performance issues with this code. You now have to spend time trying to fix a code that is not yours.
Now did you really save time ? To make things worse, you did not learn anything in the process... This method looks sexy at first sight but beware as it's a trap! When facing a new problem, you'll be better off understanding it and tackling it on your own and now! You have several solutions:
- Find a library, a webservice... which adresses the issue you're facing. What's the difference between this solution and using code snippets ? Simple: library, webservices etc... is code that is meant to be reused! They are tested, documented and maintained pieces of code... not a piece of code coming out of nowhere pasted on a blog around 2 photos from flickr.
- Use a code snippet you wrote. It is an issue you already tackled in the past and you coded something to address the issue. In this case, there is no problem in reusing your own code. You wrote it, you know it, know how it works, in which context it has been developed and which exact problem it solves.
- Code a solution by yourself. You can look for inspiration in others' code snippets but you should code the solution yourself. This way, you are sure to code something that adress your exact issue and you're improving your knowledge and skills. Plus, next time you'll be facing the problem, you'll be able to solve it in a few minutes.
It is often said that "good coders are lazy". This is true, but it's hard becoming a lazy coder and lazy coders are most of the time hard working people 
Web application implementation step 3: Framework vs Methodology
RT @thibauld Web application implementation step 3: Framework vs MethodologyNow you must be wondering why I'm so skeptical about frameworks... actually, I have nothing against frameworks in themselves (see my post on the subject), some are better than others but generally speaking, they are all useful in some way. No, the problem lies more in what people tend to expect from frameworks. Indeed, a lot of people expect that, because they're using a good framework, they'll be able to code efficiently and deliver a great web app quickly. Unfortunately, this is a wrong reasoning... your coding efficiency depends more on your methodology than on the framework you're using. Just as an illustration, coding version 1.0 of freelance business club required only 3 months, starting from scratch and without using any framework.
The ultimate rule for coding efficiently is the following: first think, then code. I will now explain in more details the methodology I personally use when building web applications. This methodology is summed up in the following schema:
Process listing
This is, by far, the most important step, the one that can save you an incredible amount of time! When you start a project, everybody has a gazillion ideas on the application features: "it must do this, that" "how about this ?" "It would be great if we could do that" "You know what would be great ? this!"... Don't get me wrong, having people brainstorming around the application potential features is great, even essential, and you should write down all these great ideas. But obviously, you won't be able to have all these features for version 1.0.
First, you should translate these features into processes. A process puts a feature into perspective. The question to ask is the following: "To which business process does this feature belong to ?". You'll be amazed eventually at how many proposed features don't belong to any real business process and are, thus, irrelevant to your application. In particular, developers (me included) often think about adding "cool" features from a technical point of view... but each "cool" feature, even the smallest ones, take time to code so it is very important to think about its real usefulness from the end user point of view. Thinking in terms of business processes rather than in term of features is a good method to separate relevant features from irrelevant ones.
At this point, don't forget the "backend" processes, that is, the ones required to run your web application. These backend processes are too often overlooked, which result in people finally spending a lot of energy to administrate their web app. And "no", PhpMyAdmin cannot be your only backend application.
Once you you've listed all your processes, the next step is to break them down in milestones. The most crucial processes will go in the first milestone, the less important ones in the next milestone.... ect... the least important ones going in the "later" pseudo-milestone. At the end of this step, you've got all your processes listed by milestone.
Now, before going any further, I encourage you to perform some research to see, for each process, if there's not an available application/api/webservice out there that you can leverage to help you implement this process faster and/or better. This can be a huge booster and might prevent you from reinventing the wheel (and probably a poorer one).
Maquetting / Wireframing / Prototyping
Now that you know which processes to implement for the next milestone, you need to "wireframe" it (in french we say "maquetter", I'm not sure of the english word for it). What is important in this step is to synchronize everybody on each screen which are to be develop. What's in a screen:
- General layout of the page
- Information displayed
- Links available
To build wireframes, I personally tend to use Impress (the PowerPoint like module of OpenOffice.org) but you can use anything you like, pen and paper are ok too... If you're a Windows user, a friend of mine uses Axure, a commercial tool dedicated to wireframing and prototyping. Once you've got all your screens ready, you may want to turn them into a real prototype to have your users validate how you will implement the business processes. On linux, I found that such a task can be easily realized with kimagemapeditor: just import screenshots of your screens and define the links between them using html image maps. You'll end up with a static website in which you can navigate easily.
Once this is done, you can have your graphical designer start working to turn these ugly wireframes into art!
URL listing
At this point, once you have all your screens and their relationships setup, it is time to decide the URL of each resource. It is very important to think about your URL strategy beforehand for several reasons. First, as you're building a web application, you cannot ignore Google and need to implement a google-friendly URL scheme. Then, thinking about your URLs help you switch mentally between your user requirements and how you will code it.
For example, in the code architecture I implemented, this step is key as I cannot code anything until I translated my URL scheme in a .htaccess (using the urlrewriting Apache module). This .htaccess determines how I organize my presentation handlers.
Database design
During the previous step, you probably began thinking about the 'objects' you are going to manipulate. It is now time to write it down by coming up with a complete and coherent database design. I cannot stress enough how much database design is key. If you fail here, you'll have to code a lot more than needed. You will not only have to paliate the inefficiencies of your database design by coding more, but you will also have to deal with poor application performance sooner than expected.
Does "referential integrity" sound familiar to you ? it should... Too few people realize how powerful a database like Mysql or Postgresql is. SQL being a declarative language, a lot of developers don't like it much but it's a pity... it can be really powerful and you can delegate _a lot_ of work to your database. Letting your database handle work saves you coding time, and will probably boost your application performance. Database design is a vast subject... If I have time, I'll probably write a post dedicated to it but meanwhile, I encourage you to read this great book : The Art of SQL.
Coding
Now you can begin coding Of course, I also have a methodology for coding. This is good practice to implement process by process (you can parallelize work depending on your team size). The most important thing being what the end user sees in the end, you should start for each process with the presentation layer, then continue with the business layer to finally end with the data layer. To ensure a good level of quality, once you finished coding a layer, validate it using dummy values from the n-1 layer (because the n-1 layer has not yet been implemented). If you follow this methodology, you'll be amazed at how fast you'll be able to develop your web application.
That's all for today, see you next post!
David Weinberger conference notes
RT @thibauld David Weinberger conference notesWhat is good with having events such as LeWeb taking in place in Paris, is that it offers great opportunities to meet people we're not used to see here. While I do not attend LeWeb, I had the opportunity today to attend a speech of David Weinberger about the Internet and how it affects and reinvents our lives.
It was not at all a technical speech but one of these speech I enjoy listening to, that help us put things into perspective. The only little problem was that David Weinberger's Macintosh crashed badly before the speech so he had to speak 'slideless'. This is why I thought it might be an interesting experience to publish the notes I took to see if what I got from his speech corresponds to what was effectively in the slides (I don't despair to see the slides one day ). So here are my notes of the speech. As I'm not a native english speaker, please note that I may have altered some of his thoughts.
First, he began by reminding us that the web has not been designed for anything in particular but only for "moving bits from one place to another" and that it is this designed-for-nothing-in-particular aspect that makes it so revolutionary and a perfect playground for innovation today.
The core purpose of the web being "only" to move bits resulted in 3 characteristics:
- Abundance. The consequence of digitalizing (transforming into bits) everything: videos, books, music, images and so on... The only limit being today's "insane" (I quote) copyright laws.
- Unalienation. The end of the 'broadcast' era, the content on the internet is our content, understand made by people like us. Everybody is a source of information.
- Connection. Hyperlinks defines the Web, they link content. In today's Internet, a lot of value has been put on top of these hyperlinks so that, through sites like social networks, it not only content that is being linked but people.
David then developed the implications of these 3 principles:
Unrealism has become realistic. Wikipedia, Linux or simply the web were all projects that people said to be utopist, unrealistic and bound to fail at the beginning. No managers, no hierarchy... this just could not work! But today they are realizations that will make the Renaissance look 'pale' in comparison.
Include everything paradigm. Whereas we used to use experts to sort / organize every piece of information, this is not the rule anymore. Today, the rule is to include everything by default as we are unable to anticipate what information will prove to be relevant to the users. Deciding what to include, what to exclude, how to organize, is a political decision that may prevent users to find what they're looking for. He illustrated his saying with with Sarah Palin: everybody was so surprised by her nomination that the most relevant piece of information we could find at that time was a page on her on Wikipedia. Had this information (her wikipedia page) been rationally analysed before her nomination, it would had been probably filtered out...
Superior collective knowledge. The Web provides the tool to leverage collective knowledge through user participation. David took an example of photos taken during of World War 2 that were held by the Library of Congress and that they decided to put on flickr (note: I found it). These photos are a good illustration of how users can significantly contribute to add valuable (or not) information that the Library of Congress experts themselves did not have (the photos were tagged more than flickr would allow).
Fading frontier between data and meta-data. Whereas we used to use meta-data to find what we are looking for (ex: find a book thanks to its ISBN), the frontier between data and meta-data is now fading. With today's abundance of information, you can now find items through data (ex: find a book because you remember its first sentence). The only remaining difference between data and meta-data being that meta-data is what you know and data is what you're looking for.
Authority by transparency. Institutional authority and credibility is slowly losing ground. Transparency is becoming the real way to establish authority. David took a relevant example using the traditional comparison between Wikipedia and Britannica. Coming back to the page of Sarah Palin, he said that the neutrality of this page is simply striking considering the extreme feelings (love or hate) that exist on Sarah Palin. But what is even more striking is how this neutrality was achieved by looking at the discussions that took place: the 2 camps after long fights and debates finally came to an agreement on a text that both sides think is neutral. Above all, everybody can see how this page was disputed and how it has been elaborated, something unthinkable with Britannica. Wikipedia, on the contrary to Britannica, takes its authority, legitimity, credibility from its transparency. In the Internet world, the more transparent, the more legitimate.
End of my notes! Clearly, it is very likely that the articulation of my notes is far less coherent than the original presentation but anyway.... I hope you got the substance-full marrow ("la substantifique moëlle" in french, I found the translation in Wikipedia, I hope it is correct and neutral 
).
Cheers,
Web application implementation step 2: build on solid foundations!
RT @thibauld Web application implementation step 2: build on solid foundations!In a previous post, I wrote about choosing the right tools to build your website. For Freelance Business Club, we chose to use the following stack: Linux (Ubuntu 8.04) / MySQL5 / Apache2 / PHP5 (with Smarty). One of the thing that influenced us in our choice was the study of Flickr.com's architecture. We knew that, provided we did things well, the chosen stack could perform well in a production environment and that it could scale.
But obviously, using the rights tools is necessary but not sufficient to build a great app. The next step is to setup your code architecture. A good code architecture / organization features:
- Reusability. You should not have to duplicate your code (unless you want to). It will make bug fixing and maintenance a lot easier.
- Flexibility. Modifying or enhancing something in your web application should be as easy as possible: adding new feautres, changing graphical chart, database engine, underlying hardware architecture... Having a great code architecture will help you make your web application evolve smoothly and quickly over time.
- SEO friendly. Unless you're working on a private web application, you'll have to deal with SEO, in particular with Google. This is why it is really important that your code architecture let you master your URLs so that it doesn't block you in your SEO strategy.
- Simplicity. It should be easy to dive into your code architecture. It should be easily comprehensible. This will help you boost the productivity of your employees and interns and will save you valuable time. Note: documentation is not the answer here. It's probably part of the answer but I tend to believe that the more obfuscated your code and architecture, the bigger the documentation...
- Performance. All the above is futile if your web application responsiveness is ridiculous. You should make sure that your code architecture does not structurally lower your application's performance. It will not prevent your developers to write bad code that will bloat your app but, at least, they will not be able to hide behing your architecture and you'll know who to blame
In short, a good code architecture should provide coding agility and application robustness. Now let's see concretely how to you achieve this result.
First, it is crucial to separate the various logic layers which compose a web application. The principle is dead simple: a layer only needs to know about its adjacent layers (above and below him). Here's a schema representing the different layers involved in a web application:
Client layer. This layer is located in the webbrowser itself. It displays to the user what has been transmitted by the Presentation layer. The only code that gets executed in this layer is Javascript.
Presentation layer. This layer's role is to:
- handle the requests coming from the client
- call the adequate business methods to fetch the results corresponding to the client request
- prepare the business variables for rendering
- build the output (consisting of HTML and CSS)
- pass the final result to the client layer
I'd like to emphasize that this is generally a good idea to separate this layer in 2 in order to separate the gathering and formatting of business variables from the final output building process. I personally like to use Smarty for this 'output building process'.
Business layer. This is the layer where real business specific code is implemented. All the application logic lies in here. Business functions are called by the Presentation layer. In most cases, they interact with the Data layer to fetch/update/insert/delete data. To minimize variable entropy, it is important at this stage that the values returned by the Business layer to the Presentation be normalized.
Data layer. All interactions with the database are concentrated in this layer. It isolates the Business code from the underlying database. All SQL queries are located in this layer.
Provided you organized your code in layers, you already made a huge leap forward in your attempt to build a maintainable web application. Layers help achieve reusability and flexibility. Now, to build a SEO friendly code architecture, you'll have to deal with Apache URL Rewriting module. No, you should NOT organize your files on the filesystem to match your URLs
Now, I know what you're going to tell me... given what I said, it's clear that one should try as much as possible to use a framework to build his web app. Indeed, frameworks help you structure your code well by giving you guidelines and conventions. Right but only partly true...The problem is that, even if code architecture is important, it's only (little) part of the problem of building a great web app. Too many people tend to rely on the framework, thinking that having the right framework will maximize their chance of building a successful web app. This would be great but it is unfortunately a wrong reasoning... Frameworks may help you code quicker and structure your code well but to build a great web application, all you need is a methodology. This will be the subject of my next post... stay tuned!
PS: My friend Arnaud started to blog (in french) about creating an identity for your web application.